Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fixes 31700] Add mTLS support for http backend by way of client cert & key, as well as enterprise cacert. #31699

Merged
merged 29 commits into from
Jan 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
a00a1ab
Add mTLS support for http backend by way of client cert & key, as wel…
Aug 27, 2022
8a7f4e9
Fix style.
Aug 27, 2022
eab391b
Skip cert validation to be sure error is related to missing client ce…
scr-oath Aug 27, 2022
cd46c94
Remove misplaced err check.
scr-oath Aug 27, 2022
0223e3c
Fix the size of test using http backend.
scr-oath Aug 27, 2022
ab1dd2b
Just for correctness, include all certs in the pem encoded cert - som…
scr-oath Aug 27, 2022
985bc53
Adjusted names as recommended in PR comments.
scr-oath Oct 13, 2022
172f958
Adjusted names to be full-length and more descriptive.
scr-oath Oct 13, 2022
ce5d217
Added full-fledged testing with mTLS http server
scr-oath Dec 15, 2022
33031da
Fix goimports.
scr-oath Oct 14, 2022
c61a9ed
Fix the names of the backend config.
scr-oath Oct 14, 2022
fb7e570
Exclusive lock for write and delete.
scr-oath Oct 14, 2022
cfd928a
Revert "Fix goimports."
scr-oath Oct 14, 2022
bbc0478
goimports just for server test.
scr-oath Oct 14, 2022
6208901
Added the go:generation for the mock.
scr-oath Oct 14, 2022
6328481
Move the TLS configuration out to make it more readable - don't repla…
scr-oath Dec 5, 2022
2ab36d6
Just switch the client/data params - felt more natural this way.
scr-oath Dec 5, 2022
6f05da8
Update internal/backend/remote-state/http/backend.go
scr-oath Dec 8, 2022
441edac
Update internal/backend/remote-state/http/testdata/gencerts.sh
scr-oath Dec 8, 2022
91baacc
Update internal/backend/remote-state/http/backend.go
scr-oath Dec 8, 2022
666d7fc
Update internal/backend/remote-state/http/backend.go
scr-oath Dec 8, 2022
6d6bf19
Update internal/backend/remote-state/http/backend.go
scr-oath Dec 8, 2022
f8a3b8b
Update internal/backend/remote-state/http/backend.go
scr-oath Dec 8, 2022
481ad34
the location of the file name is not sensitive.
scr-oath Dec 8, 2022
13bbe7c
Added error if only one of client_certificate_pem and client_private_…
scr-oath Dec 8, 2022
d4218d1
Remove testify from test cases; use t.Error* for assert and t.Fatal* …
scr-oath Dec 15, 2022
0f7b3c7
Fixed import consistency
scr-oath Dec 15, 2022
96dc440
Just use default openssl.
scr-oath Dec 16, 2022
ba0844a
Since file(...) is so trivial to use, changed the client cert, key, a…
scr-oath Dec 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ require (
github.com/shopspring/decimal v1.3.1 // indirect
github.com/spf13/cast v1.5.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/objx v0.1.1 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/ulikunitz/xz v0.5.8 // indirect
github.com/vmihailenco/msgpack/v4 v4.3.12 // indirect
github.com/vmihailenco/tagparser v0.1.1 // indirect
Expand Down
7 changes: 5 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -594,14 +594,17 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.194/go.mod h1:7sCQWVkxcsR38nffDW057DRGk8mUjK1Ing/EFOK8s8Y=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.232 h1:kwsWbh4rEw42ZDe9/812ebhbwNZxlQyZ2sTmxBOKhN4=
Expand Down
79 changes: 68 additions & 11 deletions internal/backend/remote-state/http/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,16 @@ package http
import (
"context"
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"log"
"net/http"
"net/url"
"time"

"github.com/hashicorp/go-cleanhttp"
"github.com/hashicorp/go-retryablehttp"

"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/legacy/helper/schema"
"github.com/hashicorp/terraform/internal/logging"
Expand Down Expand Up @@ -93,6 +95,24 @@ func New() backend.Backend {
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_RETRY_WAIT_MAX", 30),
Description: "The maximum time in seconds to wait between HTTP request attempts.",
},
"client_ca_certificate_pem": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_CLIENT_CA_CERTIFICATE_PEM", ""),
Description: "A PEM-encoded CA certificate chain used by the client to verify server certificates during TLS authentication.",
},
"client_certificate_pem": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_CLIENT_CERTIFICATE_PEM", ""),
Description: "A PEM-encoded certificate used by the server to verify the client during mutual TLS (mTLS) authentication.",
},
"client_private_key_pem": &schema.Schema{
Type: schema.TypeString,
Optional: true,
kmoe marked this conversation as resolved.
Show resolved Hide resolved
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_CLIENT_PRIVATE_KEY_PEM", ""),
Description: "A PEM-encoded private key, required if client_certificate_pem is specified.",
},
},
}

Expand All @@ -107,6 +127,50 @@ type Backend struct {
client *httpClient
}

// configureTLS configures TLS when needed; if there are no conditions requiring TLS, no change is made.
func (b *Backend) configureTLS(client *retryablehttp.Client, data *schema.ResourceData) error {
// If there are no conditions needing to configure TLS, leave the client untouched
skipCertVerification := data.Get("skip_cert_verification").(bool)
clientCACertificatePem := data.Get("client_ca_certificate_pem").(string)
clientCertificatePem := data.Get("client_certificate_pem").(string)
clientPrivateKeyPem := data.Get("client_private_key_pem").(string)
if !skipCertVerification && clientCACertificatePem == "" && clientCertificatePem == "" && clientPrivateKeyPem == "" {
return nil
}
if clientCertificatePem != "" && clientPrivateKeyPem == "" {
return fmt.Errorf("client_certificate_pem is set but client_private_key_pem is not")
}
if clientPrivateKeyPem != "" && clientCertificatePem == "" {
return fmt.Errorf("client_private_key_pem is set but client_certificate_pem is not")
}

// TLS configuration is needed; create an object and configure it
var tlsConfig tls.Config
client.HTTPClient.Transport.(*http.Transport).TLSClientConfig = &tlsConfig

if skipCertVerification {
// ignores TLS verification
tlsConfig.InsecureSkipVerify = true
}
if clientCACertificatePem != "" {
// trust servers based on a CA
tlsConfig.RootCAs = x509.NewCertPool()
if !tlsConfig.RootCAs.AppendCertsFromPEM([]byte(clientCACertificatePem)) {
return errors.New("failed to append certs")
}
}
if clientCertificatePem != "" && clientPrivateKeyPem != "" {
scr-oath marked this conversation as resolved.
Show resolved Hide resolved
// attach a client certificate to the TLS handshake (aka mTLS)
certificate, err := tls.X509KeyPair([]byte(clientCertificatePem), []byte(clientPrivateKeyPem))
if err != nil {
return fmt.Errorf("cannot load client certificate: %w", err)
}
tlsConfig.Certificates = []tls.Certificate{certificate}
}

return nil
}

func (b *Backend) configure(ctx context.Context) error {
data := schema.FromContextBackendConfig(ctx)

Expand Down Expand Up @@ -149,21 +213,14 @@ func (b *Backend) configure(ctx context.Context) error {

unlockMethod := data.Get("unlock_method").(string)

client := cleanhttp.DefaultPooledClient()

if data.Get("skip_cert_verification").(bool) {
// ignores TLS verification
client.Transport.(*http.Transport).TLSClientConfig = &tls.Config{
InsecureSkipVerify: true,
}
}

rClient := retryablehttp.NewClient()
rClient.HTTPClient = client
rClient.RetryMax = data.Get("retry_max").(int)
rClient.RetryWaitMin = time.Duration(data.Get("retry_wait_min").(int)) * time.Second
rClient.RetryWaitMax = time.Duration(data.Get("retry_wait_max").(int)) * time.Second
rClient.Logger = log.New(logging.LogOutput(), "", log.Flags())
if err = b.configureTLS(rClient, data); err != nil {
return err
}

b.client = &httpClient{
URL: updateURL,
Expand Down
95 changes: 95 additions & 0 deletions internal/backend/remote-state/http/mock_server_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading