Add logging during awskms auto-unseal #9794
Conversation
Adds debug and warn logging around AWS credential chain generation, specifically to help users debugging auto-unseal problems on AWS, by logging which role is being used in the case of a webidentity token. Adds a deferred call to flush the log output as well, to ensure logs are output in the event of an initialization failure.
| @@ -51,6 +55,8 @@ func (c *CredentialsConfig) GenerateCredentialChain() (*credentials.Credentials, | |||
| SecretAccessKey: c.SecretKey, | |||
| SessionToken: c.SessionToken, | |||
| }}) | |||
| c.Logger.Debug("added static credential provider", "AccessKey", c.AccessKey) | |||
There was a problem hiding this comment.
c.Logger could potentially be nil if none is passed into CredentialsConfig so these calls would panic. Let's wrap logger calls with a nil check.
builtin/credential/aws/cli.go
Outdated
| @@ -98,7 +98,7 @@ func (h *CLIHandler) Auth(c *api.Client, m map[string]string) (*api.Secret, erro | |||
| headerValue = "" | |||
| } | |||
|
|
|||
| creds, err := RetrieveCreds(m["aws_access_key_id"], m["aws_secret_access_key"], m["aws_security_token"]) | |||
| creds, err := RetrieveCreds(m["aws_access_key_id"], m["aws_secret_access_key"], m["aws_security_token"], hclog.Default()) | |||
There was a problem hiding this comment.
This does not derive any of the logging properties from the main command, so it might result in different format, log level, etc.
There was a problem hiding this comment.
How would we get the logging properties from the main command here? I haven't found a way to do that.
There was a problem hiding this comment.
The log would be from the client (result from a vault login command) for this particular case so it might be fine, though hclog.Default() logs up to Info only, so it would skip any Debug output.
There was a problem hiding this comment.
I'm not seeing anywhere else in the vault login command that verbosity levels are set, so in 2271d6c I added a log_level option to the aws cli login command, and used that to configure the logger passed to RetrieveCreds(). Or maybe it's preferable to just pass a null logger here?
Used to setup the logger for use in GenerateCredentialChain()
go.mod
Outdated
| @@ -58,7 +58,7 @@ require ( | |||
| github.com/hashicorp/go-cleanhttp v0.5.1 | |||
| github.com/hashicorp/go-gcp-common v0.6.0 | |||
| github.com/hashicorp/go-hclog v0.14.1 | |||
| github.com/hashicorp/go-kms-wrapping v0.5.12 | |||
| github.com/hashicorp/go-kms-wrapping v0.5.15 | |||
There was a problem hiding this comment.
We should be using v0.5.16 right?
There was a problem hiding this comment.
Yep, just fixed in e4f2e2b
Adds debug and warn logging around AWS credential chain generation, specifically to help users understand why auto-unseal isn't working on AWS, by logging which role is being used in the case of a webidentity token.
Also in the case of webidentity, this tests retrieving credentials, and logs a warning if it has trouble, in which case it falls through to using creds from the instance metadata, which may still fail, but with a warning logged at least users can tell why the webidentity auth failed.
Depends on hashicorp/go-kms-wrapping#22
Example logging with level=trace when auto-unseal fails:
before:
after: