Feature: aws-vault export --credentials

[mtibben]

I wonder if we should create a aws-vault export --stored-credentials PROFILE , a new flag that will only output the master credentials in the vault and never attempt to create temporary credentials.

This seems to be the use-case of #1176 and #1180 which have been raised after the introduction of credential_process as a way to source credentials in v7.

Would this help avoid creating extra profiles just for credentials?

Thoughts @ArjunDandagi @dgholz @Ketouem @jweyrich @olfway

[mtibben]

hmmm actually not sure that works

[olfway]

May be the old command "aws-vault exec --json" could be modified to ignore credential_process - so the backward compatibility would be restored?

I understand that it's deprecated, but it will give us a time to adopt our configs, upgrade aws-vault, etc

[olfway]

I wonder if we should create a aws-vault export --stored-credentials PROFILE , a new flag that will only output the master credentials in the vault and never attempt to create temporary credentials.

In my case I do want aws-vault to ask for mfa and create temporary credentials
I think the flag should be something like aws-vault export --no-credential-process

Or another way could be to add environment variables to the new credential process like AWS_VAULT_CONFIG_FILE, AWS_VAULT_PROFILE_NAME, AWS_VAULT_CREDENTIAL_PROCESS and then avoid going into infinite credential_process loop based on those variables

[mtibben]

I think the flag should be something like aws-vault export --no-credential-process

Yeah it's a decent idea

[dgholz]

That sounds good, but that's also what I thought --no-session did. Obviously it's still following credential_process, I reckon it should not (and be deprecated in favour of whichever suggestion ends up being selected from this issue).

[mtibben]

--no-session specifically disables STS GetSessionToken which aws-vault normally utilises to share MFA creds between profiles. So that's a bit different.

The use-case here is using aws-vault as a cache, which requires aws-vault to ignore the credential_process, but use the rest of the profile config when creating the credentials.

So I'm thinking 2 new flags for export

--ignore-credential-process to ignore credential_process

--credential-process-cache an alias for --format=json --ignore-credential-process

Any feedback on that?

[olfway]

For me it seems enough to have only one flag --ignore-credential-process / --no-credential-process

The --credential-process-cache looks not obvious to me and I guess it will be always used in config, so it's not too difficult to write two flags instead of this one

[olfway]

But please, consider changing aws-vault exec --json to be an alias for aws-vault export --ignore-credential-process --format=json

This will restore backward compatibility and will give us a time to upgrade

[jweyrich]

I'm not sure I followed your idea. But I'll try to explain my scenario:

This is my $HOME/.aws/config:

[profile company-prod]
region=sa-east-1
credential_process=aws-vault export --no-session --format=json company-prod

The profile was configured via aws-vault add company-prod. I use the macOS keychain.

I used to run aws --profile=company-prod s3 ls, but since v7 this no longer works.
I was originally using credential_process=aws-vault exec company-prod --json, then I updated the command to use export hoping it would fix it for v7.
Am I doing something wrong?

I'm running macOS 13.1, and this is the --debug output:

2023/03/07 11:34:30 aws-vault v7.0.1
2023/03/07 11:34:30 Using prompt driver: terminal
2023/03/07 11:34:30 Loading config file /Users/myuser/.aws/config
2023/03/07 11:34:30 Parsing config file /Users/myuser/.aws/config
2023/03/07 11:34:30 [keyring] Considering backends: [keychain]
2023/03/07 11:34:30 profile company-prod: using credential process
2023/03/07 11:34:30 [keyring] Querying keychain for service="aws-vault", keychain="aws-vault.keychain"
2023/03/07 11:34:30 [keyring] Found 6 results
2023/03/07 11:34:30 [keyring] Querying keychain for service="aws-vault", keychain="aws-vault.keychain"
2023/03/07 11:34:30 [keyring] Found 6 results
2023/03/07 11:34:30 [keyring] Querying keychain for service="aws-vault", account="credential_process,REDACTED,,-62135596800", keychain="aws-vault.keychain"
2023/03/07 11:34:30 [keyring] No results found
<... wait for various minutes>
aws-vault: error: exec: Failed to get credentials for company-prod: running command "aws-vault export --no-session --format=json company-prod": exit status 1

[dgholz]

--no-session specifically disables STS GetSessionToken which aws-vault normally utilises to share MFA creds between profiles. So that's a bit different.

I don't see the difference: skipping the STS calls means the original AWS credentials are used. And for MFA creds, I think this means the actual AWS credentials that may be protected with MFA? I think that matches what you proposed in the original post.

edit: I read the code and now it's clear to me what you meant.

I still think, whatever solution you decide on, that --no-session should also behave similarly (to ignore credential_process in the AWS config). That's to support using aws-vault from credential_process & avoid it calling itself.

edit: thinking about it more, this would break @jmczerk's use-case. I guess there needs to be a option that can be passed to aws-vault for when it's being configured to be called as a credential_process. I like @olfway's suggested --no-credential-process for that!

The use-case here is using aws-vault as a cache

This is the new behaviour from #1087? I feel like I want aws-vault to handle securely accessing my AWS credentials, assuming roles from profiles, and caching the session credentials. I see how the last part could be used for caching non-session credentials from external processes, but I think that makes more sense as an extension of aws-vault add.

[jmczerk]

Apologies for introducing this issue.

Given my use case seems to be less common than those impacted, would it make sense to take the approach of #1117 where the the aws-vault external credential provider uses an aws-vault-specific config option (i.e. aws_vault_credential_process in the referenced PR) rather than shadowing the AWS CLI's credential_process directly?

[mtibben]

The core problem here is that there are a few different modes aws-vault is used in. And unfortunately many configs have been set up ambiguously by specifying multiple credential sources.

I have collected the use-cases that I understand aws-vault is being used in.

Use-case 1: aws-vault is the executor and provides the environment

One group of people (myself and my team included) uses aws-vault exclusively as a runner, where aws-vault provides the environment and runs a command.

[profile my_profile_master]
# master credentials stored in aws-vault

[profile my_profile_role]
source_profile=my_profile_master
role_arn=xxx

aws-vault exec my_profile_master ./my-command   # success, uses sts session generated by aws-vault
aws-vault exec my_profile_role ./my-command     # success, uses role creds generated by aws-vault

AWS_PROFILE=my_profile_master ./my-command      # Not expected to be functional
AWS_PROFILE=my_profile_role ./my-command        # Not expected to be functional

In this scenario, the profile name and aws config is used exclusively by aws-vault, which provides the environment for the command to run in.

This is a very unix-y and 12-factor approach. It's the original (and I consider the primary) use-case of aws-vault - it's why its aws-vault exec exists.

Use-case 2: aws-vault is a "master credentials vault" for AWS SDK

A second group of people use aws-vault in credential_process to provide master creds in the config. This is more in-line with the AWS SDK way of approaching the problem via credential_process and AWS_PROFILE

[profile my_profile_master]
credential_process = aws-vault6 exec --json --no-session my_profile_master  # aws-vault7 export --format=json --no-session my_profile_master

[profile my_profile_role]
source_profile=my_profile_master
role_arn=xxx

aws-vault exec my_profile_master ./my-command   # Not expected to be functional
aws-vault exec my_profile_role ./my-command     # Not expected to be functional

AWS_PROFILE=my_profile_master ./my-command      # v6: success (uses aws-vault master creds)
                                                # v7: broken
AWS_PROFILE==my_profile_role ./my-command       # v6: success (uses aws-vault master creds + SDK role)
                                                # v7: broken

The problem here is that my_profile_master profile is used in 2 different contexts

1. AWS SDK uses it to execute credential_process
2. aws-vault uses it to work out how to create credentials (and now in v7 executes credential_process as part of that)

Use-case 3: aws-vault is a "MFA session cache" for AWS SDK

Very similar to Use-case 2, another group of people want aws-vault to cache STS MFA credentials between profiles. This means they are not forced to re-authenticate with MFA every time they switch profiles.

[profile my_profile_master]
mfa_serial=mmm
credential_process = aws-vault6 exec --json my_profile_master  # aws-vault7 export --format=json my_profile_master

[profile my_profile_role]
source_profile=my_profile_master
mfa_serial=mmm
role_arn=xxx1

[profile my_profile_role2]
source_profile=my_profile_master
mfa_serial=mmm
role_arn=xxx2

aws-vault exec my_profile_master ./my-command   # Not expected to be functional
aws-vault exec my_profile_role ./my-command     # Not expected to be functional

AWS_PROFILE=my_profile_master ./my-command      # v6: success (uses aws-vault session)
                                                # v7: broken
AWS_PROFILE=my_profile_role ./my-command        # v6: success (uses aws-vault session + SDK role)
                                                # v7: broken

This config has the same problem as Use-case 2, but there is also an additional point of confusion. The confusion here is that the user has set master creds via aws-vault add my_profile_master, but expecting credential_process = aws-vault export my_profile_master to return a session token. This requires aws-vault to have access to the config file in order to read config like the mfa_serial.

Use-case 4: aws-vault caches alternative credential sources like sso_start_url, etc

aws-vault caches credentials from alternative credential sources like sso_start_url, web_identity_token_process. In #1087, this was extended to support credential_process as well.

[profile my_profile_using_sso]
sso_start_url = https://mycompany.awsapps.com/start

[profile my_profile_using_process]
credential_process = my-custom-creds-cmd

aws-vault exec my_profile_using_sso ./my-command       # success, uses aws-vault caching
aws-vault exec my_profile_using_process ./my-command   # success, uses aws-vault caching

AWS_PROFILE=my_profile_using_sso ./my-command          # success, no caching
AWS_PROFILE=my_profile_using_process ./my-command      # success, no caching

Solutions

There are a few possible solutions to the problems identified in use-case 2 and 3.

Solution 1. Use a separate profile name for the master creds

Use-case 2:

[profile my_profile_vault]
# master credentials stored via 'aws-vault add my_profile_vault'

[profile my_profile_master]
# get master credentials
credential_process = aws-vault7 export --format=json --no-session my_profile_vault

[profile my_profile_role]
# master creds sourced, role assumed
source_profile=my_profile_master
role_arn=xxx
mfa_serial=mmm

Use-case 3:

[profile my_profile_vault]
# master credentials stored via 'aws-vault add my_profile_vault'
mfa_serial=mmm

[profile my_profile_session]
# get STS session
credential_process = aws-vault7 export --format=json my_profile_vault
mfa_serial=mmm

[profile my_profile_role]
# session sourced, role assumed
source_profile=my_profile_session
role_arn=xxx
mfa_serial=mmm

Pros:

• This solution is functional right now

Cons:

• requires a separate profile name for the master creds, so some folks need to update their configs. But that's why aws-vault v7 is a major version bump.

Solution 2. Tell aws-vault to output master credentials only with --credentials

[profile my_profile_master]
mfa_serial=mmm
credential_process = aws-vault7 export --credentials --format=json my_profile_master

Cons:

• This addresses use-case 2, but doesn't address use-case 3
• Confusing when used in conjunction with --no-session

Solution 3. Tell aws-vault to not load config with --no-config

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --no-config --format=json my_profile

This is an interesting idea because it cuts through to the root cause of the problem - both AWS SDK and aws-vault are using the config file. If the config file is actually not required by aws-vault, then it can be ignored completely. But then if it is required, we're back to square 1

Pros:

• Simple to understand - config files are not loaded at all, only the vaulting and STS session functionality of aws-vault is used

Cons:

• MFA config won't be loaded for use-case 3
• What other use-cases would this break?

Solution 4. Tell aws-vault to ignore credential_process in the active profile with --ignore-credential-process

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --ignore-credential-process --format=json my_profile

Pros:

• surgically targets the problem

Cons:

• feels arbitrary to have a flag that only applies to one config in one profile

Solution 5. Tell aws-vault to disable credential_process in all profiles with --disable-credential-process

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --disable-credential-process --format=json my_profile

Pros:

• generally targets the problem

Cons:

• untargeted, i'm sure someone will come up with a use-case where this is not desirable

Solution 6. Change --no-session to mean "only return master credentials" instead of "don't use STS sessions"

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --no-session --format=json my_profile

Cons:

• This addresses use-case 2, but doesn't address use-case 3

Solution 7. Add a aws_vault_ignore_credential_process to config

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --format=json my_profile
aws_vault_ignore_credential_process=true

Pros:

• gives aws-vault the context to ignore the credential_process in the active profile

Cons:

• some users need to update their configs (but that's why it's a major version bump)

Solution 8. aws-vault ignores credential_process and only recognises aws_vault_credential_process

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --format=json my_profile

[profile my_profile_using_process]
aws_vault_credential_process = my-custom-creds-cmd

Pros:

• keeps existing configs working

Cons:

• breaks ability to execute AWS_PROFILE=my_profile_using_sso ./my-command in Use-case 4
• aws-vault doesn't support credential_process

Solution 9. Preference stored credentials over other credential sources

As suggested by @dgholz, aws-vault could prefer stored credentials over other credential sources. This would mean that if you have a profile with both stored credentials and credential_process set, aws-vault would use the stored credentials instead of executing the credential_process.

We would likely want to log the conflict and the fact that the stored credentials are being used.

Pros

• keeps existing configs working
• matches AWS SDK behaviour (which prefers the shared credentials file over credential_process)

---

Over to you. Have I missed any Use-cases? Are there any other solutions? What do you think of my commentary and the solutions I've proposed?

[olfway]

I'm using aws-vault similar to case 3 (mfa session cache) and actually v6 works for both aws-vault exec profile ./mycmd.sh and AWS_PROFILE=profile ./mycmd.sh

I vote for solution 4 (--ignore-credential-process) - it fixes problem, will allow to use aws-vault exec or mycmd directly and no need to add extra profiles

I prefer it more than solution 7 (aws_vault_ignore_credential_process=true) because command line flag is more flexible – I can add it to credential_process or use it while running aws-vault export directly from my shell or some other script

[Ketouem]

Regardless of the future chose solution, please note that this new flag should be documented in (https://github.com/99designs/aws-vault/blob/master/USAGE.md#using-credential_process)[the USAGE documentation]. I've stumbled upon the "initial bug" by actually following it and others may fall into the same trap as I did.

[dgholz]

@mtibben thanks for gathering these use cases. I actually have another use case: combined 1 & 2 & 3:

aws-vault exec my_profile_master ./my-command   # success, uses sts session generated by aws-vault
aws-vault exec my_profile_role ./my-command     # success, uses role creds generated by aws-vault

AWS_PROFILE=my_profile_master ./my-command      # success (uses sts session generated by aws-vault)
AWS_PROFILE=my_profile_role ./my-command        # success (uses sts session generated by aws-vault + SDK role)

which worked fine in v6, and means I get prompted once for my MFA token while the session is still active, regardless of which tool I use.

My proposal for a solution is: don't call credential_process if there's stored credentials that match the profile. I've hacked it together in dgholz#1. Do you think this is worth continuing?

[mtibben]

another use case: combined 1 & 2 & 3

Sorry @dgholz I don't exactly understand what that means. Can you provide a simple-as-possible example of the config, commands, and expected behaviour?

My proposal for a solution is: don't call credential_process if there's stored credentials that match the profile. I've hacked it together in dgholz#1. Do you think this is worth continuing?

🤔 Simple and effective. It might just work. Great idea @dgholz. I'll add it to the list

[mtibben]

A pre-release fix has been released v7.0.2-beta2 release with the Solution 9 fix (#1183) suggested by @dgholz.

Please test it and provide any feedback to ensure that this is doing the right thing

[mtibben]

Fixed in #1183