Adding credential helper #300

frezbo · 2018-10-10T13:43:34Z

Enhances #299

Signed-off-by: Noel Georgi [email protected]

Signed-off-by: Noel Georgi <[email protected]>

frezbo · 2018-10-10T14:12:07Z

@mtibben @teeberg This should be good to go

teeberg · 2018-10-11T23:44:25Z

yeaaah, really cool :)) thanks for getting on that so quickly!!

teeberg · 2018-10-22T15:06:22Z

Any chance to get this merged?

mtibben · 2018-11-16T03:58:29Z

I was hoping for a review from @lox as he's most familiar with that code

lox · 2018-11-16T04:31:42Z

cli/exec.go

+	NoSession        bool
+}
+
+type AwsCredentialHelperData struct {


Could we get a code comment explaining what this is for future generations?

lox · 2018-11-16T04:32:47Z

cli/exec.go

@@ -53,6 +64,10 @@ func ConfigureExecCommand(app *kingpin.Application) {
 		Short('m').
 		StringVar(&input.MfaToken)

+	cmd.Flag("json", "AWS credential helper").


https://www.google.com/search?hl=en&q=%22aws%20credential%20helper%22 yields nothing of use, might there be a clearer way to describe this?

lox · 2018-11-16T04:34:14Z

This looks like super straight forward code, but would be good to help folks figure out it's there and how it works. Perhaps also a README addition @teeberg?

Signed-off-by: Noel Georgi <[email protected]>

frezbo · 2018-12-03T13:29:05Z

@lox I have updated the PR.

teeberg · 2018-12-19T14:51:18Z

Is there anything still missing here?

lox · 2019-01-03T22:46:49Z

I see a lot of whitespace changes in here, is that an issue with your gofmt @frezbo or is there an issue with the file?

lox · 2019-01-03T22:49:19Z

I'm going to merge this and give it a cleanup in master. Thnaks @frezbo!

pecigonzalo · 2019-01-24T13:26:18Z

Do you think we can add some more info about how to use this? like an example .aws/config? Im have tried a couple of variants and I cant get it to work.

frezbo · 2019-01-24T13:41:57Z

@pecigonzalo this:

[profile product]
region=us-east-1
credential_process=aws-vault exec -j product

teeberg · 2019-01-24T16:08:52Z

and then instead of manually calling aws-vault, you only need to make sure that you have AWS_PROFILE=product in your environment, e.g. with the above config, call

AWS_PROFILE=product aws sts get-caller-identity

pecigonzalo · 2019-01-24T20:57:44Z

Yeah, that is what I did, but I get access denied. I'll put my config in a moment, as I'm using roles and MFA, it might be that it gets in the way, altho I tried a few combos.

…

On Thu, Jan 24, 2019, 5:08 PM Jonas Trappenberg ***@***.*** wrote: and then instead of manually calling aws-vault, you only need to make sure that you have AWS_PROFILE=product in your environment, e.g. with the above config, call AWS_PROFILE=product aws sts get-caller-identity — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#300 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJcvieraCbfC7eCMa1MxwWUzsPdAEQA3ks5vGdqYgaJpZM4XVXk_> .

lox · 2019-01-24T21:31:46Z

Agreed some more docs would be really helpful for folks!

pecigonzalo · 2019-01-25T08:26:46Z

Here are my list of keys:

Profile                  Credentials              Sessions                 
=======                  ===========              ========                 
main.root                main.root                -                        
dev.admin               main.root                -

Here is my config:

[profile main.root]
region=eu-central-1

[profile dev.admin]
source_profile=main.root
region=eu-central-1
mfa_serial=arn:aws:iam::123123123:mfa/gonzalo.peci
role_arn=arn:aws:iam::123123123:role/main-root-admin
output=json

I tried doing:

[profile main.root]
region=eu-central-1
credential_process=aws-vault exec -j main.root

[profile dev.admin]
source_profile=main.root
region=eu-central-1
mfa_serial=arn:aws:iam::123123123:mfa/gonzalo.peci
role_arn=arn:aws:iam::123123123:role/main-root-admin
output=json

and many combinations of putting the credential_process on the dev.admin profile, removing/adding the role_arn or mfa_serial or source_profile etc.
At best I got an access denied from a aws --profile dev.admin s3 ls call. Meaning I got creds, but probably they did not assume the role or did not have MFA session token.

frezbo · 2019-01-25T08:57:22Z

@pecigonzalo I wonder if its the periods . in your profile names, can you try using -. BTW what does aws-vault exec -j main.root return, does that ask for any prompts?

StevenACoffman · 2019-01-31T18:54:28Z

I can successfully use the credential_process for a credentialed source role, but not to use that source role to assume another role. I created a shim bash script to supply my MFA to rule MFA out as an issue.

For instance I get this error:

$ AWS_PROFILE=engineer /usr/local/bin/aws sts get-caller-identity

The source_profile "personal" must specify either static credentials or an assume role configuration

This is with an .aws/config like:

[profile personal]
region = us-east-1
output = json
mfa_serial = arn:aws:iam::123456:mfa/scoffman
credential_process = /Users/scoffman/bin/aws-credential-helper.sh

[profile engineer]
role_arn = arn:aws:iam::123456:role/DevelopmentV1
source_profile = personal
mfa_serial = arn:aws:iam::123456:mfa/scoffman
region = us-east-1
output = json

The shim bash script is just this:

#!/bin/bash
export TOTP="$(2fa ${AWS_MFA_NAME:-personal})"
if [[ -n "${TOTP:-}" ]]
then
  aws-vault exec --mfa-token=${TOTP} -j personal
else
    echo "No MFA TOTP! 2fa did not find a MFA TOTP."
fi

AWS Support says:

Currently, for assuming a role that has a source_profile attribute using CLI, the source_profile has to have a static credential or it is a profile that is configured to use assume-role [1]. Thus, the credential from credential_process cannot be used since they are external process and considered floating variables.

[1] https://docs.aws.amazon.com/cli/latest/topic/config-vars.html

So it looks like in a single profile, you could use credential_process or source_profile but not both.

frezbo · 2019-02-01T03:45:05Z

@StevenACoffman yeh that makes sense, I mostly use credential_process with terraform where the same code deploy stuff to multiple accounts based on the profile name. For profiles that need MFA auth I first do aws-vault exec <profile> -- bash and type in the MFA, so that the actual tool using the profile does not prompt for MFA.

StevenACoffman · 2019-02-01T12:23:04Z

AWS support says:

I can confirm my previous response that for the sts assume-role API, the credential from source_profile cannot be referenced from a profile that uses credential_process since it is an external process. I will start a feature request with the service team and see if they can incorporate credential_process in assume-role in a future update.

After testing various ways and aws-vault, and since your bash script is an advanced version of aws sts get-session-token API, a straight forward workaround is to make use of the existing script to extract the AccessKeyId, SecretAccessKey, SessionToken string and export them to the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN respectively similar to this documentation [1]. With this you can use sts assume-role API directly without using the profile “engineer”.

[1] https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/

StevenACoffman · 2019-02-05T17:12:28Z

This appears to work:

$ AWS_PROFILE=helper /usr/local/bin/aws sts get-caller-identity

The source_profile personal must specify static credentials.
The assumed role profile engineer must specify a role_arn, and source_profile engineer
The credential_process profile helper must specify aws-vault profile engineer

This is with an .aws/config like:

[profile personal]
region = us-east-1
output = json
mfa_serial = arn:aws:iam::123456:mfa/scoffman

[profile engineer]
role_arn = arn:aws:iam::123456:role/DevelopmentV1
source_profile = personal
mfa_serial = arn:aws:iam::123456:mfa/scoffman
region = us-east-1
output = json

[profile helper]
credential_process = /Users/scoffman/bin/aws-credential-helper.sh
mfa_serial = arn:aws:iam::594813696195:mfa/scoffman
region = us-east-1
output = json

The aws-credential-helper.sh shim bash script is just this:

#!/bin/bash
export TOTP="$(2fa ${AWS_MFA_NAME:-engineer})"
if [[ -n "${TOTP:-}" ]]
then
  aws-vault exec --mfa-token=${TOTP} -j engineer
else
    echo "No MFA TOTP! 2fa did not find a MFA TOTP."
fi

I do not think the shim that supplies MFA is necessary, but haven't tested it.

pecigonzalo · 2019-02-05T18:23:17Z

BTW, I tried without the . but it did not work.
It make sense tho, as the aws-vault profile is also using config form the aws/config file, like which profile to assume etc. I guess we just need to document how/when to use this to avoid confusion.

StevenACoffman · 2019-02-05T19:42:45Z

@pecigonzalo
Try:

[profile main_root]
region=eu-central-1
credential_process=aws-vault exec -j main_root

[profile dev_admin] 
source_profile=main_root
region=eu-central-1
mfa_serial=arn:aws:iam::123123123:mfa/gonzalo.peci
role_arn=arn:aws:iam::123123123:role/main-root-admin 
output=json

[profile dev_admin_helper] 
region=eu-central-1
mfa_serial=arn:aws:iam::123123123:mfa/gonzalo.peci
output=json
credential_process=aws-vault exec -j dev_admin

Then test with:

$ AWS_PROFILE=dev_admin_helper /usr/local/bin/aws sts get-caller-identity

eikeon · 2019-02-15T19:24:15Z

@StevenACoffman That works for me except it's eating the prompt. For example if I hit enter when it hangs (I didn't see the prompt) then I see: "Error when retrieving credentials from custom-process: Enter token for arn:xxxxxxxxx: "

eikeon · 2019-02-15T19:40:37Z

Ah, I see there's a --prompt

aws-vault exec -j --prompt=osascript my_profile_name works for me

Enable aws-okta to be a invoked as an external process via the AWS CLI "credential_process" option. Adapted from 99designs/aws-vault#300 Reference: https://docs.aws.amazon.com/cli/latest/topic/config-vars.html#sourcing-credentials-from-external-processes

mhumeSF · 2019-06-28T18:29:03Z

So for this to work we need two profiles for every assume_role profile?

1 for assume_profile
1 for helper

[profile default]
credential_process=aws-vault exec -j default

[profile dev]
source_profile = default
role_arn = arn:aws:iam::222222222222:role/dev
mfa_serial = arn:aws:iam::111111111111:mfa/[email protected]

[profile dev_helper]
mfa_serial = arn:aws:iam::111111111111:mfa/[email protected]
credential_process=aws-vault exec -j --prompt=osascript dev

StevenACoffman · 2019-06-29T22:58:49Z

That's not how I have it working.
I thought aws-vault needs to have the credentials stored in a non-default profile.
e.g. aws-vault add source --keychain login and then have the config more like this:

[default]
credential_process = aws-vault exec -j dev
region = us-east-1
output = json
 
[profile source]
region = us-east-1
mfa_serial = arn:aws:iam::111111111111:mfa/me
 
[profile dev]
mfa_serial = arn:aws:iam::111111111111:mfa/me
region = us-east-1
role_arn = arn:aws:iam::111111111111:role/dev
source_profile = source

default depends on dev which depends on source.

lox · 2019-06-30T00:15:33Z

I thought aws-vault needs to have the credentials stored in a non-default profile.

Hmmm, interesting, I can't think of a reason why creds couldn't be stored in the default, that said, I've never done that. I have source profiles too for credentials.

glenjamin · 2019-08-06T09:14:07Z

Could this be added to the main documentation somewhere? This looks like just what I was looking for but I only found it via a colleague

mhumeSF · 2019-08-29T23:51:29Z

I was hung up on this so adding my notes here for anyone else.

Set AWS_SDK_LOAD_CONFIG=true in environment variables.
Static credentials are added via aws-vault add my-aws-account

# ~/.aws/config
[profile my-aws-account]

[profile _source_blue]
source_profile=my-aws-account
role_arn=arn:aws:iam::111111111111:role/blue
mfa_serial=arn:aws:iam::000000000000:mfa/[email protected]

[profile blue]
credential_process=aws-vault exec _source_blue --json

This allows one to switch profiles by just setting the AWS_PROFILE variable. And now I can add AWS_PROFILE=blue in a .envrc and when I cd into that directory I am prompted for keychain and mfa; but only if no session is available or expired.

mtibben · 2019-08-30T04:02:16Z

Yes absolutely that would be great @glenjamin, please feel free to submit a PR

Adding credential helper

d15847e

Signed-off-by: Noel Georgi <[email protected]>

frezbo changed the title ~~Adding credential helper~~ WIP: Adding credential helper Oct 10, 2018

Fix version data type and time format

7e03b6b

Signed-off-by: Noel Georgi <[email protected]>

frezbo changed the title ~~WIP: Adding credential helper~~ Adding credential helper Oct 10, 2018

mtibben requested a review from lox October 12, 2018 00:04

lox reviewed Nov 16, 2018

View reviewed changes

lox mentioned this pull request Nov 18, 2018

Using nested profiles / role assumptions #310

Closed

Update code with comments

4646fed

Signed-off-by: Noel Georgi <[email protected]>

lox merged commit ab99799 into 99designs:master Jan 3, 2019

frezbo deleted the feature/credential_helper branch January 4, 2019 03:02

FernandoMiguel mentioned this pull request Jul 10, 2019

Bind aws-vault to env vars possible? #393

Closed

teeberg mentioned this pull request Jul 21, 2019

Add command to print credentials as JSON for aws-cli's "credential_process" configuration #299

Closed

jychen7 mentioned this pull request Aug 17, 2019

Query about using aws-vault with terraform to manage resources in multiple aws accounts #296

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding credential helper #300

Adding credential helper #300

frezbo commented Oct 10, 2018

frezbo commented Oct 10, 2018

teeberg commented Oct 11, 2018

teeberg commented Oct 22, 2018

mtibben commented Nov 16, 2018

lox Nov 16, 2018

lox Nov 16, 2018

lox commented Nov 16, 2018

frezbo commented Dec 3, 2018

teeberg commented Dec 19, 2018

lox commented Jan 3, 2019

lox commented Jan 3, 2019

pecigonzalo commented Jan 24, 2019

frezbo commented Jan 24, 2019

teeberg commented Jan 24, 2019

pecigonzalo commented Jan 24, 2019 via email

lox commented Jan 24, 2019

pecigonzalo commented Jan 25, 2019 •

edited

Loading

frezbo commented Jan 25, 2019

StevenACoffman commented Jan 31, 2019 •

edited

Loading

frezbo commented Feb 1, 2019

StevenACoffman commented Feb 1, 2019

StevenACoffman commented Feb 5, 2019 •

edited

Loading

pecigonzalo commented Feb 5, 2019

StevenACoffman commented Feb 5, 2019 •

edited

Loading

eikeon commented Feb 15, 2019

eikeon commented Feb 15, 2019

mhumeSF commented Jun 28, 2019

StevenACoffman commented Jun 29, 2019

lox commented Jun 30, 2019

glenjamin commented Aug 6, 2019

mhumeSF commented Aug 29, 2019

mtibben commented Aug 30, 2019

Adding credential helper #300

Adding credential helper #300

Conversation

frezbo commented Oct 10, 2018

frezbo commented Oct 10, 2018

teeberg commented Oct 11, 2018

teeberg commented Oct 22, 2018

mtibben commented Nov 16, 2018

lox Nov 16, 2018

Choose a reason for hiding this comment

lox Nov 16, 2018

Choose a reason for hiding this comment

lox commented Nov 16, 2018

frezbo commented Dec 3, 2018

teeberg commented Dec 19, 2018

lox commented Jan 3, 2019

lox commented Jan 3, 2019

pecigonzalo commented Jan 24, 2019

frezbo commented Jan 24, 2019

teeberg commented Jan 24, 2019

pecigonzalo commented Jan 24, 2019 via email

lox commented Jan 24, 2019

pecigonzalo commented Jan 25, 2019 • edited Loading

frezbo commented Jan 25, 2019

StevenACoffman commented Jan 31, 2019 • edited Loading

frezbo commented Feb 1, 2019

StevenACoffman commented Feb 1, 2019

StevenACoffman commented Feb 5, 2019 • edited Loading

pecigonzalo commented Feb 5, 2019

StevenACoffman commented Feb 5, 2019 • edited Loading

eikeon commented Feb 15, 2019

eikeon commented Feb 15, 2019

mhumeSF commented Jun 28, 2019

StevenACoffman commented Jun 29, 2019

lox commented Jun 30, 2019

glenjamin commented Aug 6, 2019

mhumeSF commented Aug 29, 2019

mtibben commented Aug 30, 2019

pecigonzalo commented Jan 25, 2019 •

edited

Loading

StevenACoffman commented Jan 31, 2019 •

edited

Loading

StevenACoffman commented Feb 5, 2019 •

edited

Loading

StevenACoffman commented Feb 5, 2019 •

edited

Loading