Bump AWS SDK to version v1.37.7 to support AWS SSO #1537

z0mbix · 2021-02-09T14:16:15Z

I've bumped the SDK version and ran go mod tidy. The only testing I've done is run make build and run a terragrunt plan/apply in a few modules, but seems to work as expected.

It relates to #1129

thapabishwa · 2021-02-10T03:05:42Z

Hi All,
I ran terragrunt plan/apply against the PR. However I encountered following error.

Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): SSOProviderInvalidToken: the SSO session has expired or is invalid
caused by: expected RFC3339 timestamp: parsing time "2021-02-10T11:02:51UTC" as "2006-01-02T15:04:05Z07:00": cannot parse "UTC" as "Z07:00"

brikis98

Thanks for the PR! I'll kick off tests now to see if it works with no other code changes.

That said, how do we know this now works with SSO? Are there no other changes needed to support that?

z0mbix · 2021-02-10T22:15:04Z

You need to login first with aws sso login --profile something as described here:

https://aws.amazon.com/blogs/developer/aws-sso-support-in-the-aws-sdk-for-go/

I've been using this build all day with no problems.

z0mbix · 2021-02-10T22:18:10Z

Also, there was an issue in the terraform repo where if you have a credential_process configured for getting the SSO credentials, then this can and probably will cause an error, so make sure you remove that.

thapabishwa

I did use the sso command to begin with. And also removed the credential_process. Still have the issue.

~ >>> aws sso login --profile MyProfile
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.us-west-2.amazonaws.com/

Then enter the code:

<CODE-HERE>
Opening in existing browser session.
Successully logged into Start URL: https://<url>.awsapps.com/start#/
~ >>>  terragrunt apply  --terragrunt-tfpath terraform0.12
ERRO[0000] Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): SSOProviderInvalidToken: the SSO session has expired or is invalid
caused by: expected RFC3339 timestamp: parsing time "2021-02-11T13:53:59UTC" as "2006-01-02T15:04:05Z07:00": cannot parse "UTC" as "Z07:00" 
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1 
~>>> cat ~/.aws/config                                                                       
[profile MyProfile]
sso_start_url = https://<url>.awsapps.com/start#/
sso_region = us-west-2
sso_account_id = <account>
sso_role_name = <Role-Name>
region = us-east-2
output = json

thapabishwa · 2021-02-11T06:01:08Z

I did use the sso command to begin with. And also removed the credential_process. Still have the issue.

~ >>> aws sso login --profile MyProfile
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.us-west-2.amazonaws.com/

Then enter the code:

<CODE-HERE>
Opening in existing browser session.
Successully logged into Start URL: https://<url>.awsapps.com/start#/
~ >>>  terragrunt apply  --terragrunt-tfpath terraform0.12
ERRO[0000] Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): SSOProviderInvalidToken: the SSO session has expired or is invalid
caused by: expected RFC3339 timestamp: parsing time "2021-02-11T13:53:59UTC" as "2006-01-02T15:04:05Z07:00": cannot parse "UTC" as "Z07:00" 
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1 
~>>> cat ~/.aws/config                                                                       
[profile MyProfile]
sso_start_url = https://<url>.awsapps.com/start#/
sso_region = us-west-2
sso_account_id = <account>
sso_role_name = <Role-Name>
region = us-east-2
output = json

Things go all the way back to sdk.
aws/aws-sdk-go#3324
golang/go#20555

brikis98 · 2021-02-11T10:19:12Z

You need to login first with aws sso login --profile something as described here:

https://aws.amazon.com/blogs/developer/aws-sso-support-in-the-aws-sdk-for-go/

I've been using this build all day with no problems.

Hm, a few questions:

That blog post you linked to says that we must enable SharedConfigState: session.SharedConfigEnable in the Go code. Do we need to do that in Terragrunt then?
How does aws sso login work if you need to switch across several accounts frequently? E.g., I want to run one command in dev, then another in stage, then back to dev, and so on. Do you only have one active login on your entire computer at a time? Or if you do aws sso login --profile <foo> and then aws sso login --profile <bar>, then you can actively use foo or bar profiles until those creds expire?

z0mbix · 2021-02-11T14:53:56Z

You need to login first with aws sso login --profile something as described here:
https://aws.amazon.com/blogs/developer/aws-sso-support-in-the-aws-sdk-for-go/
I've been using this build all day with no problems.

Hm, a few questions:

That blog post you linked to says that we must enable SharedConfigState: session.SharedConfigEnable in the Go code. Do we need to do that in Terragrunt then?

How does aws sso login work if you need to switch across several accounts frequently? E.g., I want to run one command in dev, then another in stage, then back to dev, and so on. Do you only have one active login on your entire computer at a time? Or if you do aws sso login --profile <foo> and then aws sso login --profile <bar>, then you can actively use foo or bar profiles until those creds expire?

You only need to run aws sso login once, then you are authenticated against all profiles, so switching is as simple as setting the correct profile, so this kind of thing works:

$ aws sso login --nonprod
$ aws s3 ls --profile nonprod
$ aws s3 ls --profile prod

I don't believe any further changes are needed, but it's hard for me to test this change doesn't break current behaviour.

brikis98 · 2021-02-12T20:03:53Z

You only need to run aws sso login once, then you are authenticated against all profiles,

Ohhh, interesting. So it must be getting some sort of temp creds from SSO that then allow it to assume any role you need?

I don't believe any further changes are needed, but it's hard for me to test this change doesn't break current behaviour.

Hm, yea, it's tough. The only item I'm left scratching my head about is why the blog post recommends adding that SharedConfigState: session.SharedConfigEnable if it's not required? Does Terragrunt still work for you if you add that setting?

sylr · 2021-02-15T18:23:21Z

@z0mbix can you share how your ~/.aws/config and ~/.aws/credentials look like because I myself can't login in all the SSO accounts at once:

$ aws sso login
Missing the following required SSO configuration values: sso_start_url, sso_region, sso_role_name, sso_account_id. To make sure this profile is properly configured to use SSO, please run: aws configure sso

z0mbix · 2021-02-16T16:53:51Z

[profile nonprod]
region = eu-west-1
output = json
sso_start_url = https://company.awsapps.com/start
sso_region = eu-west-1
sso_account_id = 1111111111111
sso_role_name = NonProdAdmin

[profile prod]
region = eu-west-1
output = json
sso_start_url = https://company.awsapps.com/start
sso_region = eu-west-1
sso_account_id = 2222222222222
sso_role_name = ProdAdmin

I only need to run aws sso login --profile nonprod which gives me access to both accounts.

karlpokus · 2021-02-19T14:46:17Z

Hm, yea, it's tough. The only item I'm left scratching my head about is why the blog post recommends adding that SharedConfigState: session.SharedConfigEnable if it's not required? Does Terragrunt still work for you if you add that setting?

I bumped the aws-sdk-go to v1.37.14 today to use sso credentials for a cli I'm making and I can only confirm that yes SharedConfigState is needed. I think .aws/config is actually called the shared config.

sess := session.Must(session.NewSessionWithOptions(session.Options{
    SharedConfigState: session.SharedConfigEnable,
    Profile:           <profileName>,
}))

and here's the readme from the pr https://github.com/aws/aws-sdk-go/pull/3755/files

By default NewSession will only load credentials from the shared credentials file (.aws/credentials). If the AWS_SDK_LOAD_CONFIG environment variable is set to a truthy value the Session will be created from the configuration values from the shared config (.aws/config) and shared credentials (.aws/credentials) files. Using the NewSessionWithOptions with SharedConfigState set to SharedConfigEnable will create the session as if the AWS_SDK_LOAD_CONFIG environment variable was set.

https://docs.aws.amazon.com/sdk-for-go/api/aws/session/

brikis98 · 2021-02-22T15:50:12Z

@z0mbix per @karlpokus' comment above, would you be up for updating the PR to set SharedConfigState: session.SharedConfigEnable?

UrosCvijan · 2021-02-26T21:23:17Z

When this is merged, does it mean that terragrunt will support SSO credentials from ~/.aws/config without any other modifications etc.. ? Main reason I am interested is cause I want to use my SSO credentials and to jump between accounts/profiles during the same terragrunt run.

oscarmorasu · 2021-03-03T18:38:36Z

That's correct @UrosCvijan , no more tricks or workarounds or external tools, the native SSO credentials in the GO SDK would take care of everything. All you have to do is log in with the aws sso login command, and have all your profiles for the different accounts setup in your ~/.aws/config file. The ~/.aws/credentials file would be empty

@z0mbix,

Is there a way we can help to wrap this up and get this PR merged?

swordfish444 · 2021-03-10T23:33:32Z

As temporary workaround, I was able to do the following:

[profile my-profile]
sso_start_url = xxxxxxx
sso_region = us-west-2
sso_account_id = xxxxxxx
sso_role_name = Admin
region = us-west-2
output = json

[profile my-profile2]
region = us-west-2
output = json
credential_process = aws-sso-util credential-process --profile my-profile

Then set your environment variable AWS_PROFILE or AWS_DEFAULT_PROFILE to my-profile2.

ryno75 · 2021-03-10T23:47:53Z

@z0mbix per @karlpokus' comment above, would you be up for updating the PR to set SharedConfigState: session.SharedConfigEnable?

is this not already set here? https://github.com/gruntwork-io/terragrunt/blob/master/aws_helper/config.go#L54

brikis98 · 2021-03-11T09:50:06Z

@z0mbix per @karlpokus' comment above, would you be up for updating the PR to set SharedConfigState: session.SharedConfigEnable?

is this not already set here? https://github.com/gruntwork-io/terragrunt/blob/master/aws_helper/config.go#L54

🤦

Hahah, you're right! OK, merging this and releasing. Thank you @z0mbix!

brikis98 · 2021-03-11T09:52:28Z

https://github.com/gruntwork-io/terragrunt/releases/tag/v0.28.9

* Fix dead link in multiple aws accounts docs (gruntwork-io#1563) * Fix dead link in multiple aws accounts docs The link to AWS docs is now 404. The corrected link seems to most closely resemble the intended target. Other options to consider: https://aws.amazon.com/organizations/getting-started/best-practices/ https://docs.aws.amazon.com/controltower/latest/userguide/aws-multi-account-landing-zone.html * Link to AWS best practices for multi account docs * Whitespace removal (gruntwork-io#1573) * Fix empty outputs (gruntwork-io#1568) If stack run finished without errors, `summarizePlanAllErrors()` receives empty buffer and outputs empty line. This change ensures that only non-empty outputs are getting logged. Related: gruntwork-io#1541 * doc: contributing: fix broken link to circleci (gruntwork-io#1580) * Bump AWS SDK to version v1.37.7 to support AWS SSO (gruntwork-io#1537) * Add TargetPrefix as config input to access bucket logging (gruntwork-io#1507) * adding target-prefix ro access bucket logging * Updating test & example ! Note that this needs the terratest PR (gruntwork-io/terratest#767) to be merged in to work & be tested. * Updating Terratest dependency * testing for target prefix * Updating docs * Renaming folder * Updating to Debugf * Adding default value * WIP - parsing for TFstatelogs * Updating logic & docs * Adding a new test for default TargetPrefix in remote backend config * Introduce validate-inputs, which can be used to check for variable alignment (gruntwork-io#1572) * Introduce terragrunt-input-info, which can be used to check for variable alignment * Apply suggestions from code review Co-authored-by: Zack Proser <[email protected]> * Tidy go modules * Renamed input-info to validate-inputs * Switch missing required vars to errors * Handle -var and -var-file args * Update cli/validate_inputs.go Co-authored-by: Yevgeniy Brikman <[email protected]> * Make sure to check for dynamically passed in CLI args * Fix build * Handle automatically loaded var files * Remove plan args check * Clarify difference between getTerraformInputNamesFromVarFiles and getTerraformInputNamesFromCLIArgs * Address PR nit to move example in docs Co-authored-by: Zack Proser <[email protected]> Co-authored-by: Yevgeniy Brikman <[email protected]> * Use go1.16 to build arm64 binaries (gruntwork-io#1585) * Bump creack/pty to 1.1.11 (gruntwork-io#1582) Co-authored-by: Andy Bohne <[email protected]> * Add ability to specify working directory of hooks (gruntwork-io#1588) * Add ability to specify working directory of hooks * Fix build * Support dynamodb_endpoint attribute of S3 backend (gruntwork-io#1586) * Clarify non-interactive will not include external dependencies (gruntwork-io#1593) * add getTerragruntSource helper function (gruntwork-io#1575) * add getTerragruntSource helper function * update docs * update docs and tests for get_terragrunt_source_cli_flag() function * add use cases for get_terragrunt_source_cli_flag * Recursively extract forcedgetters until there are none (gruntwork-io#1594) * Remove all usage of get-plugins=false which is removed in 0.15.0 (gruntwork-io#1618) * Fix validate-inputs to support null defaults (gruntwork-io#1613) * Clarify context of find_in_parent_folders (gruntwork-io#1623) Co-authored-by: Paul <[email protected]> Co-authored-by: Yoriyasu Yano <[email protected]> Co-authored-by: amnk <[email protected]> Co-authored-by: Marco Molteni <[email protected]> Co-authored-by: David Wooldridge <[email protected]> Co-authored-by: Ina Stoyanova <[email protected]> Co-authored-by: Zack Proser <[email protected]> Co-authored-by: Yevgeniy Brikman <[email protected]> Co-authored-by: Andy Bohne <[email protected]> Co-authored-by: Andy Bohne <[email protected]> Co-authored-by: Alexey Remizov <[email protected]> Co-authored-by: Syed Hussain <[email protected]> Co-authored-by: David Alger <[email protected]>

Bump AWS SDK to version v1.37.7 to support AWS SSO

2de0241

z0mbix requested review from brikis98 and yorinasub17 as code owners February 9, 2021 14:16

z0mbix mentioned this pull request Feb 9, 2021

AWS CLIv2 and AWS SSO auth fails #1129

Closed

brikis98 approved these changes Feb 10, 2021

View reviewed changes

thapabishwa reviewed Feb 11, 2021

View reviewed changes

brikis98 merged commit ed46f1a into gruntwork-io:master Mar 11, 2021

This was referenced Apr 21, 2021

Terragrunt Not Finding SSO Credentials #1599

Closed

remote_state backend S3 bucket parameters not set when using an assumed role profile #1650

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump AWS SDK to version v1.37.7 to support AWS SSO #1537

Bump AWS SDK to version v1.37.7 to support AWS SSO #1537

z0mbix commented Feb 9, 2021 •

edited

Loading

thapabishwa commented Feb 10, 2021 •

edited

Loading

brikis98 left a comment

z0mbix commented Feb 10, 2021

z0mbix commented Feb 10, 2021

thapabishwa left a comment

thapabishwa commented Feb 11, 2021 •

edited

Loading

brikis98 commented Feb 11, 2021

z0mbix commented Feb 11, 2021

brikis98 commented Feb 12, 2021

sylr commented Feb 15, 2021

z0mbix commented Feb 16, 2021

karlpokus commented Feb 19, 2021 •

edited

Loading

brikis98 commented Feb 22, 2021

UrosCvijan commented Feb 26, 2021

oscarmorasu commented Mar 3, 2021

swordfish444 commented Mar 10, 2021

ryno75 commented Mar 10, 2021

brikis98 commented Mar 11, 2021

brikis98 commented Mar 11, 2021

Bump AWS SDK to version v1.37.7 to support AWS SSO #1537

Bump AWS SDK to version v1.37.7 to support AWS SSO #1537

Conversation

z0mbix commented Feb 9, 2021 • edited Loading

thapabishwa commented Feb 10, 2021 • edited Loading

brikis98 left a comment

Choose a reason for hiding this comment

z0mbix commented Feb 10, 2021

z0mbix commented Feb 10, 2021

thapabishwa left a comment

Choose a reason for hiding this comment

thapabishwa commented Feb 11, 2021 • edited Loading

brikis98 commented Feb 11, 2021

z0mbix commented Feb 11, 2021

brikis98 commented Feb 12, 2021

sylr commented Feb 15, 2021

z0mbix commented Feb 16, 2021

karlpokus commented Feb 19, 2021 • edited Loading

brikis98 commented Feb 22, 2021

UrosCvijan commented Feb 26, 2021

oscarmorasu commented Mar 3, 2021

swordfish444 commented Mar 10, 2021

ryno75 commented Mar 10, 2021

brikis98 commented Mar 11, 2021

brikis98 commented Mar 11, 2021

z0mbix commented Feb 9, 2021 •

edited

Loading

thapabishwa commented Feb 10, 2021 •

edited

Loading

thapabishwa commented Feb 11, 2021 •

edited

Loading

karlpokus commented Feb 19, 2021 •

edited

Loading