Aws::SharedCredentials does not appear to respect the source_profile option

[geekifier]

Example of ~/.aws/config

[profile account1]
region = us-east-1

[profile account2]
role_arn = arn:aws:iam::XXXXX:role/STSRole
source_profile = account1
mfa_serial = arn:aws:iam::XXXXX:mfa/adminuser

Example of ~/.aws/credentials

[account1]
aws_access_key_id = XXX
aws_secret_access_key = XXX

Expected behavior:

Aws::AssumeRoleCredentials uses credentials of the IAM user in account1, in order to assume STS role in account2.

What Actually happens:

Aws::AssumeRoleCredentials uses the default credentials to assume the STS role in account2. This fails due to lack of access.

Either Aws::SharedCredentials does not respect the source_profile directive, or Aws:AssumeRoleCredentials does not use the SharedCredentials object to obtain it's access key (possibly both).

So this issue might include a hidden request for AssumeRoleCredentials to support passing it an instance of SharedCredentials, instead of passing it the role_arn manually. Currently, I am parsing the ~/.aws/config file manually and extract role_arn from there.

Sample Ruby snippet:

require 'aws-sdk'
require 'inifile'

def aws_init(profile = 'default')

    # conform to https://github.com/aws/aws-sdk-ruby/blob/master/CHANGELOG.md#240-2016-07-19
    ENV['AWS_SDK_LOAD_CONFIG'] = 'true'

    # string for looking up aws_config, account for default profile
    configSearch = profile == 'default' ? profile : "profile #{profile}"

    # load .aws/config from $HOME
    configFile = IniFile.load(ENV['HOME'] + '/.aws/config')

    # load profile settings for --profile
    @configData = configFile[configSearch]

    # Create new Credentials object from profile
    credentials = Aws::SharedCredentials.new(profile_name: profile)

    # if ~/.aws/config has the role_arn specified, assume STS role

    if !@configData['role_arn'].nil?

        role_credentials = Aws::AssumeRoleCredentials.new(
            role_arn: @configData['role_arn'],
            role_session_name: profile
        )

        ENV['AWS_ACCESS_KEY_ID'] = role_credentials.credentials.access_key_id
        ENV['AWS_SECRET_ACCESS_KEY'] = role_credentials.credentials.secret_access_key
        ENV['AWS_SECURITY_TOKEN'] = role_credentials.credentials.session_token
    else
        # if no role_arn specifed in AWS_CONFIG, provide current user creds

        ENV['AWS_ACCESS_KEY_ID'] = credentials.credentials.access_key_id
        ENV['AWS_SECRET_ACCESS_KEY'] = credentials.credentials.secret_access_key
        ENV['AWS_SECURITY_TOKEN'] = credentials.credentials.session_token
    end
end

aws_init('account2')

[awood45]

If you're setting ENV within the process, it may not be recognized as we check at initial SDK load time.

To be clear, which failure mode are you seeing?

1. Your client has AssumeRole credentials, but the role was assumed using the wrong profile credentials.
2. Your client is using static credentials from your config file.

I'm going to suspect 2, but want to be sure so I'm checking the correct use case.

[awood45]

Actually wait, I'm sorry, I misunderstood. I was looking at the behavior of the default credential provider chain, your example is something different. In your file above, you're not using the Shared Config logic for assuming a role, you're calling Aws::AssumeRoleCredentials directly and providing your own profile.

Is the script above the exact (or approximate) script you're trying to debug?

[geekifier]

Thinking some more about this, I believe what I am seeing is that there seems to be no way to pass credentials parsed from the config file using SharedCredentials into AssumeRoleCredentials. This might be simply my lack of understanding of the API, as I am not a Ruby dev.

But I would like to essentially be able to construct a SharedCredentials instance, which then can be used by AssumeRoleCredentials to obtain the access keys the token. Basically, assume STS role defined in config profile foo, using credentials defined in profile bar.

The use case here is that I am building a wrapper for another product that does not support STS. So I am obtaining the temporary credentials from the target STS role, using whatever local credentials that are specified under a profile in the config file. I am then getting those credentials, and passing them along to a new sub-process using environmental variables.

I hope this helps to clarify my issue/request.

[geekifier]

Another way I have tried this:

credentials = Aws::SharedCredentials.new("config_profile_name")

sts = Aws::STS::Client.new(credentials: credentials)
sts.assume_role({ role_arn = "ARN", role_session_name = "Session"})

This resulted in the following error:
/Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.4.3/lib/aws-sdk-core/plugins/request_signer.rb:100:in require_credentials': unable to sign request without credentials set (Aws::Errors::MissingCredentialsError).`

The error only occurs when I specify a CLI profile that sources credentials from another CLI profile. E.g. 'default' profile has actual API keys in the credentials file. Other CLI profiles use the source_profile directive to reference those credentials. They don't themselves have API keys, only the role_arn parameter and region is specified.

So the root of my problem, other than my trial-and-error programming approach, seems to be the fact that Aws:SharedCredentials does not seem to load the credentials from source_profile, unlike the CLI.

My setup is pretty vanilla, as it follows the example listed in AWS docs:

# In ~/.aws/credentials:
[development]
aws_access_key_id=foo
aws_secret_access_key=bar

# In ~/.aws/config
[profile crossaccount]
role_arn=arn:aws:iam:...
source_profile=development

[awood45]

Is there any reason not to just use the default credential provider chain? For your example above, if you ran Ruby with the ENV variable already set (not set in code), this should work:

client = Aws::S3::Client.new(profile: "account2", region: REGION)

Substitute S3 for whichever client you need. This will follow the config resolution path for AssumeRole, and will provide automatically refreshed credentials.

[geekifier]

Thanks so much for the continued assistance!

I tried the example you have provided in aws.rb. It does appear to work, as long as the environmental variable is set, and MFA is not required. Could you provide a quick example of how I would make this work with MFA?

Here's what I tried:

Aws> client = Aws::STS::Client.new(profile: "CLIENTPROFILE", region: "us-west-1")
[Aws::STS::Client 403 0.179872 0 retries] assume_role(role_session_name:"default_session",role_arn:"arn:aws:iam::XXXX:role/XXXXX",external_id:nil,serial_number:"arn:aws:iam::XXXXXX:mfa/XXXXX") Aws::STS::Errors::AccessDenied MultiFactorAuthenticaiton failed, must provide both MFA serial number and one time pass code.
Aws::STS::Errors::AccessDenied: MultiFactorAuthenticaiton failed, must provide both MFA serial number and one time pass code.
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.4.3/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'

When initializing a new STS client, there is nowhere to provide the token_code.

Again, my use case here is that I need to export raw access_key_id, secret_key and session_token from assumerole, so that I can pass it onto an app that does not support profiles/STS.

[awood45]

Starting with the base class used by this system: Let's say you wanted to assume a given role_arn with source credentials in profile account1. You'd make the following call:

sts_client = Aws::STS::Client.new(profile: 'account1', region: REGION)
credentials = Aws::AssumeRoleCredentials.new(
  client: client,
  role_arn: role_arn,
  role_session_name: "mysession",
  serial_number: mfa_serial,
  token_code: token_code
)
s3_client = Aws::S3::Client.new(credentials: credentials, region: REGION)

If you start from the basic usage pattern, sourcing values from config is a matter of parsing that config source. Currently, the method used by the default chain is behind a private API, so I can take it as a feature request to expose a method to directly create an MFA Assume Role call with config values.

[geekifier]

OK, I think we finally got to the bottom of my request. Sorry if I haven't done a good job explaining this.

Looks like in the meantime I will be forced to implement my own parser for the config (or some other way to supply all of the STS parameters) to make this easy for the users.

[markfink]

the suggested AssumeRoleCredentials code works for me. I use this for reading credentials and config:
https://github.com/a2ikm/aws_config

and STDIN.get.chomp to read in the token_code

[awood45]

Added to feature request backlog.

[mullermp]

Reopening - deprecating usage of Feature Requests backlog markdown file.

[tarcinil]

This does not respect the following as well.

role_arn = foo
credential_source = Ec2InstanceMetadata

This is because of how theaws-sdk-core/shared_config.rb#credentials
which calls aws-sdk-core/shared_config.rb#credentials_from_shared
which then calls the final aws-sdk-core/shared_config.rb#credentials_from_profile

This method only contains the following code:

def credentials_from_profile(prof_config)
    creds = Credentials.new(
    prof_config['aws_access_key_id'],
    prof_config['aws_secret_access_key'],
    prof_config['aws_session_token']
    )
    if credentials_complete(creds)
    creds
    else
    nil
    end
end

This method should be updated to check to support all role_arn and both source_profile
and credential_sources.

[alextwoods]

I believe this has been fixed with PR #2223 - and you should be able to pass profile when creating a client and the credential resolution chain will correctly use role_arn/credential_source to create AssumeRoleCredentials from the parsed config for the given profile.

I'm going to close this out but feel free to re-open if this doesn't fix the underlying issue for you.