Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot connect to ec2 instance via aws_ssm if AWS_SESSION_TOKEN is missing #343

Closed
ru-rocker opened this issue Jan 4, 2021 · 6 comments · Fixed by #514 or #535
Closed

Cannot connect to ec2 instance via aws_ssm if AWS_SESSION_TOKEN is missing #343

ru-rocker opened this issue Jan 4, 2021 · 6 comments · Fixed by #514 or #535
Labels
bug This issue/PR relates to a bug has_pr module module plugins plugin (any type) python3 traceback

Comments

@ru-rocker
Copy link
Contributor

SUMMARY

Cannot connect to ec2 instance via aws_ssm if AWS_SESSION_TOKEN is not set.
(I do not configure any AWS variables in OS via aws configure)

ISSUE TYPE
  • Bug Report
COMPONENT NAME

ansible-collections/community.aws/blob/main/plugins/connection/aws_ssm.py

ANSIBLE VERSION
ansible 2.9.9
  config file = None
  configured module search path = ['/Users/it-ops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.7 (default, Mar 10 2020, 15:43:03) [Clang 11.0.0 (clang-1100.0.33.17)]
CONFIGURATION

OS / ENVIRONMENT

OSX 10.14.6 (18G3020)

STEPS TO REPRODUCE

I created a role to ping ec2 instance.

# roles/ssm/tasks/main.yml
- name: ping
  ping:

# roles/ssm/defaults/main.yml
ansible_connection: aws_ssm
ansible_aws_ssm_region: ap-southeast-1
ansible_aws_ssm_access_key_id: XXXXXXXXX
ansible_aws_ssm_secret_access_key: XXXXXXXXX
ansible_aws_ssm_instance_id: i-XXXXXXXX
ansible_aws_ssm_bucket_name: bucket-name

I guess this is related with this loc:

    def _get_boto_client(self, service, region_name=None):
        ''' Gets a boto3 client based on the STS token '''

        aws_access_key_id = self.get_option('access_key_id')
        aws_secret_access_key = self.get_option('secret_access_key')
        aws_session_token = self.get_option('session_token')
        if aws_access_key_id is None or aws_secret_access_key is None or aws_session_token is None:
            aws_access_key_id = os.environ.get("AWS_ACCESS_KEY_ID", None)
            aws_secret_access_key = os.environ.get("AWS_SECRET_ACCESS_KEY", None)
            aws_session_token = os.environ.get("AWS_SESSION_TOKEN", None)

Because I do not need aws_session_token (yet) to connect to ec2 instances via aws_ssm to do a ping, and I do not set any OS environment variables, all my aws_access_key_id and aws_secret_access_key are set to None again because the aws_session_token is empty.

Suggested fixes:

    def _get_boto_client(self, service, region_name=None):
        ''' Gets a boto3 client based on the STS token '''

        aws_access_key_id = self.get_option('access_key_id')
        aws_secret_access_key = self.get_option('secret_access_key')
        aws_session_token = self.get_option('session_token')

        if aws_access_key_id is None:
            aws_access_key_id = os.environ.get("AWS_ACCESS_KEY_ID", None)
        if aws_secret_access_key is None:
            aws_secret_access_key = os.environ.get("AWS_SECRET_ACCESS_KEY", None)
        if aws_session_token is None:
            aws_session_token = os.environ.get("AWS_SESSION_TOKEN", None)
EXPECTED RESULTS
TASK [ssm : ping] *****************************************************************************************************************************************************************************
ok: [ec2]

PLAY RECAP ************************************************************************************************************************************************************************************
ec2                        : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ACTUAL RESULTS
PLAY [nodes] **********************************************************************************************************************************************************************************

TASK [ssm : ping] *****************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.NoCredentialsError: Unable to locate credentials
fatal: [ec2]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}

PLAY RECAP ************************************************************************************************************************************************************************************
ec2                        : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
@ansibullbot
Copy link

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot
Copy link

@ansibullbot ansibullbot added bug This issue/PR relates to a bug module module needs_triage plugins plugin (any type) python3 traceback labels Jan 13, 2021
@gravesm
Copy link
Member

gravesm commented Mar 19, 2021

@ru-rocker thank you for filing the issue and the suggested fix. Would you like to open a PR for this?

@ansibullbot
Copy link

@dekimsey
Copy link
Contributor

Perhaps this issue could be addressed differently. The ansible options parser already supports the ability to pick-up environment variables. Why not use that instead?
Right now these fallbacks including region and remote_addr behavior could be addressed by updating configs.

Say something like:

  session_token:
    description: The STS session token to use when connecting via session-manager.
    vars:
    - name: ansible_aws_ssm_session_token
    env:
    - name: AWS_SESSION_TOKEN
      version_added: 1.5.0
    version_added: 1.3.0

I just had to address this in our environment because as designed, the code will override the configured vars with None's because there are no such environment variables set. I'll look at attaching a pull request to address this since I just had to work-around it manually.

dekimsey added a commit to dekimsey/community.aws that referenced this issue Mar 30, 2021
This addresses issues concerning connections failures due to not needing
session tokens. If the any one of the connection vars are not set, *all*
the connection vars are reset to their environmental value, or `None`.
This causes mysterious CredentialsNotFound errors. Further, because
handling was being done in the code, this fallback behavior was not documented.

Fixes ansible-collections#343
@ru-rocker
Copy link
Contributor Author

Sorry for the late reply.

The reason I opened this issue is I need dynamic access_key_id and access_secret_key_id due to security requirements.
I need to store those keys inside some password manager and it is prohibited to store them as environment variables.

ru-rocker added a commit to ru-rocker/community.aws that referenced this issue Apr 9, 2021
ru-rocker added a commit to ru-rocker/community.aws that referenced this issue Apr 9, 2021
ru-rocker added a commit to ru-rocker/community.aws that referenced this issue Apr 9, 2021
tremble pushed a commit that referenced this issue Apr 9, 2021
* fix issue #343 for missing AWS_SESSION_TOKEN
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this issue Jul 19, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this issue Jul 19, 2021
danielcotton pushed a commit to danielcotton/community.aws that referenced this issue Nov 23, 2021
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jan 31, 2023
aws_ssm connection - Move connection vars environment handling into options

SUMMARY
This fix moves a number of connection related variables to the options parsing step instead of inline. This has the added effect of documenting their existence and making overriding them more consistent with Ansible's UX.
Fixes #343
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME
aws_ssm
ADDITIONAL INFORMATION


There were a couple of other minor changes related to logging and silencing curl's progress info outside of the connection vars themselves. I'm happy to pull them out if desired and submit them as a separate PR.
I added fallback on hostnames lookup to match SSH's host handling since that's the defacto connection plugin. This incidentally fixes the way delegation is reported (it didn't show the -> delegated host bit in the logs).
Of note, the ec2.py module sets the instance_id and placement on instances it detects so I added it as first-class fallback for instance_id and region parameters respectively. The get_options parser doesn't handle nested variable lookups, so I had to modify the lookup slightly.

Reviewed-by: Jill R <None>
Reviewed-by: Guillaume GILL <None>
Reviewed-by: Mark Chappell <None>
patchback bot pushed a commit that referenced this issue Jan 31, 2023
aws_ssm connection - Move connection vars environment handling into options

SUMMARY
This fix moves a number of connection related variables to the options parsing step instead of inline. This has the added effect of documenting their existence and making overriding them more consistent with Ansible's UX.
Fixes #343
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
aws_ssm
ADDITIONAL INFORMATION

There were a couple of other minor changes related to logging and silencing curl's progress info outside of the connection vars themselves. I'm happy to pull them out if desired and submit them as a separate PR.
I added fallback on hostnames lookup to match SSH's host handling since that's the defacto connection plugin. This incidentally fixes the way delegation is reported (it didn't show the -> delegated host bit in the logs).
Of note, the ec2.py module sets the instance_id and placement on instances it detects so I added it as first-class fallback for instance_id and region parameters respectively. The get_options parser doesn't handle nested variable lookups, so I had to modify the lookup slightly.

Reviewed-by: Jill R <None>
Reviewed-by: Guillaume GILL <None>
Reviewed-by: Mark Chappell <None>
(cherry picked from commit 94d1295)
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jan 31, 2023
)

[PR #514/94d12952 backport][stable-5] aws_ssm connection - Move connection vars environment handling into options

This is a backport of PR #514 as merged into main (94d1295).
SUMMARY
This fix moves a number of connection related variables to the options parsing step instead of inline. This has the added effect of documenting their existence and making overriding them more consistent with Ansible's UX.
Fixes #343
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME
aws_ssm
ADDITIONAL INFORMATION


There were a couple of other minor changes related to logging and silencing curl's progress info outside of the connection vars themselves. I'm happy to pull them out if desired and submit them as a separate PR.
I added fallback on hostnames lookup to match SSH's host handling since that's the defacto connection plugin. This incidentally fixes the way delegation is reported (it didn't show the -> delegated host bit in the logs).
Of note, the ec2.py module sets the instance_id and placement on instances it detects so I added it as first-class fallback for instance_id and region parameters respectively. The get_options parser doesn't handle nested variable lookups, so I had to modify the lookup slightly.

Reviewed-by: Mark Chappell <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue/PR relates to a bug has_pr module module plugins plugin (any type) python3 traceback
Projects
None yet
4 participants