aws_ssm lookup - AWS_PROFILE in environment blocks authentication tokens from environment

[kgignatyev]

Summary

Here is playbook

aws_secret lookup works but aws_ssm lookup does not. It fails with message:
Error was a <class 'ansible.errors.AnsibleError'>, original message: SSM lookup exception: An error occurred (AccessDenied) when calling the AssumeRole operation: ..... SSM lookup exception: An error occurred (AccessDenied) when calling the AssumeRole operation:

command line works fine

aws ssm get-parameter --name=/auth0/user/iam_admin/password    

environment is configured with AWS_SESSION_TOKEN obtained with using MFA

AWS_ACCESS_KEY_ID=zzzzz
AWS_PROFILE=YYYYYY
AWS_REGION=us-west-2
AWS_SECRET_ACCESS_KEY=XXXXX
AWS_SESSION_TOKEN=TTTTT

the playbook above starts working if AWS_PROFILE is unset

So, it looks like aws_ssm is does not prioritize use of AWS_SESSION_TOKEN

Issue Type

Bug Report

Component Name

aws_ssm

Ansible Version

$ ansible --version

ansible [core 2.13.5]
config file = None
configured module search path = ['/Users/kignatyev/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.10/site-packages/ansible
ansible collection location = /Users/kignatyev/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.10.8 (main, Oct 13 2022, 10:17:43) [Clang 14.0.0 (clang-1400.0.29.102)]
jinja version = 3.1.2
libyaml = True

Collection Versions

$ ansible-galaxy collection list

/usr/local/lib/python3.10/site-packages/ansible_collections

Collection Version

---

amazon.aws 3.5.0
ansible.netcommon 3.1.3
ansible.posix 1.4.0
ansible.utils 2.6.1
ansible.windows 1.11.1
arista.eos 5.0.1
awx.awx 21.7.0
azure.azcollection 1.13.0
check_point.mgmt 2.3.0
chocolatey.chocolatey 1.3.1
cisco.aci 2.2.0
cisco.asa 3.1.0
cisco.dnac 6.6.0
cisco.intersight 1.0.19
cisco.ios 3.3.2
cisco.iosxr 3.3.1
cisco.ise 2.5.5
cisco.meraki 2.11.0
cisco.mso 2.0.0
cisco.nso 1.0.3
cisco.nxos 3.2.0
cisco.ucs 1.8.0
cloud.common 2.1.2
cloudscale_ch.cloud 2.2.2
community.aws 3.6.0
community.azure 1.1.0
community.ciscosmb 1.0.5
community.crypto 2.7.0
community.digitalocean 1.22.0
community.dns 2.3.3
community.docker 2.7.1
community.fortios 1.0.0
community.general 5.7.0
community.google 1.0.0
community.grafana 1.5.3
community.hashi_vault 3.3.1
community.hrobot 1.5.2
community.libvirt 1.2.0
community.mongodb 1.4.2
community.mysql 3.5.1
community.network 4.0.1
community.okd 2.2.0
community.postgresql 2.2.0
community.proxysql 1.4.0
community.rabbitmq 1.2.2
community.routeros 2.3.0
community.sap 1.0.0
community.sap_libs 1.3.0
community.skydive 1.0.0
community.sops 1.4.1
community.vmware 2.10.0
community.windows 1.11.0
community.zabbix 1.8.0
containers.podman 1.9.4
cyberark.conjur 1.2.0
cyberark.pas 1.0.14
dellemc.enterprise_sonic 1.1.2
dellemc.openmanage 5.5.0
dellemc.os10 1.1.1
dellemc.os6 1.0.7
dellemc.os9 1.0.4
f5networks.f5_modules 1.20.0
fortinet.fortimanager 2.1.5
fortinet.fortios 2.1.7
frr.frr 2.0.0
gluster.gluster 1.0.2
google.cloud 1.0.2
hetzner.hcloud 1.8.2
hpe.nimble 1.1.4
ibm.qradar 2.1.0
ibm.spectrum_virtualize 1.10.0
infinidat.infinibox 1.3.3
infoblox.nios_modules 1.4.0
inspur.ispim 1.1.0
inspur.sm 2.2.0
junipernetworks.junos 3.1.0
kubernetes.core 2.3.2
mellanox.onyx 1.0.0
netapp.aws 21.7.0
netapp.azure 21.10.0
netapp.cloudmanager 21.20.1
netapp.elementsw 21.7.0
netapp.ontap 21.24.1
netapp.storagegrid 21.11.1
netapp.um_info 21.8.0
netapp_eseries.santricity 1.3.1
netbox.netbox 3.8.0
ngine_io.cloudstack 2.2.4
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.2
openstack.cloud 1.10.0
openvswitch.openvswitch 2.1.0
ovirt.ovirt 2.2.3
purestorage.flasharray 1.14.0
purestorage.flashblade 1.10.0
purestorage.fusion 1.1.1
sensu.sensu_go 1.13.1
servicenow.servicenow 1.0.6
splunk.es 2.1.0
t_systems_mms.icinga_director 1.31.0
theforeman.foreman 3.7.0
vmware.vmware_rest 2.2.0
vultr.cloud 1.1.0
vyos.vyos 3.0.1
wti.remote 1.0.4

/Users/kignatyev/.ansible/collections/ansible_collections

Collection Version

---

amazon.aws 5.1.0
community.general 3.3.2

AWS SDK versions

$ pip show boto boto3 botocore

pip show boto boto3 botocore

WARNING: Package(s) not found: boto
Name: boto3
Version: 1.25.4
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:

Name: botocore
Version: 1.28.4
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

OS X 12.6.1

Steps to Reproduce

    - name: "read roach connection info"
      set_fact:
        roach_connection_info_dict: "{{lookup('amazon.aws.aws_secret', 'infra/'+env+'/roach/connection_info',  on_missing='error')}}"

    - name: "read iam admin password"
      set_fact:
        iam_admin_password: "{{lookup('amazon.aws.aws_ssm', '/auth0/user/iam_admin/password') }}"

Expected Results

"read iam admin password" works in MFA session

Actual Results

TASK [read iam admin password] ********************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'amazon.aws.aws_ssm'. Error was a <class 'ansible.errors.AnsibleError'>, original message: SSM lookup exception: An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::000000:user/aaaa/ZZZZ is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::111111:role/developer. SSM lookup exception: An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::000000:user/aaaa/ZZZZ is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::111111:role/developer"}

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• [plugins/lookup/aws_ssm.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/lookup/aws_ssm.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[tremble]

@kgignatyev, thank you for taking the time to open this issue.

Release 3.5.0 is no longer supported, and support for passing the combination of profile and authentication tokens was dropped in release 4.0.0.

It looks like there's an edge case where passing them as environment variables isn't properly caught and results in the profile having precedence and authentication tokens being ignored in our 4.x and 5.x releases, however the issue is resolved in our 'main' branch by #1174, and I've triggered the backport process for our 5.x branches. As such I'm going to close this issue.