Prompted multiple times when using MFA based access to accounts #874

apsamuel · 2022-06-16T14:54:19Z

Summary

When using the amazon.aws inventory module to execute ansible tasks against EC2 instances grouped by tag, I am facing an issue where I am being prompted (MFA codes) once for each member of a group. The result is running a task against a group with 4 active instances, which will prompt for MFA 4 times. My expectation is that it would request MFA once, and possibly cache the credential for reuse similar to how the AWS CLI does.

Issue Type

Bug Report

Component Name

amazon.aws

Ansible Version

$ ansible --version

ansible [core 2.12.6]
  config file = /Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg
  configured module search path = ['/Users/aaronsamuel/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/aaronsamuel/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/aaronsamuel/opt/anaconda3/bin/ansible
  python version = 3.9.7 (default, Sep 16 2021, 08:50:36) [Clang 10.0.0 ]
  jinja version = 2.11.3
  libyaml = True

Collection Versions

# /Users/aaronsamuel/.ansible/collections/ansible_collections
Collection Version
---------- -------
amazon.aws 3.3.0  

# /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    2.3.0  
ansible.netcommon             2.6.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.10.0 
arista.eos                    3.1.0  
awx.awx                       19.4.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.2.0  
cisco.aci                     2.2.0  
cisco.asa                     2.1.0  
cisco.dnac                    6.4.0  
cisco.intersight              1.0.19 
cisco.ios                     2.8.1  
cisco.iosxr                   2.9.0  
cisco.ise                     1.2.1  
cisco.meraki                  2.6.2  
cisco.mso                     1.4.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.9.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.1  
cloudscale_ch.cloud           2.2.2  
community.aws                 2.5.0  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.3.2  
community.digitalocean        1.19.0 
community.dns                 2.2.0  
community.docker              2.6.0  
community.fortios             1.0.0  
community.general             4.8.2  
community.google              1.0.0  
community.grafana             1.4.0  
community.hashi_vault         2.5.0  
community.hrobot              1.4.0  
community.kubernetes          2.0.1  
community.kubevirt            1.0.0  
community.libvirt             1.1.0  
community.mongodb             1.4.0  
community.mysql               2.3.8  
community.network             3.3.0  
community.okd                 2.2.0  
community.postgresql          1.7.4  
community.proxysql            1.4.0  
community.rabbitmq            1.2.1  
community.routeros            2.1.0  
community.sap                 1.0.0  
community.sap_libs            1.1.0  
community.skydive             1.0.0  
community.sops                1.2.2  
community.vmware              1.18.0 
community.windows             1.10.0 
community.zabbix              1.7.0  
containers.podman             1.9.3  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            4.4.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.17.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.6  
frr.frr                       1.0.4  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.4  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.2.2  
inspur.sm                     1.3.0  
junipernetworks.junos         2.10.0 
kubernetes.core               2.3.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.17.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.19.1
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.0  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.1  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   1.6.6  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.29.0 
theforeman.foreman            2.2.0  
vmware.vmware_rest            2.1.5  
vyos.vyos                     2.8.0  
wti.remote                    1.0.3

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.24.6
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages
Requires: s3transfer, jmespath, botocore
Required-by: 
---
Name: botocore
Version: 1.27.6
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/aaronsamuel/opt/anaconda3/lib/python3.9/site-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3

Configuration

$ ansible-config dump --only-changed
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ STDIN
       │ Size: -
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ DEFAULT_HOST_LIST(/Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg) = ['/Users/aaronsamuel/devops/rearc/noop-etc/ansible/aws_ec2.yaml']
   2   │ HOST_KEY_CHECKING(/Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg) = False
   3   │ INVENTORY_ENABLED(/Users/aaronsamuel/devops/rearc/noop-etc/ansible/ansible.cfg) = ['aws_ec2']

OS / Environment

Operating System: macOS m1 chip

❯ sw_vers
ProductName:    macOS
ProductVersion: 12.4
BuildVersion:   21F79

in rosetta terminal

Target OS: Amazon Linux 2

Steps to Reproduce

❯ ANSIBLE_CACHE_PLUGIN=memory ANSIBLE_INVENTORY_CACHE_PLUGIN=memory ANSIBLE_CONFIG=./ansible.cfg AWS_PROFILE=profile-name ansible service_control -a 'uptime' -u ec2-user --verbose --become --become-method=sudo
Using /Users/aaronsamuel/devops/project/etc/ansible/ansible.cfg as config file
Enter MFA code for arn:aws:iam::XXXXX:mfa/[email protected]: 
Enter MFA code for arn:aws:iam::XXXX:mfa/[email protected]:

Expected Results

I expected the MFA prompting to happen once.

Actual Results

# I am prompted for MFA seemingly once for each member of the target group
❯ ANSIBLE_CACHE_PLUGIN=memory ANSIBLE_INVENTORY_CACHE_PLUGIN=memory ANSIBLE_CONFIG=./ansible.cfg AWS_PROFILE=profile-name ansible service_control -a 'uptime' -u ec2-user --verbose --become --become-method=sudo
Using /Users/aaronsamuel/devops/project/etc/ansible/ansible.cfg as config file
Enter MFA code for arn:aws:iam::XXXXX:mfa/[email protected]: 
Enter MFA code for arn:aws:iam::XXXX:mfa/[email protected]:

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

ansibullbot · 2022-06-16T15:01:19Z

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

tremble · 2022-11-16T15:25:56Z

@apsamuel I'm guessing you're providing the credentials using the assume-role-provider pattern https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#assume-role-provider ?

I've been doing some work to try and improve things here on #1271. The underlying problem relates to how the AWS SDK and APIs work: To query a region for EC2 Instances you have to establish and authenticate to each of the regions individually. Unfortunately, the APIs simply do not permit you to reuse the connections between regions (the WebUI uses some additional tricks not available to the CLI/SDK).

With the code in #1271 (won't be generally available until we release 6.0.0 because some of the changes are invasive) things should have improved somewhat. It's at a state where when "regions" is specified, you're now only prompted for an MFA token once per region (previously you may be prompted a couple of times). If you only specify one region, you're only prompted for your MFA once.

There is a separate feature which also helps things, "iam_role_arn". This causes the code to call "sts:AssumeRole" (performing the MFA authentication when required) and then the code has a session token which can be reused. However, to do so, you have to specify "iam_role_arn" in the inventory file, and the role has to permit re-assuming itself without MFA, for example:

Trust Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/AssumableRole"
            }
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "Bool": {
                    "aws:MultiFactorAuthPresent": "true"
                }
            }
        }
    ]
}

aws_ec2.yml (inventory file)

plugin: amazon.aws.aws_ec2

profile: ansible_test_assumable
assume_role_arn: 'arn:aws:iam::123456789012:role/AssumableRole'

tremble · 2023-02-06T09:32:24Z

Not much more can be done here. We have to open separate connections for each region, and unless someone specifies assuming a role then AWS will request a new token for each region.

With the Web UI they explicitly get a session token in the background which can be reused, we have a separate module to get these tokens, but we can only get them in specific circumstances, and I'm not sure it's appropriate for us to try and magically do this in the background, since we can't reuse them.

Release 3.0.1 commit SUMMARY First patch Release ISSUE TYPE Docs Pull Request Feature Pull Request COMPONENT NAME entire collection ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None> Reviewed-by: Mark Woolley <[email protected]>

ansibullbot added bug This issue/PR relates to a bug needs_triage python3 labels Jun 16, 2022

jillr added waiting_on_contributor Needs help. Feel free to engage to get things unblocked and removed needs_triage labels Jun 28, 2022

tremble mentioned this issue Nov 16, 2022

Refactor inventory plugin connection handling #1271

Merged

ansibullbot added the has_pr label Nov 16, 2022

tremble closed this as completed Feb 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prompted multiple times when using MFA based access to accounts #874

Prompted multiple times when using MFA based access to accounts #874

apsamuel commented Jun 16, 2022

ansibullbot commented Jun 16, 2022

tremble commented Nov 16, 2022

tremble commented Feb 6, 2023

Prompted multiple times when using MFA based access to accounts #874

Prompted multiple times when using MFA based access to accounts #874

Comments

apsamuel commented Jun 16, 2022

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

ansibullbot commented Jun 16, 2022

tremble commented Nov 16, 2022

tremble commented Feb 6, 2023