Cleanup shared 'EC2' parameters (#1172) (#1173)

[PR #1172/06cecace backport][stable-5] Cleanup shared 'EC2' parameters

This is a backport of PR #1172 as merged into main (06cecac).
SUMMARY
As seen in both #1170 and ansible-collections/community.aws#1458 the "EC2_" prefix for a number of fall-back environment variables results in confusion.  "EC2_" implies it's specific to the EC2 API, not all of AWS.

Deprecates the ec2_ parameter aliases
Deprecates the EC2_ environment variables
Deprecates the security_token/aws_security_token/access_token parameter aliases which were based on the original boto parameter names

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
plugins/module_utils/botocore.py
plugins/module_utils/modules.py
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell <None>

changelogs/fragments/1172-credentials_parameters.yml

@@ -0,0 +1,57 @@
+minor_changes:
+- amazon.aws collection - The ``aws_access_key`` parameter has been renamed to ``access_key``,
+  ``access_key`` was previously an alias for this parameter and ``aws_access_key`` remains as an
+  alias.  This change should have no observable effect for users outside the module/plugin documentation.
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``aws_secret_key`` parameter has been renamed to ``secret_key``,
+  ``secret_key`` was previously an alias for this parameter and ``aws_secret_key`` remains as an
+  alias.  This change should have no observable effect for users outside the module/plugin documentation.
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``security_token`` parameter has been renamed to ``session_token``,
+  ``security_token`` was previously an alias for this parameter and ``security_token`` remains as an
+  alias.  This change should have no observable effect for users outside the module/plugin documentation.
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+
+deprecated_features:
+- amazon.aws collection - The ``ec2_access_key`` alias for the ``access_key`` parameter has been
+  deprecated and will be removed in a release after 2024-12-01.  Please use the ``access_key``
+  name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``ec2_secret_key`` alias for the ``secret_key`` parameter has been
+  deprecated and will be removed in a release after 2024-12-01.  Please use the ``secret_key``
+  name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``access_token`` alias for the ``session_token`` parameter has been
+  deprecated and will be removed in a release after 2024-12-01.  Please use the ``session_token``
+  name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``security_token`` alias for the ``session_token`` parameter has been
+  deprecated and will be removed in a release after 2024-12-01.  Please use the ``session_token``
+  name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``aws_security_token`` alias for the ``session_token`` parameter has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``access_token`` alias for the ``session_token`` parameter has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - The ``ec2_region`` alias for the ``region`` parameter has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``region`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
+
+- amazon.aws collection - Support for the ``EC2_URL`` and ``S3_URL`` environment variables has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``endpoint_url`` parameter or ``AWS_ENDPOINT_URL`` environment variable instead
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - Support for the ``EC2_REGION`` environment variable has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``region`` parameter or ``AWS_REGION`` environment variable instead
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - Support for the ``EC2_SECURITY_TOKEN`` environment variable has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``session_token`` parameter or ``AWS_SESSION_TOKEN`` environment variable instead
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - Support for the ``EC2_SECRET_KEY`` environment variable has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``secret_key`` parameter or ``AWS_SECRET_ACCESS_KEY`` environment variable instead
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).
+- amazon.aws collection - Support for the ``EC2_ACCESS_KEY`` environment variable has
+  been deprecated and will be removed in a release after 2024-12-01.  Please use the
+  ``access_key`` parameter or ``AWS_ACCESS_KEY_ID`` environment variable instead
+  (https://github.com/ansible-collections/amazon.aws/pull/1172).

plugins/doc_fragments/aws.py

@@ -12,76 +12,120 @@ class ModuleDocFragment(object):
     # AWS only documentation fragment
     DOCUMENTATION = r'''
 options:
-  debug_botocore_endpoint_logs:
+  access_key:
     description:
-      - Use a botocore.endpoint logger to parse the unique (rather than total) "resource:action" API calls made during a task, outputing
-        the set to the resource_actions key in the task results. Use the aws_resource_action callback to output to total list made during
-        a playbook. The ANSIBLE_DEBUG_BOTOCORE_LOGS environment variable may also be used.
-    type: bool
-    default: 'no'
-  endpoint_url:
+      - AWS access key ID.
+      - See the AWS documentation for more information about access tokens
+        U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
+      - The C(AWS_ACCESS_KEY_ID), C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY)
+        environment variables may also be used in decreasing order of
+        preference.
+      - The I(aws_access_key) and I(profile) options are mutually exclusive.
+      - The I(aws_access_key_id) alias was added in release 5.1.0 for
+        consistency with the AWS botocore SDK.
+      - The I(ec2_access_key) alias has been deprecated and will be removed in a
+        release after 2024-12-01.
+      - Support for the C(EC2_ACCESS_KEY) environment variable has been
+        deprecated and will be removed in a release after 2024-12-01.
+    type: str
+    aliases: ['aws_access_key_id', 'aws_access_key', 'ec2_access_key']
+  secret_key:
     description:
-      - URL to use to connect to EC2 or your Eucalyptus cloud (by default the module will use EC2 endpoints).
-        Ignored for modules where region is required. Must be specified for all other modules if region is not used.
-        If not set then the value of the EC2_URL environment variable, if any, is used.
+      - AWS secret access key.
+      - See the AWS documentation for more information about access tokens
+        U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
+      - The C(AWS_SECRET_ACCESS_KEY), C(AWS_SECRET_KEY), or C(EC2_SECRET_KEY)
+        environment variables may also be used in decreasing order of
+        preference.
+      - The I(secret_key) and I(profile) options are mutually exclusive.
+      - The I(aws_secret_access_key) alias was added in release 5.1.0 for
+        consistency with the AWS botocore SDK.
+      - The I(ec2_secret_key) alias has been deprecated and will be removed in a
+        release after 2024-12-01.
+      - Support for the C(EC2_SECRET_KEY) environment variable has been
+        deprecated and will be removed in a release after 2024-12-01.
     type: str
-    aliases: [ ec2_url, aws_endpoint_url, s3_url ]
-  aws_secret_key:
+    aliases: ['aws_secret_access_key', 'aws_secret_key', 'ec2_secret_key']
+  session_token:
     description:
-      - C(AWS secret key). If not set then the value of the C(AWS_SECRET_ACCESS_KEY), C(AWS_SECRET_KEY), or C(EC2_SECRET_KEY) environment variable is used.
-      - The I(aws_secret_key) and I(profile) options are mutually exclusive.
+      - AWS STS session token for use with temporary credentials.
+      - See the AWS documentation for more information about access tokens
+        U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
+      - The C(AWS_SESSION_TOKEN), C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN)
+        environment variables may also be used in decreasing order of preference.
+      - The I(security_token) and I(profile) options are mutually exclusive.
+      - Aliases I(aws_session_token) and I(session_token) were added in release
+        3.2.0, with the parameter being renamed from I(security_token) to
+        I(session_token) in release 6.0.0.
+      - The I(security_token), I(aws_security_token), and I(access_token)
+        aliases have been deprecated and will be removed in a release after
+        2024-12-01.
+      - Support for the C(EC2_SECRET_KEY) and C(AWS_SECURITY_TOKEN) environment
+        variables has been deprecated and will be removed in a release after
+        2024-12-01.
     type: str
-    aliases: [ ec2_secret_key, secret_key ]
-  aws_access_key:
+    aliases: ['aws_session_token', 'security_token', 'aws_security_token', 'access_token']
+  profile:
     description:
-      - C(AWS access key). If not set then the value of the C(AWS_ACCESS_KEY_ID), C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY) environment variable is used.
-      - The I(aws_access_key) and I(profile) options are mutually exclusive.
+      - A named AWS profile to use for authentication.
+      - See the AWS documentation for more information about named profiles
+        U(https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html).
+      - The C(AWS_PROFILE) environment variable may also be used.
+      - The I(profile) option is mutually exclusive with the I(aws_access_key),
+        I(aws_secret_key) and I(security_token) options.
     type: str
-    aliases: [ ec2_access_key, access_key ]
-  security_token:
+    aliases: ['aws_profile']
+
+  endpoint_url:
     description:
-      - C(AWS STS security token). If not set then the value of the C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN) environment variable is used.
-      - The I(security_token) and I(profile) options are mutually exclusive.
-      - Aliases I(aws_session_token) and I(session_token) have been added in version 3.2.0.
+      - URL to connect to instead of the default AWS endpoints.  While this
+        can be used to connection to other AWS-compatible services the
+        amazon.aws and community.aws collections are only tested against
+        AWS.
+      - The  C(AWS_URL) or C(EC2_URL) environment variables may also be used,
+        in decreasing order of preference.
+      - The I(ec2_url) and I(s3_url) aliases have been deprecated and will be
+        removed in a release after 2024-12-01.
+      - Support for the C(EC2_URL) environment variable has been deprecated and
+        will be removed in a release after 2024-12-01.
     type: str
-    aliases: [ aws_session_token, session_token, aws_security_token, access_token ]
+    aliases: ['ec2_url', 'aws_endpoint_url', 's3_url' ]
   aws_ca_bundle:
     description:
-      - "The location of a CA Bundle to use when validating SSL certificates."
-      - "Note: The CA Bundle is read 'module' side and may need to be explicitly copied from the controller if not run locally."
+      - The location of a CA Bundle to use when validating SSL certificates.
+      - The C(AWS_CA_BUNDLE) environment variable may also be used.
     type: path
   validate_certs:
     description:
-      - When set to "no", SSL certificates will not be validated for
+      - When set to C(false), SSL certificates will not be validated for
         communication with the AWS APIs.
+      - Setting I(validate_certs=false) is strongly discouraged, as an
+        alternative, consider setting I(aws_ca_bundle) instead.
     type: bool
-    default: yes
-  profile:
-    description:
-      - The I(profile) option is mutually exclusive with the I(aws_access_key), I(aws_secret_key) and I(security_token) options.
-    type: str
-    aliases: [ aws_profile ]
+    default: true
   aws_config:
     description:
       - A dictionary to modify the botocore configuration.
-      - Parameters can be found at U(https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore.config.Config).
+      - Parameters can be found in the AWS documentation
+        U(https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore.config.Config).
     type: dict
+  debug_botocore_endpoint_logs:
+    description:
+      - Use a C(botocore.endpoint) logger to parse the unique (rather than total)
+        C("resource:action") API calls made during a task, outputing the set to
+        the resource_actions key in the task results. Use the
+        C(aws_resource_action) callback to output to total list made during
+        a playbook.
+      - The C(ANSIBLE_DEBUG_BOTOCORE_LOGS) environment variable may also be used.
+    type: bool
+    default: false
 notes:
-  - If parameters are not set within the module, the following
-    environment variables can be used in decreasing order of precedence
-    C(AWS_URL) or C(EC2_URL),
-    C(AWS_PROFILE) or C(AWS_DEFAULT_PROFILE),
-    C(AWS_ACCESS_KEY_ID) or C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY),
-    C(AWS_SECRET_ACCESS_KEY) or C(AWS_SECRET_KEY) or C(EC2_SECRET_KEY),
-    C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN),
-    C(AWS_REGION) or C(EC2_REGION),
-    C(AWS_CA_BUNDLE)
-  - When no credentials are explicitly provided the AWS SDK (boto3) that
-    Ansible uses will fall back to its configuration files (typically
-    C(~/.aws/credentials)).
+  - B(Caution:) Environment variables and configuration files are read from the
+    Ansible 'host' context and not the 'controller' context.  Files may need to
+    be explicitly copied to the 'host'.
+  - The AWS SDK (boto3) that Ansible uses may also read defaults for credentials
+    and other settings, such as the region, from its configuration files in the
+    Ansible 'host' context (typically C(~/.aws/credentials)).
     See U(https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html)
     for more information.
-  - C(AWS_REGION) or C(EC2_REGION) can be typically be used to specify the
-    AWS region, when required, but this can also be defined in the
-    configuration files.
 '''

plugins/doc_fragments/ec2.py

@@ -14,8 +14,17 @@ class ModuleDocFragment(object):
 options:
     region:
         description:
-          - The AWS region to use. If not specified then the value of the AWS_REGION or EC2_REGION environment variable, if any, is used.
-            See U(http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region)
+          - The AWS region to use.
+          - For global services such as IAM, Route53 and CloudFront, I(region)
+            is ignored.
+          - The C(AWS_REGION) or C(EC2_REGION) environment variables may also
+            be used.
+          - See the Amazon AWS documentation for more information
+            U(http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region).
+          - The C(ec2_region) alias has been deprecated and will be removed in
+            a release after 2024-12-01
+          - Support for the C(EC2_REGION) environment variable has been
+            deprecated and will be removed in a release after 2024-12-01.
         type: str
         aliases: [ aws_region, ec2_region ]
 '''

plugins/module_utils/botocore.py

@@ -167,9 +167,9 @@ def get_aws_connection_info(module, boto3=None):
     # access_key
 
     endpoint_url = module.params.get('endpoint_url')
-    access_key = module.params.get('aws_access_key')
-    secret_key = module.params.get('aws_secret_key')
-    security_token = module.params.get('security_token')
+    access_key = module.params.get('access_key')
+    secret_key = module.params.get('secret_key')
+    session_token = module.params.get('session_token')
     region = get_aws_region(module)
     profile_name = module.params.get('profile')
     validate_certs = module.params.get('validate_certs')
@@ -184,7 +184,7 @@ def get_aws_connection_info(module, boto3=None):
         if os.environ.get('AWS_DEFAULT_PROFILE'):
             profile_name = os.environ.get('AWS_DEFAULT_PROFILE')
 
-    if profile_name and (access_key or secret_key or security_token):
+    if profile_name and (access_key or secret_key or session_token):
         module.fail_json(msg="Passing both a profile and access tokens is not supported.")
 
     if not endpoint_url:
@@ -194,45 +194,54 @@ def get_aws_connection_info(module, boto3=None):
             endpoint_url = os.environ['EC2_URL']
 
     if not access_key:
+        # AWS_ACCESS_KEY_ID is the one supported by the AWS CLI
+        # AWS_ACCESS_KEY is to match up with our parameter name
         if os.environ.get('AWS_ACCESS_KEY_ID'):
             access_key = os.environ['AWS_ACCESS_KEY_ID']
         elif os.environ.get('AWS_ACCESS_KEY'):
             access_key = os.environ['AWS_ACCESS_KEY']
+        # Deprecated - 'EC2' implies just EC2, but is global
         elif os.environ.get('EC2_ACCESS_KEY'):
             access_key = os.environ['EC2_ACCESS_KEY']
         else:
             # in case access_key came in as empty string
             access_key = None
 
     if not secret_key:
+        # AWS_SECRET_ACCESS_KEY is the one supported by the AWS CLI
+        # AWS_SECRET_KEY is to match up with our parameter name
         if os.environ.get('AWS_SECRET_ACCESS_KEY'):
             secret_key = os.environ['AWS_SECRET_ACCESS_KEY']
         elif os.environ.get('AWS_SECRET_KEY'):
             secret_key = os.environ['AWS_SECRET_KEY']
+        # Deprecated - 'EC2' implies just EC2, but is global
         elif os.environ.get('EC2_SECRET_KEY'):
             secret_key = os.environ['EC2_SECRET_KEY']
         else:
             # in case secret_key came in as empty string
             secret_key = None
 
-    if not security_token:
-        if os.environ.get('AWS_SECURITY_TOKEN'):
-            security_token = os.environ['AWS_SECURITY_TOKEN']
-        elif os.environ.get('AWS_SESSION_TOKEN'):
-            security_token = os.environ['AWS_SESSION_TOKEN']
+    if not session_token:
+        # AWS_SESSION_TOKEN is supported by the AWS CLI
+        if os.environ.get('AWS_SESSION_TOKEN'):
+            session_token = os.environ['AWS_SESSION_TOKEN']
+        # Deprecated - boto
+        elif os.environ.get('AWS_SECURITY_TOKEN'):
+            session_token = os.environ['AWS_SECURITY_TOKEN']
+        # Deprecated - 'EC2' implies just EC2, but is global
         elif os.environ.get('EC2_SECURITY_TOKEN'):
-            security_token = os.environ['EC2_SECURITY_TOKEN']
+            session_token = os.environ['EC2_SECURITY_TOKEN']
         else:
             # in case secret_token came in as empty string
-            security_token = None
+            session_token = None
 
     if not ca_bundle:
         if os.environ.get('AWS_CA_BUNDLE'):
             ca_bundle = os.environ.get('AWS_CA_BUNDLE')
 
     boto_params = dict(aws_access_key_id=access_key,
                        aws_secret_access_key=secret_key,
-                       aws_session_token=security_token)
+                       aws_session_token=session_token)
 
     if profile_name:
         boto_params = dict(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None)