Merge branch 'dev-eks_fargate_profile' of https://github.com/tjarra/c…

…ommunity.aws into dev-eks_fargate_profile

changelogs/fragments/721-wafv2_web_acl.yml

@@ -0,0 +1,3 @@
+minor_changes:
+- wafv2_web_acl - Extended the wafv2_web_acl module to also take the ``custom_response_bodies`` argument (https://github.com/ansible-collections/community.aws/pull/721).
+- wafv2_web_acl - Documentation updates wafv2_web_acl and aws_waf_web_acl (https://github.com/ansible-collections/community.aws/pull/721).

changelogs/fragments/790-ec2_lc-add-throughput-param-support.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- ec2_lc - add support for throughput parameter (https://github.com/ansible-collections/community.aws/pull/790).

changelogs/fragments/825-fix-elb-wait.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - elb_instance - `wait` parameter is no longer ignored (https://github.com/ansible-collections/community.aws/pull/826)

changelogs/fragments/843-add_aws_secret_resource_policy_support.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- aws_secret - Add ``resource_policy`` parameter (https://github.com/ansible-collections/community.aws/pull/843).

changelogs/fragments/857-lambda-wait-before.yml

@@ -0,0 +1,3 @@
+bugfixes: 
+  - execute_lambda - Wait for Lambda function State = Active before executing (https://github.com/ansible-collections/community.aws/pull/857)
+  - lambda - Wait for Lambda function State = Active & LastUpdateStatus = Successful before updating (https://github.com/ansible-collections/community.aws/pull/857)

changelogs/fragments/865-add-backoff-retry-logic-route53_zone.yml

@@ -0,0 +1,3 @@
+bugfixes:
+  - Add backoff retry logic to route53_zone (https://github.com/ansible-collections/community.aws/pull/865).
+  - Add backoff retry logic to route53_info (https://github.com/ansible-collections/community.aws/pull/865).

changelogs/fragments/872-ec2_placement_group_partition_strategy.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- ec2_placement_group - add support for partition strategy and partition count (https://github.com/ansible-collections/community.aws/pull/872).

changelogs/fragments/878-fix-iops-updates-rds.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - rds_instance - Fix updates of ``iops`` or ``allocated_storage`` for ``io1`` DB instances when only one value is changing (https://github.com/ansible-collections/community.aws/pull/878).

changelogs/fragments/880-add-table-class-param.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - dynamodb_table - the ``table_class`` parameter has been added (https://github.com/ansible-collections/community.aws/pull/880).

changelogs/fragments/881-cloudfront-bug.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - cloudfront_distribution - Dont pass ``s3_origin_access_identity_enabled`` to API request (https://github.com/ansible-collections/community.aws/pull/881).

changelogs/fragments/893-refactor-iam_managed_policy.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - iam_managed_policy - refactor module adding ``check_mode`` and better AWSRetry backoff logic (https://github.com/ansible-collections/community.aws/pull/893).

changelogs/fragments/894-add-check_mode-elb_application_lb.yml

@@ -0,0 +1,3 @@
+minor_changes:
+  - elb_application_lb - add check_mode support and refactor integration tests (https://github.com/ansible-collections/community.aws/pull/894)
+  - elb_application_lb_info - update documentation and refactor integration tests (https://github.com/ansible-collections/community.aws/pull/894)

changelogs/fragments/898-boto-removal.yaml

@@ -0,0 +1,6 @@
+breaking_changes:
+- script_inventory_ec2 - The ec2.py inventory script has been moved to a new repository.
+  The script can now be downloaded from https://github.com/ansible-community/contrib-scripts/blob/main/inventory/ec2.py and has been removed from this collection.
+  We recommend migrating from the script to the amazon.aws.ec2 inventory plugin.  (https://github.com/ansible-collections/community.aws/pull/898)
+- community.aws collection - The ``community.aws`` collection has now dropped support for and any requirements upon the original ``boto`` AWS SDK, and now uses the ``boto3``/``botocore`` AWS SDK
+  (https://github.com/ansible-collections/community.aws/pull/898).

changelogs/fragments/913-tg-dereg-conn-param.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - elb_target_group - add support for parameter ``deregistration_connection_termination`` (https://github.com/ansible-collections/community.aws/pull/913).

changelogs/fragments/917-add-launch-template-metadata-parameters.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- ec2_launch_template - Add metadata options parameter ``http_protocol_ipv6`` and ``instance_metadata_tags`` (https://github.com/ansible-collections/community.aws/pull/917).

changelogs/fragments/936-stabilize-ec2-eip.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - ec2_eip - refactor module by fixing check_mode and more clear return obj. added integration tests (https://github.com/ansible-collections/community.aws/pull/936)

galaxy.yml

@@ -1,6 +1,6 @@
 namespace: community
 name: aws
-version: 3.0.1
+version: 4.0.0-dev0
 readme: README.md
 authors:
   - Ansible (https://github.com/ansible)

plugins/module_utils/sns.py

@@ -0,0 +1,125 @@
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+import re
+import copy
+
+try:
+    import botocore
+except ImportError:
+    pass  # handled by AnsibleAWSModule
+
+from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
+from ansible_collections.amazon.aws.plugins.module_utils.ec2 import AWSRetry
+from ansible_collections.amazon.aws.plugins.module_utils.ec2 import camel_dict_to_snake_dict
+
+
+@AWSRetry.jittered_backoff()
+def _list_topics_with_backoff(client):
+    paginator = client.get_paginator('list_topics')
+    return paginator.paginate().build_full_result()['Topics']
+
+
+@AWSRetry.jittered_backoff(catch_extra_error_codes=['NotFound'])
+def _list_topic_subscriptions_with_backoff(client, topic_arn):
+    paginator = client.get_paginator('list_subscriptions_by_topic')
+    return paginator.paginate(TopicArn=topic_arn).build_full_result()['Subscriptions']
+
+
+@AWSRetry.jittered_backoff(catch_extra_error_codes=['NotFound'])
+def _list_subscriptions_with_backoff(client):
+    paginator = client.get_paginator('list_subscriptions')
+    return paginator.paginate().build_full_result()['Subscriptions']
+
+
+def list_topic_subscriptions(client, module, topic_arn):
+    try:
+        return _list_topic_subscriptions_with_backoff(client, topic_arn)
+    except is_boto3_error_code('AuthorizationError'):
+        try:
+            # potentially AuthorizationError when listing subscriptions for third party topic
+            return [sub for sub in _list_subscriptions_with_backoff(client)
+                    if sub['TopicArn'] == topic_arn]
+        except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
+            module.fail_json_aws(e, msg="Couldn't get subscriptions list for topic %s" % topic_arn)
+    except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:  # pylint: disable=duplicate-except
+        module.fail_json_aws(e, msg="Couldn't get subscriptions list for topic %s" % topic_arn)
+
+
+def list_topics(client, module):
+    try:
+        topics = _list_topics_with_backoff(client)
+    except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
+        module.fail_json_aws(e, msg="Couldn't get topic list")
+    return [t['TopicArn'] for t in topics]
+
+
+def topic_arn_lookup(client, module, name):
+    # topic names cannot have colons, so this captures the full topic name
+    all_topics = list_topics(client, module)
+    lookup_topic = ':%s' % name
+    for topic in all_topics:
+        if topic.endswith(lookup_topic):
+            return topic
+
+
+def compare_delivery_policies(policy_a, policy_b):
+    _policy_a = copy.deepcopy(policy_a)
+    _policy_b = copy.deepcopy(policy_b)
+    # AWS automatically injects disableSubscriptionOverrides if you set an
+    # http policy
+    if 'http' in policy_a:
+        if 'disableSubscriptionOverrides' not in policy_a['http']:
+            _policy_a['http']['disableSubscriptionOverrides'] = False
+    if 'http' in policy_b:
+        if 'disableSubscriptionOverrides' not in policy_b['http']:
+            _policy_b['http']['disableSubscriptionOverrides'] = False
+    comparison = (_policy_a != _policy_b)
+    return comparison
+
+
+def canonicalize_endpoint(protocol, endpoint):
+    # AWS SNS expects phone numbers in
+    # and canonicalizes to E.164 format
+    # See <https://docs.aws.amazon.com/sns/latest/dg/sms_publish-to-phone.html>
+    if protocol == 'sms':
+        return re.sub('[^0-9+]*', '', endpoint)
+    return endpoint
+
+
+def get_info(connection, module, topic_arn):
+    name = module.params.get('name')
+    topic_type = module.params.get('topic_type')
+    state = module.params.get('state')
+    subscriptions = module.params.get('subscriptions')
+    purge_subscriptions = module.params.get('purge_subscriptions')
+    subscriptions_existing = module.params.get('subscriptions_existing', [])
+    subscriptions_deleted = module.params.get('subscriptions_deleted', [])
+    subscriptions_added = module.params.get('subscriptions_added', [])
+    subscriptions_added = module.params.get('subscriptions_added', [])
+    topic_created = module.params.get('topic_created', False)
+    topic_deleted = module.params.get('topic_deleted', False)
+    attributes_set = module.params.get('attributes_set', [])
+    check_mode = module.check_mode
+
+    info = {
+        'name': name,
+        'topic_type': topic_type,
+        'state': state,
+        'subscriptions_new': subscriptions,
+        'subscriptions_existing': subscriptions_existing,
+        'subscriptions_deleted': subscriptions_deleted,
+        'subscriptions_added': subscriptions_added,
+        'subscriptions_purge': purge_subscriptions,
+        'check_mode': check_mode,
+        'topic_created': topic_created,
+        'topic_deleted': topic_deleted,
+        'attributes_set': attributes_set,
+    }
+    if state != 'absent':
+        if topic_arn in list_topics(connection, module):
+            info.update(camel_dict_to_snake_dict(connection.get_topic_attributes(TopicArn=topic_arn)['Attributes']))
+            info['delivery_policy'] = info.pop('effective_delivery_policy')
+        info['subscriptions'] = [camel_dict_to_snake_dict(sub) for sub in list_topic_subscriptions(connection, module, topic_arn)]
+
+    return info

plugins/modules/aws_acm.py

@@ -25,18 +25,22 @@
 __metaclass__ = type
 
 
-DOCUMENTATION = '''
+DOCUMENTATION = r'''
+---
 module: aws_acm
-short_description: Upload and delete certificates in the AWS Certificate Manager service
+short_description: >
+  Upload and delete certificates in the AWS Certificate Manager service
 version_added: 1.0.0
 description:
-  - Import and delete certificates in Amazon Web Service's Certificate Manager (AWS ACM).
+  - >
+    Import and delete certificates in Amazon Web Service's Certificate
+    Manager (AWS ACM).
   - >
     This module does not currently interact with AWS-provided certificates.
     It currently only manages certificates provided to AWS by the user.
-  - The ACM API allows users to upload multiple certificates for the same domain name,
-    and even multiple identical certificates.
-    This module attempts to restrict such freedoms, to be idempotent, as per the Ansible philosophy.
+  - The ACM API allows users to upload multiple certificates for the same domain
+    name, and even multiple identical certificates. This module attempts to
+    restrict such freedoms, to be idempotent, as per the Ansible philosophy.
     It does this through applying AWS resource "Name" tags to ACM certificates.
   - >
     When I(state=present),
@@ -57,63 +61,71 @@
     this task will fail.
   - >
     When I(state=absent) and I(certificate_arn) is defined,
-    this module will delete the ACM resource with that ARN if it exists in this region,
-    and succeed without effect if it doesn't exist.
+    this module will delete the ACM resource with that ARN if it exists in this
+    region, and succeed without effect if it doesn't exist.
   - >
-    When I(state=absent) and I(domain_name) is defined,
-    this module will delete all ACM resources in this AWS region with a corresponding domain name.
+    When I(state=absent) and I(domain_name) is defined, this module will delete
+    all ACM resources in this AWS region with a corresponding domain name.
     If there are none, it will succeed without effect.
   - >
     When I(state=absent) and I(certificate_arn) is not defined,
-    and I(domain_name) is not defined,
-    this module will delete all ACM resources in this AWS region with a corresponding I(Name) tag.
+    and I(domain_name) is not defined, this module will delete all ACM resources
+    in this AWS region with a corresponding I(Name) tag.
     If there are none, it will succeed without effect.
-  - Note that this may not work properly with keys of size 4096 bits, due to a limitation of the ACM API.
+  - >
+    Note that this may not work properly with keys of size 4096 bits, due to a
+    limitation of the ACM API.
 options:
   certificate:
     description:
       - The body of the PEM encoded public certificate.
       - Required when I(state) is not C(absent).
-      - If your certificate is in a file, use C(lookup('file', 'path/to/cert.pem')).
+      - >
+        If your certificate is in a file,
+        use C(lookup('file', 'path/to/cert.pem')).
     type: str
-
   certificate_arn:
     description:
       - The ARN of a certificate in ACM to delete
       - Ignored when I(state=present).
-      - If I(state=absent), you must provide one of I(certificate_arn), I(domain_name) or I(name_tag).
+      - >
+        If I(state=absent), you must provide one of
+        I(certificate_arn), I(domain_name) or I(name_tag).
       - >
         If I(state=absent) and no resource exists with this ARN in this region,
         the task will succeed with no effect.
       - >
-        If I(state=absent) and the corresponding resource exists in a different region,
-        this task may report success without deleting that resource.
+        If I(state=absent) and the corresponding resource exists in a different
+        region, this task may report success without deleting that resource.
     type: str
     aliases: [arn]
-
   certificate_chain:
     description:
       - The body of the PEM encoded chain for your certificate.
-      - If your certificate chain is in a file, use C(lookup('file', 'path/to/chain.pem')).
+      - >
+        If your certificate chain is in a file,
+        use C(lookup('file', 'path/to/chain.pem')).
       - Ignored when I(state=absent)
     type: str
-
   domain_name:
     description:
       - The domain name of the certificate.
       - >
         If I(state=absent) and I(domain_name) is specified,
         this task will delete all ACM certificates with this domain.
-      - Exactly one of I(domain_name), I(name_tag)  and I(certificate_arn) must be provided.
+      - >
+        Exactly one of I(domain_name), I(name_tag) and I(certificate_arn)
+        must be provided.
       - >
         If I(state=present) this must not be specified.
         (Since the domain name is encoded within the public certificate's body.)
     type: str
     aliases: [domain]
-
   name_tag:
     description:
-      - The unique identifier for tagging resources using AWS tags, with key I(Name).
+      - >
+        The unique identifier for tagging resources using AWS tags,
+        with key I(Name).
       - This can be any set of characters accepted by AWS for tag values.
       - >
         This is to ensure Ansible can treat certificates idempotently,
@@ -124,15 +136,15 @@
         I(certificate_arn), I(domain_name) or I(name_tag).
     type: str
     aliases: [name]
-
   private_key:
     description:
       - The body of the PEM encoded private key.
       - Required when I(state=present).
       - Ignored when I(state=absent).
-      - If your private key is in a file, use C(lookup('file', 'path/to/key.pem')).
+      - >
+        If your private key is in a file,
+        use C(lookup('file', 'path/to/key.pem')).
     type: str
-
   state:
     description:
       - >
@@ -148,8 +160,9 @@
 author:
   - Matthew Davis (@matt-telstra) on behalf of Telstra Corporation Limited
 extends_documentation_fragment:
-- amazon.aws.aws
-- amazon.aws.ec2
+  - amazon.aws.aws
+  - amazon.aws.ec2
+
 '''
 
 EXAMPLES = '''

plugins/modules/aws_direct_connect_virtual_interface.py

@@ -404,7 +404,7 @@ def create_vi(client, public, associated_id, creation_params):
         :param public: a boolean
         :param associated_id: a link aggregation group ID or connection ID to associate
                               with the virtual interface.
-        :param creation_params: a dict of parameters to use in the boto call
+        :param creation_params: a dict of parameters to use in the AWS SDK call
         :return The ID of the created virtual interface
     '''
     err_msg = "Failed to create virtual interface"

plugins/modules/aws_glue_job.py

@@ -245,9 +245,7 @@
 
 from ansible_collections.amazon.aws.plugins.module_utils.core import AnsibleAWSModule
 from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
-from ansible_collections.amazon.aws.plugins.module_utils.ec2 import ansible_dict_to_boto3_tag_list
 from ansible_collections.amazon.aws.plugins.module_utils.ec2 import AWSRetry
-from ansible_collections.amazon.aws.plugins.module_utils.ec2 import boto3_tag_list_to_ansible_dict
 from ansible_collections.amazon.aws.plugins.module_utils.ec2 import compare_aws_tags
 from ansible_collections.amazon.aws.plugins.module_utils.iam import get_aws_account_info