Addressing Style S3 (ssm_connection) - choose between path, virtual o…

…r auto. (#1633) (#1669)

[PR #1633/8503d80d backport][stable-5] Addressing Style S3 (ssm_connection) - choose between path, virtual or auto.

This is a backport of PR #1633 as merged into main (8503d80).
SUMMARY
Added the chance of setting up the addressing style for S3 URLs, this fixes #637
ISSUE TYPE


Feature Pull Request

COMPONENT NAME
aws_ssm
ADDITIONAL INFORMATION
I came across this issue when I was trying to connect and execute tasks on a fresh AWS set-up (new S3 bucket + EC2), I'm using the latest (5.1.0) release and Ansible 5.10 (as per pip show).
The issue itself is the same as #637 and the last comment is the fix, there's another PR addressing this issue #786 but is quiet since May
Also AWS seems to be deprecating PATH addressing style: https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access
Thanks to timburnet-systematica and james-masson for their initial work on this.

Reviewed-by: Mark Chappell <None>

changelogs/fragments/1633-s3-url-address-style.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - aws_ssm - add ``ansible_aws_ssm_s3_addressing_style`` to allow setting the S3 addressing style (https://github.com/ansible-collections/community.aws/pull/1633).

plugins/connection/aws_ssm.py

@@ -87,10 +87,36 @@
     vars:
     - name: ansible_aws_ssm_document
     version_added: 5.2.0
+  s3_addressing_style:
+    description:
+    - The addressing style to use when using S3 URLs.
+    - When the S3 bucket isn't in the same region as the Instance
+      explicitly setting the addressing style to 'virtual' may be necessary
+      U(https://repost.aws/knowledge-center/s3-http-307-response) as this forces
+      the use of a specific endpoint.
+    choices: [ 'path', 'virtual', 'auto' ]
+    default: 'auto'
+    version_added: 5.2.0
+    vars:
+    - name: ansible_aws_ssm_s3_addressing_style
 '''
 
 EXAMPLES = r'''
 
+# Wait for SSM Agent to be available on the Instance
+- name: Wait for connection to be available
+  vars:
+    ansible_connection: aws_ssm
+    ansible_aws_ssm_bucket_name: nameofthebucket
+    ansible_aws_ssm_region: us-west-2
+    # When the S3 bucket isn't in the same region as the Instance
+    # Explicitly setting the addressing style to 'virtual' may be necessary
+    # https://repost.aws/knowledge-center/s3-http-307-response
+    ansible_aws_ssm_s3_addressing_style: virtual
+  tasks:
+    - name: Wait for connection
+      wait_for_connection:
+
 # Stop Spooler Process on Windows Instances
 - name: Stop Spooler Service on Windows Instances
   vars:
@@ -708,7 +734,10 @@ def _get_boto_client(self, service, region_name=None, profile_name=None):
 
         client = session.client(
             service,
-            config=Config(signature_version="s3v4")
+            config=Config(
+                signature_version="s3v4",
+                s3={'addressing_style': self.get_option('s3_addressing_style')}
+            )
         )
         return client
 

tests/integration/targets/connection_aws_ssm_addressing/aliases

@@ -0,0 +1,4 @@
+time=20m
+
+cloud/aws
+connection_aws_ssm

tests/integration/targets/connection_aws_ssm_addressing/aws_ssm_integration_test_setup.yml

@@ -0,0 +1,9 @@
+- hosts: localhost
+  roles:
+    - role: ../setup_connection_aws_ssm
+      vars:
+        target_os: fedora
+        encrypted_bucket: False
+        s3_bucket_region: 'eu-central-1'
+        s3_addressing_style: virtual
+        test_suffix: addressing

tests/integration/targets/connection_aws_ssm_addressing/aws_ssm_integration_test_teardown.yml

@@ -0,0 +1,5 @@
+- hosts: localhost
+  tasks:
+    - include_role:
+        name: ../setup_connection_aws_ssm
+        tasks_from: cleanup.yml

tests/integration/targets/connection_aws_ssm_addressing/meta/main.yml

@@ -0,0 +1,3 @@
+dependencies:
+  - connection
+  - setup_connection_aws_ssm

tests/integration/targets/connection_aws_ssm_addressing/runme.sh

@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+
+PLAYBOOK_DIR=$(pwd)
+set -eux
+
+CMD_ARGS=("$@")
+
+# Destroy Environment
+cleanup() {
+
+    cd "${PLAYBOOK_DIR}"
+    ansible-playbook -c local aws_ssm_integration_test_teardown.yml "${CMD_ARGS[@]}"
+
+}
+
+trap "cleanup" EXIT
+
+# Setup Environment
+ansible-playbook -c local aws_ssm_integration_test_setup.yml "$@"
+
+# Export the AWS Keys
+set +x
+. ./aws-env-vars.sh
+set -x
+
+cd ../connection
+
+# Execute Integration tests
+INVENTORY="${PLAYBOOK_DIR}/ssm_inventory" ./test.sh \
+    -e target_hosts=aws_ssm \
+    "$@"

tests/integration/targets/connection_aws_ssm_encrypted_s3/aws_ssm_integration_test_setup.yml

@@ -4,3 +4,4 @@
       vars:
         target_os: fedora
         encrypted_bucket: True
+        test_suffix: encrypteds3

tests/integration/targets/connection_aws_ssm_ssm_document/aws_ssm_integration_test_setup.yml

@@ -4,3 +4,4 @@
       vars:
         target_os: fedora
         use_ssm_document: True
+        test_suffix: document

tests/integration/targets/setup_connection_aws_ssm/defaults/main.yml

@@ -40,11 +40,6 @@ ami_details:
             </powershell>
         os_type: windows
 
-# see:
-# - https://github.com/mattclay/aws-terminator/pull/181
-# - https://github.com/ansible-collections/community.aws/pull/763
-encrypted_s3_bucket_name: ssm-encrypted-test-bucket
-
-s3_bucket_name: "{{ resource_prefix }}-connection-ssm"
+s3_bucket_name: "{{ tiny_prefix }}-connection-ssm-{{ test_suffix | default(target_os) }}"
 kms_key_name: "{{ resource_prefix }}-connection-ssm"
 ssm_document_name: "{{ resource_prefix }}-connection-ssm"

tests/integration/targets/setup_connection_aws_ssm/templates/inventory-combined.aws_ssm.j2

@@ -32,6 +32,9 @@ ansible_aws_ssm_plugin=/usr/local/sessionmanagerplugin/bin/session-manager-plugi
 ansible_python_interpreter=/usr/bin/env python3
 local_tmp=/tmp/ansible-local-{{ tiny_prefix }}
 ansible_aws_ssm_bucket_name={{ s3_bucket_name }}
+{% if s3_addressing_style | default(False) %}
+ansible_aws_ssm_s3_addressing_style={{ s3_addressing_style }}
+{% endif %}
 {% if encrypted_bucket | default(False) %}
 {% if not (s3_bucket_encryption | default(False)) %}
 ansible_aws_ssm_bucket_sse_mode='aws:kms'