Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[s3_object] An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied #2114

Closed
1 task done
dhoppe opened this issue May 28, 2024 · 4 comments · Fixed by #2206
Closed
1 task done

[s3_object] An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied #2114

dhoppe opened this issue May 28, 2024 · 4 comments · Fixed by #2206
Labels
jira

Comments

@dhoppe
Copy link

dhoppe commented May 28, 2024

Summary

When I try to get a presigned URL for a specific version of an S3 object, the Ansible playbook fails. If I remove the version everything works fine.

Since the following command works fine, I can rule out missing permissions.

aws s3api get-object-tagging --bucket bt-installer --key public/aws-cli/awscli-exe-linux-x86_64.zip
{
    “VersionId”: “zpKl8rWJVw.dFnAm9fWzmpI4rDbfkSno”,
    “TagSet”: []
}

Since this is an S3 bucket including ACL and Access for other AWS accounts, I suspect that the get_object_tagging function is missing the ExpectedBucketOwner parameter.

https://botocore.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/get_object_tagging.html

Issue Type

Bug Report

Component Name

s3_object

Ansible Version

$ ansible --version
ansible [core 2.15.6]
  config file = /home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg
  configured module search path = ['/home/dennis.hoppe.ext/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/dennis.hoppe.ext/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
# /usr/share/ansible/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               6.5.4
ansible.netcommon                        5.3.0
ansible.posix                            1.5.4
ansible.utils                            3.1.0
ansible.windows                          2.3.0
community.aws                            6.4.0
community.crypto                         2.19.1
community.docker                         3.9.0
community.general                        8.6.0
community.library_inventory_filtering_v1 1.0.1
community.windows                        2.2.0
crowdstrike.falcon                       4.4.0

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/lib/python3/dist-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.34.113
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.34.113
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
ANSIBLE_FORCE_COLOR(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = True
CONFIG_FILE() = /home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg
DEFAULT_CALLBACK_PLUGIN_PATH(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = ['/usr/share/ansible/plugins/callback', '/home/dennis.hoppe.ext/bt-cloudops-ansible/plugins/c>
DEFAULT_FILTER_PLUGIN_PATH(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = ['/usr/share/ansible/plugins/filter', '/home/dennis.hoppe.ext/bt-cloudops-ansible/plugins/filte>
DEFAULT_FORKS(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = 50
DEFAULT_HOST_LIST(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = ['/home/dennis.hoppe.ext/bt-cloudops-ansible/aws_ec2.yml']
DEFAULT_LOAD_CALLBACK_PLUGINS(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = True
DEFAULT_ROLES_PATH(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = ['/home/dennis.hoppe.ext/bt-cloudops-ansible/roles']
DEFAULT_STDOUT_CALLBACK(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = community.general.yaml
DEFAULT_TIMEOUT(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = 30
DEFAULT_VARS_PLUGIN_PATH(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = ['/usr/share/ansible/plugins/vars', '/home/dennis.hoppe.ext/bt-cloudops-ansible/plugins/vars']
DEFAULT_VAULT_PASSWORD_FILE(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = /home/dennis.hoppe.ext/bt-cloudops-ansible/vault_pass.sh
DISPLAY_SKIPPED_HOSTS(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = False
INVENTORY_ENABLED(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = ['amazon.aws.aws_ec2', 'host_list']
WIN_ASYNC_STARTUP_TIMEOUT(/home/dennis.hoppe.ext/bt-cloudops-ansible/ansible.cfg) = 10

OS / Environment

Ubuntu 22.04

Steps to Reproduce

- name: Get presigned URL
  amazon.aws.s3_object:
    bucket: "{{ role_bt_common_get_packages_s3_bucket }}"
    object: "{{ role_bt_common_get_packages_s3_dir }}{{ local_bt_common_get_package_config.dir | default('') }}/{{ local_bt_common_get_package_config.file }}"
    version: "{{ local_bt_common_get_package_config.s3_version | default(omit) }}"
    region: "{{ role_bt_common_get_packages_s3_region }}"
    mode: geturl
  delegate_to: localhost
  register: result_s3_object

Expected Results

I expect to receive a presigned URL for a specific version of an S3 object.

Actual Results

The full traceback is:
Traceback (most recent call last):
  File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1716908566.518546-1479237-165880899412666/AnsiballZ_s3_object.py", line 107, in <module>
    _ansiballz_main()
  File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1716908566.518546-1479237-165880899412666/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1716908566.518546-1479237-165880899412666/AnsiballZ_s3_object.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
  File "/usr/lib/python3.10/runpy.py", line 224, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1535, in <module>
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1522, in main
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1262, in s3_object_do_geturl
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 952, in get_current_object_tags_dict
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py", line 105, in deciding_wrapper
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
  File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 1021, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied
fatal: [i-05fbc5cb84deeea26_asg-metadefender-image-ubuntu22-mdtest-202405281455 -> localhost]: FAILED! => changed=false
  module_stderr: |-
    Traceback (most recent call last):
      File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1716908566.518546-1479237-165880899412666/AnsiballZ_s3_object.py", line 107, in <module>
        _ansiballz_main()
      File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1716908566.518546-1479237-165880899412666/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1716908566.518546-1479237-165880899412666/AnsiballZ_s3_object.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
      File "/usr/lib/python3.10/runpy.py", line 224, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1535, in <module>
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1522, in main
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1262, in s3_object_do_geturl
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 952, in get_current_object_tags_dict
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py", line 105, in deciding_wrapper
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
      File "/tmp/ansible_amazon.aws.s3_object_payload_d_78h5il/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
      File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 565, in _api_call
        return self._make_api_call(operation_name, kwargs)
      File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 1021, in _make_api_call
        raise error_class(parsed_response, operation_name)
    botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@yarons
Copy link

yarons commented Jun 17, 2024 •
edited
Loading

I'm also experiencing the same behavior, but even without specifying the version. Is there a way I can contribute to this issue?

@alinabuzachis alinabuzachis mentioned this issue Jul 30, 2024
Merged
@alinabuzachis
Copy link
Collaborator

alinabuzachis commented Jul 30, 2024 •
edited
Loading

@dhoppe @yarons Can you please let me know if this PR solves your issue https://github.com/ansible-collections/amazon.aws/pull/2206/files? Thanks.

@dhoppe I would also suggest to use the latest version of the amazon.aws collection with is 8.1.0.

@alinabuzachis
Copy link
Collaborator

alinabuzachis commented Jul 30, 2024 •
edited
Loading

Also, can you make sure you query the same bucket and object from the CLI and the Ansible playbook?

@dhoppe
Copy link
Author

dhoppe commented Aug 1, 2024 •
edited
Loading

@alinabuzachis I used version 6.5.4 of the amazon.aws collection and applied the changes manually, but still get an error message.

Same AWS account as S3 bucket:

TASK [bt-common : (get_package/aws-cli-v2) - show information] *****************
ok: [i-0b22ddbf1e38f1e43_Ansible2] =>
  msg: 'get package aws-cli-v2(file: /aws-cli/awscli-exe-linux-x86_64.zip, version: sqyYyd0Mfo2RXW4Qd6I_9nkdzquKPbhQ)'

TASK [bt-common : (linux_get_package/aws-cli-v2) - check existing package] *****
changed: [i-0b22ddbf1e38f1e43_Ansible2]

TASK [bt-common : (linux_get_package/aws-cli-v2) - create directory] ***********
ok: [i-0b22ddbf1e38f1e43_Ansible2]

TASK [bt-common : (linux_get_package/aws-cli-v2) - include method] *************
included: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/get_package_method_http.yml for i-0b22ddbf1e38f1e43_Ansible2

Different AWS account than S3 bucket:

TASK [bt-common : (get_packages) - reset results] ******************************
task path: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/get_packages.yml:7
ok: [i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714] => changed=false
  ansible_facts:
    role_bt_common_get_packages_results: []

TASK [bt-common : (get_packages) - include get_package.yml] ********************
task path: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/get_packages.yml:12
statically imported: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/linux_get_package.yml
statically imported: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/windows_get_package.yml
included: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/get_package.yml for i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714 => (item=aws-cli-v2)

TASK [bt-common : (get_package/aws-cli-v2) - show information] *****************
task path: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/get_package.yml:7
ok: [i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714] =>
  msg: 'get package aws-cli-v2(file: /aws-cli/awscli-exe-linux-x86_64.zip, version: ZxnThJopo8cjYP1m76b4E2wKO3vqhb_2)'
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'echo ~ubuntu && sleep 0'"'"''
<10.142.123.205> (0, b'/home/ubuntu\n', b'')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp `"&& mkdir "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392 `" && echo ansible-tmp-1722583342.418873-2412772-60069411098392="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392 `" ) && sleep 0'"'"''
<10.142.123.205> (0, b'ansible-tmp-1722583342.418873-2412772-60069411098392=/home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392\n', b'')
Using module file /usr/lib/python3/dist-packages/ansible/modules/stat.py
<10.142.123.205> PUT /home/dennis.hoppe.ext/.ansible/tmp/ansible-local-2408463b1y3gc4r/tmpub96x3o1 TO /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392/AnsiballZ_stat.py
<10.142.123.205> SSH: EXEC sftp -b - -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' '[10.142.123.205]'
<10.142.123.205> (0, b'sftp> put /home/dennis.hoppe.ext/.ansible/tmp/ansible-local-2408463b1y3gc4r/tmpub96x3o1 /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392/AnsiballZ_stat.py\n', b'')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392/ /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392/AnsiballZ_stat.py && sleep 0'"'"''
<10.142.123.205> (0, b'', b'')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' -tt 10.142.123.205 '/bin/sh -c '"'"'sudo -H -S -n  -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-vaqkafobugzbqqdjxljmbmzmmwnroixb ; /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392/AnsiballZ_stat.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<10.142.123.205> (0, b'\r\n{"changed": false, "stat": {"exists": false}, "invocation": {"module_args": {"path": "/root/ansible/packages/awscli-exe-linux-x86_64.zip", "checksum_algorithm": "sha256", "follow": false, "get_md5": false, "get_checksum": true, "get_mime": true, "get_attributes": true}}}\r\n', b'Shared connection to 10.142.123.205 closed.\r\n')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.418873-2412772-60069411098392/ > /dev/null 2>&1 && sleep 0'"'"''
<10.142.123.205> (0, b'', b'')

TASK [bt-common : (linux_get_package/aws-cli-v2) - check existing package] *****
task path: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/linux_get_package.yml:2
changed: [i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714] => changed=true
  invocation:
    module_args:
      checksum_algorithm: sha256
      follow: false
      get_attributes: true
      get_checksum: true
      get_md5: false
      get_mime: true
      path: /root/ansible/packages/awscli-exe-linux-x86_64.zip
  stat:
    exists: false
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'echo ~ubuntu && sleep 0'"'"''
<10.142.123.205> (0, b'/home/ubuntu\n', b'')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp `"&& mkdir "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361 `" && echo ansible-tmp-1722583342.9660547-2412805-171167195452361="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361 `" ) && sleep 0'"'"''
<10.142.123.205> (0, b'ansible-tmp-1722583342.9660547-2412805-171167195452361=/home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361\n', b'')
Using module file /usr/lib/python3/dist-packages/ansible/modules/file.py
<10.142.123.205> PUT /home/dennis.hoppe.ext/.ansible/tmp/ansible-local-2408463b1y3gc4r/tmpoqehak43 TO /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361/AnsiballZ_file.py
<10.142.123.205> SSH: EXEC sftp -b - -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' '[10.142.123.205]'
<10.142.123.205> (0, b'sftp> put /home/dennis.hoppe.ext/.ansible/tmp/ansible-local-2408463b1y3gc4r/tmpoqehak43 /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361/AnsiballZ_file.py\n', b'')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361/ /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361/AnsiballZ_file.py && sleep 0'"'"''
<10.142.123.205> (0, b'', b'')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' -tt 10.142.123.205 '/bin/sh -c '"'"'sudo -H -S -n  -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-ganhojraaziproazqlnypywhywyaxfsk ; /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361/AnsiballZ_file.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<10.142.123.205> (0, b'\r\n{"path": "/root/ansible/packages", "changed": true, "diff": {"before": {"path": "/root/ansible/packages", "state": "absent"}, "after": {"path": "/root/ansible/packages", "state": "directory"}}, "uid": 0, "gid": 0, "owner": "root", "group": "root", "mode": "0755", "state": "directory", "size": 4096, "invocation": {"module_args": {"path": "/root/ansible/packages", "state": "directory", "mode": 493, "recurse": false, "force": false, "follow": true, "modification_time_format": "%Y%m%d%H%M.%S", "access_time_format": "%Y%m%d%H%M.%S", "unsafe_writes": false, "_original_basename": null, "_diff_peek": null, "src": null, "modification_time": null, "access_time": null, "owner": null, "group": null, "seuser": null, "serole": null, "selevel": null, "setype": null, "attributes": null}}}\r\n', b'Shared connection to 10.142.123.205 closed.\r\n')
<10.142.123.205> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<10.142.123.205> SSH: EXEC ssh -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey,keyboard-interactive -C -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=accept-new -o KbdInteractiveAuthentication=yes -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=30 -o 'ControlPath="/home/dennis.hoppe.ext/.ansible/cp/6bdf7aa0dc"' 10.142.123.205 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1722583342.9660547-2412805-171167195452361/ > /dev/null 2>&1 && sleep 0'"'"''
<10.142.123.205> (0, b'', b'')

TASK [bt-common : (linux_get_package/aws-cli-v2) - create directory] ***********
task path: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/linux_get_package.yml:24
changed: [i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714] => changed=true
  diff:
    after:
      path: /root/ansible/packages
      state: directory
    before:
      path: /root/ansible/packages
      state: absent
  gid: 0
  group: root
  invocation:
    module_args:
      _diff_peek: null
      _original_basename: null
      access_time: null
      access_time_format: '%Y%m%d%H%M.%S'
      attributes: null
      follow: true
      force: false
      group: null
      mode: 493
      modification_time: null
      modification_time_format: '%Y%m%d%H%M.%S'
      owner: null
      path: /root/ansible/packages
      recurse: false
      selevel: null
      serole: null
      setype: null
      seuser: null
      src: null
      state: directory
      unsafe_writes: false
  mode: '0755'
  owner: root
  path: /root/ansible/packages
  size: 4096
  state: directory
  uid: 0

TASK [bt-common : (linux_get_package/aws-cli-v2) - include method] *************
task path: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/linux_get_package.yml:33
included: /home/dennis.hoppe.ext/bt-cloudops-ansible/roles/bt-common/tasks/get_package_method_http.yml for i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714
The full traceback is:
Traceback (most recent call last):
  File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1722583343.3654213-2412816-200982014619212/AnsiballZ_s3_object.py", line 107, in <module>
    _ansiballz_main()
  File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1722583343.3654213-2412816-200982014619212/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1722583343.3654213-2412816-200982014619212/AnsiballZ_s3_object.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
  File "/usr/lib/python3.10/runpy.py", line 224, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1545, in <module>
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1532, in main
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1271, in s3_object_do_geturl
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 965, in get_current_object_tags_dict
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py", line 105, in deciding_wrapper
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
  File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 1021, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied
fatal: [i-0f1444cdce7106cf8_asg-metadefender-image-ubuntu22-mdtest-202408020714 -> localhost]: FAILED! => changed=false
  module_stderr: |-
    Traceback (most recent call last):
      File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1722583343.3654213-2412816-200982014619212/AnsiballZ_s3_object.py", line 107, in <module>
        _ansiballz_main()
      File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1722583343.3654213-2412816-200982014619212/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/dennis.hoppe.ext/.ansible/tmp/ansible-tmp-1722583343.3654213-2412816-200982014619212/AnsiballZ_s3_object.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
      File "/usr/lib/python3.10/runpy.py", line 224, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1545, in <module>
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1532, in main
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1271, in s3_object_do_geturl
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 965, in get_current_object_tags_dict
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py", line 105, in deciding_wrapper
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
      File "/tmp/ansible_amazon.aws.s3_object_payload_m3n01l8h/ansible_amazon.aws.s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
      File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 565, in _api_call
        return self._make_api_call(operation_name, kwargs)
      File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 1021, in _make_api_call
        raise error_class(parsed_response, operation_name)
    botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

aws s3api get-object-tagging --bucket bt-installer --key public/aws-cli/awscli-exe-linux-x86_64.zip
{
    "VersionId": "sqyYyd0Mfo2RXW4Qd6I_9nkdzquKPbhQ",
    "TagSet": []
}

Just to be sure, I applied the changes to version 8.1.0 of the amazon.aws collection, but still no luck.

Edit: I did some debugging using AWS CloudTrail and realised that I had to add s3:GetBucketOwnershipControls to the S3 bucket policy, but it looks like I still have some misconfiguration regarding the Object Ownership / ACL of the S3 bucket. I will look into that.

aws s3api get-bucket-ownership-controls --bucket bt-installer --expected-bucket-owner 963812274078

An error occurred (OwnershipControlsNotFoundError) when calling the GetBucketOwnershipControls operation: The bucket ownership controls were not found

patchback bot pushed a commit that referenced this issue Aug 27, 2024
@alinabuzachis @patchback
s3_object - Add support for expcted_bucket_owner option. (#2206)
534b1de 
SUMMARY

Closes #2114

ISSUE TYPE

Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
(cherry picked from commit e7db692)
@patchback patchback bot mentioned this issue Aug 27, 2024
Merged
softwarefactory-project-zuul bot pushed a commit that referenced this issue Aug 27, 2024
@patchback
s3_object - Add support for expcted_bucket_owner option. (#2206) (#2250)
e55ddd8 
This is a backport of PR #2206 as merged into main (e7db692).
SUMMARY

Closes #2114

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
braydencw1 pushed a commit to braydencw1/amazon.aws that referenced this issue Aug 29, 2024
@alinabuzachis @braydencw1
s3_object - Add support for expcted_bucket_owner option. (ansible-col…
fe5e65c 
…lections#2206)

SUMMARY

Closes ansible-collections#2114

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
jira
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

s3_object - Add support for expcted_bucket_owner option alinabuzachis/amazon.aws
4 participants
@yarons @dhoppe @hakbailey @alinabuzachis