Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3_sync module does not function with bucket shared across multiple was accounts. #58

Closed
queglay opened this issue Apr 27, 2020 · 4 comments · Fixed by #1330
Closed
Labels
affects_2.10 bug This issue/PR relates to a bug has_pr module module needs_verified plugins plugin (any type) traceback

Comments

@queglay
Copy link

queglay commented Apr 27, 2020

SUMMARY

Doing the s3 sync operation in a shell, as the same user, has no problem, but the s3_sync module in ansible errors with:

4899 MODULE FAILURE
4900 See stdout/stderr for the exact error
4901 MODULE_STDERR:
4902 Traceback (most recent call last):
4903   File "<stdin>", line 102, in <module>
4904   File "<stdin>", line 94, in _ansiballz_main
4905   File "<stdin>", line 40, in invoke_module
4906   File "/usr/lib/python2.7/runpy.py", line 188, in run_module
4907     fname, loader, pkg_name)
4908   File "/usr/lib/python2.7/runpy.py", line 82, in _run_module_code
4909     mod_name, mod_fname, mod_loader, pkg_name)
4910   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
4911     exec code in run_globals
4912   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 544, in <module>
4913   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 526, in main
4914   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 405, in filter_list
4915   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 390, in head_s3
4916 Exception: An error occurred (403) when calling the HeadObject operation: Forbidden
ISSUE TYPE

Bug Report
COMPONENT NAME

s3_sync

ANSIBLE VERSION

ansible 2.9.7
CONFIGURATION

ANSIBLE_FORCE_COLOR(env: ANSIBLE_FORCE_COLOR) = True
ANSIBLE_PIPELINING(/deployuser/ansible.cfg) = True
ANSIBLE_SSH_RETRIES(/deployuser/ansible.cfg) = 10
DEFAULT_CALLBACK_WHITELIST(/deployuser/ansible.cfg) = [u'profile_tasks']
DEFAULT_GATHER_SUBSET(/deployuser/ansible.cfg) = [u'!hardware # this line may help deal with an issue where a bad nfs mount will prevent ansible from connecting
DEFAULT_HOST_LIST(env: ANSIBLE_INVENTORY) = [u'/vagrant/ansible/hosts']
DEFAULT_LOAD_CALLBACK_PLUGINS(/deployuser/ansible.cfg) = True
DEFAULT_LOG_PATH(/deployuser/ansible.cfg) = /deployuser/tmp/ansible_log
DEFAULT_STDOUT_CALLBACK(env: ANSIBLE_STDOUT_CALLBACK) = debug

OS / ENVIRONMENT

Ununtu 16.04

STEPS TO REPRODUCE

In this example I show the playbook which can run the s3 sync operation as a shell fine, but using the ansible module fails.

The playbook:

- name: Get the current caller identity information
  aws_caller_info:
  register: caller_info
  become_user: deadlineuser

- name: Sync deadline to s3
  shell: |
    set -x
    aws sts get-caller-identity
    cd {{ deadline_linux_installers_tar | dirname }}/
    aws s3 sync . s3://{{ installers_bucket }}/ --exclude "*" --include "{{ deadline_linux_installers_tar | basename }}"
  become_user: deadlineuser
  tags:
  - install
  - sync_installers

- name: "Ensure deadline {{ deadline_linux_installers_tar | dirname }}/{{ deadline_linux_installers_tar | basename }} exists in the s3 bucket {{ installers_bucket }} - Push if it doesn't."
  s3_sync:
    bucket: "{{ installers_bucket }}"
    file_root: "{{ deadline_linux_installers_tar | dirname }}"
    include: "{{ deadline_linux_installers_tar | basename }}"
    mode: push
  become_user: deadlineuser
  tags:
  - install
  - sync_installers

The only thing slightly diffferent in this scenario to others I have had success with is that the bucket allows access to two AWS accounts. the pemissions on the bucket are:

{
    "Version": "2012-10-17",
    "Id": "s3ProdDevSharePolicy",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::254735172:root",
                    "arn:aws:iam::326573574:root"
                ]
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::software.firehawkvfx.com",
                "arn:aws:s3:::software.firehawkvfx.com/*"
            ]
        }
    ]
}

I have replaced the user id's in this log with a random number, but in both tests, they match.

EXPECTED RESULTS

s3_sync should function the same as the shell command.

ACTUAL RESULTS

Here is the error:

4863 TASK [deadlinedb : Get the current caller identity information] ****************
4864 Monday 27 April 2020  13:32:44 +0930 (0:00:01.728)       0:00:06.116 ********** 
4865 ok: [firehawkgateway] => {
4866     "account": "254735172", 
4867     "arn": "arn:aws:iam::254735172:user/storage_user", 
4868     "changed": false, 
4869     "user_id": "DSFHSDFJSFGJSFGJKSFGJ"
4870 }
4871 TASK [deadlinedb : Sync deadline to s3] ****************************************
4872 Monday 27 April 2020  13:32:47 +0930 (0:00:03.204)       0:00:09.320 ********** 
4873 changed: [firehawkgateway] => {
4874     "changed": true, 
4875     "cmd": "set -x\naws sts get-caller-identity\ncd /deployuser/downloads/\naws s3 sync . s3://software.firehawkvfx.com/ --exclude \"*\" --include \"Deadline-10.1.1.3-linux-installers.tar\"\n", 
4876     "delta": "0:00:02.143070", 
4877     "end": "2020-04-27 13:32:50.163394", 
4878     "rc": 0, 
4879     "start": "2020-04-27 13:32:48.020324"
4880 }
4881 STDOUT:
4882 {
4883     "Account": "254735172", 
4884     "UserId": "DSFHSDFJSFGJSFGJKSFGJ", 
4885     "Arn": "arn:aws:iam::254735172:user/storage_user"
4886 }
4887 STDERR:
4888 + aws sts get-caller-identity
4889 + cd /deployuser/downloads/
4890 + aws s3 sync . s3://software.firehawkvfx.com/ --exclude * --include Deadline-10.1.1.3-linux-installers.tar
4891 TASK [deadlinedb : Ensure deadline /deployuser/downloads exists in the s3 bucket software.firehawkvfx.com - Push if it doesn't.] ***
4892 Monday 27 April 2020  13:32:49 +0930 (0:00:02.296)       0:00:11.616 ********** 
4893 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Exception: An error occurred (403) when calling the HeadObject operation: Forbidden
4894 fatal: [firehawkgateway]: FAILED! => {
4895     "changed": false, 
4896     "rc": 1
4897 }
4898 MSG:
4899 MODULE FAILURE
4900 See stdout/stderr for the exact error
4901 MODULE_STDERR:
4902 Traceback (most recent call last):
4903   File "<stdin>", line 102, in <module>
4904   File "<stdin>", line 94, in _ansiballz_main
4905   File "<stdin>", line 40, in invoke_module
4906   File "/usr/lib/python2.7/runpy.py", line 188, in run_module
4907     fname, loader, pkg_name)
4908   File "/usr/lib/python2.7/runpy.py", line 82, in _run_module_code
4909     mod_name, mod_fname, mod_loader, pkg_name)
4910   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
4911     exec code in run_globals
4912   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 544, in <module>
4913   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 526, in main
4914   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 405, in filter_list
4915   File "/tmp/ansible_s3_sync_payload_91htk3/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 390, in head_s3
4916 Exception: An error occurred (403) when calling the HeadObject operation: Forbidden

You can see that the bash shell operation has no problem, only the s3_sync module does.

@ansibullbot
Copy link

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot
Copy link

@tremble
Copy link
Contributor

tremble commented Jul 9, 2022

Thanks for taking the time to open this issue. I'm sorry it's taken so long to respond.

The issue is actually more generic than being "cross-account", we attempt to describe the existing file prior to uploading, if permissions don't allow HEAD calls to the bucket/file then the upload currently fails.

#1330 includes a change which should fix the issue you're seeing.

@ansibullbot
Copy link

softwarefactory-project-zuul bot pushed a commit that referenced this issue Jul 9, 2022
s3_sync - Improve error handling when testing for existing files

SUMMARY
fixes: #58

Simplifies handling of '404' codes (use is_boto3_error_code)
Assume 403 files need updating (it's the best we can do, and mimics aws cli)
Allows Boto3 exceptions to fall through to the outer try/except clause and cleanly fail rather than rethrowing it as an Exception()

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
s3_sync
ADDITIONAL INFORMATION

Reviewed-by: Joseph Torcasso <None>
patchback bot pushed a commit that referenced this issue Jul 9, 2022
s3_sync - Improve error handling when testing for existing files

SUMMARY
fixes: #58

Simplifies handling of '404' codes (use is_boto3_error_code)
Assume 403 files need updating (it's the best we can do, and mimics aws cli)
Allows Boto3 exceptions to fall through to the outer try/except clause and cleanly fail rather than rethrowing it as an Exception()

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
s3_sync
ADDITIONAL INFORMATION

Reviewed-by: Joseph Torcasso <None>
(cherry picked from commit 40984a4)
patchback bot pushed a commit that referenced this issue Jul 9, 2022
s3_sync - Improve error handling when testing for existing files

SUMMARY
fixes: #58

Simplifies handling of '404' codes (use is_boto3_error_code)
Assume 403 files need updating (it's the best we can do, and mimics aws cli)
Allows Boto3 exceptions to fall through to the outer try/except clause and cleanly fail rather than rethrowing it as an Exception()

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
s3_sync
ADDITIONAL INFORMATION

Reviewed-by: Joseph Torcasso <None>
(cherry picked from commit 40984a4)
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jul 10, 2022
…) (#1335)

[PR #1330/40984a40 backport][stable-4] s3_sync - Improve error handling when testing for existing files

This is a backport of PR #1330 as merged into main (40984a4).
SUMMARY
fixes: #58

Simplifies handling of '404' codes (use is_boto3_error_code)
Assume 403 files need updating (it's the best we can do, and mimics aws cli)
Allows Boto3 exceptions to fall through to the outer try/except clause and cleanly fail rather than rethrowing it as an Exception()

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
s3_sync
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell <None>
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jul 10, 2022
…) (#1334)

[PR #1330/40984a40 backport][stable-3] s3_sync - Improve error handling when testing for existing files

This is a backport of PR #1330 as merged into main (40984a4).
SUMMARY
fixes: #58

Simplifies handling of '404' codes (use is_boto3_error_code)
Assume 403 files need updating (it's the best we can do, and mimics aws cli)
Allows Boto3 exceptions to fall through to the outer try/except clause and cleanly fail rather than rethrowing it as an Exception()

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
s3_sync
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects_2.10 bug This issue/PR relates to a bug has_pr module module needs_verified plugins plugin (any type) traceback
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants