Update S3_SYNC module to be FIPS complaint

[nkalatho]

Summary

When running an Ansible task using the s3_sync module (https://docs.ansible.com/ansible/2.9/modules/s3_sync_module.html) on a Jenkins worker where FIPS is enabled, we got the following error.

ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

Full error description below:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips
11:32:29 fatal: [127.0.0.1]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File "/opt/cms/data/jenkins/.ansible/tmp/ansible-tmp-1632324747.86-39306657754486/AnsiballZ_s3_sync.py", line 102, in \n _ansiballz_main()\n File "/opt/cms/data/jenkins/.ansible/tmp/ansible-tmp-1632324747.86-39306657754486/AnsiballZ_s3_sync.py", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File "/opt/cms/data/jenkins/.ansible/tmp/ansible-tmp-1632324747.86-39306657754486/AnsiballZ_s3_sync.py", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.cloud.amazon.s3_sync', init_globals=None, run_name='main', alter_sys=True)\n File "/usr/lib64/python2.7/runpy.py", line 176, in run_module\n fname, loader, pkg_name)\n File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code\n mod_name, mod_fname, mod_loader, pkg_name)\n File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code\n exec code in run_globals\n File "/tmp/ansible_s3_sync_payload_cpIDqe/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 544, in \n File "/tmp/ansible_s3_sync_payload_cpIDqe/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 525, in main\n File "/tmp/ansible_s3_sync_payload_cpIDqe/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 347, in calculate_local_etag\n File "/tmp/ansible_s3_sync_payload_cpIDqe/ansible_s3_sync_payload.zip/ansible/modules/cloud/amazon/s3_sync.py", line 277, in calculate_multipart_etag\nValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Our code looks like this:

• name: sync dist folder to s3 bucket
  s3_sync:
  bucket: "{{s3_bucket}}"
  file_root: "{{ ambda_download_dir }}/package/dist"
  mime_map: {
  ".svg": "image/svg+xml",
  ".json": "application/json",
  }
  region: "{{ lambda_region }}"
  file_change_strategy: force
  register: sync_output

The same works fine on Jenkins workers with FIPS disabled.

Issue Type

Feature Idea

Component Name

s3_sync

Additional Information

Our code looks like this:

• name: sync dist folder to s3 bucket
  s3_sync:
  bucket: "{{s3_bucket}}"
  file_root: "{{ ambda_download_dir }}/package/dist"
  mime_map: {
  ".svg": "image/svg+xml",
  ".json": "application/json",
  }
  region: "{{ lambda_region }}"
  file_change_strategy: force
  register: sync_output

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• [plugins/modules/s3_sync.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/s3_sync.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @jillr @markuman @s-hertel @tremble @wimnat
click here for bot help

[markuman]

@nkalatho thx for contributing this FIPS issue.

First, it would be awesome to know your ansible version and collection versions.

ansible --version
ansible-galaxy collection list

Because your python 2.7 runtime is very old, and the referencing error line 277 does not match with current code base.

Furthermore, this is (imo) a false positive. FIPS does not allow md5 algorithm in context of cryptographic, because it is weak.
But ansible is not doing any cryptograpic context here, it is using it to compare local files with upstream files in the bucket. And imo, aws boto3 doesn't support any other algorithm.

ETag (string) --

Entity tag that identifies the newly created object's data. Objects with different object data will have different entity tags. The entity tag is an opaque string. The entity tag may or may not be an MD5 digest of the object data. If the entity tag is not an MD5 digest of the object data, it will contain one or more nonhexadecimal characters and/or will consist of less than 32 or more than 32 hexadecimal digits.

I'm sorry, but there is nothing we can do about it.
The only thing what you can do about it, is to set the file_change_strategy parameter to data_size or force. I guess that should prevent that python is running the hashlib.md5() function and FIPS won't interrupt the play.

[jillr]

Hi @nkalatho, are you using GovCloud region? If that is the case, unfortunately the Ansible community does not have access to GovCloud and can not test with s3-fips.us-gov-east-1.amazonaws.com endpoints so our support for this is best-effort and based on standard AWS regions and partitions.