S3 Lifecycle module is missing a cast to integer for transitions list

[Razique]

Summary

Hi,
I have noticed that when using the community.aws.s3_lifecycle module, AWS S3 complains that values passed for the transition_days of a life cycle transition are of string type instead of integer:

"msg": "Parameter validation failed:\nInvalid type for parameter LifecycleConfiguration.Rules[0].Transitions[0].Days, value: 5, type: <class 'str'>, valid types: <class 'int'>",

This is what I have for my transition rule:

- name: Manage S3 bucket lifecycle rules in AWS
  community.aws.s3_lifecycle:
    name: "<BUCKET NAME>"
    state: present
    status: enabled
    expiration_days: "{{ s3_bucket.lifecycle.expiration_days | int }}"
    transitions:
      - storage_class: "{{ s3_bucket.lifecycle.storage_class }}"
        transition_days: "{{ s3_bucket.lifecycle.transition_days }}"

With expiration_days having a value of 30 and transition_days having a value of 15.

Looking at the code, I suspect that passing the type instructs Ansible to perform an internal conversion:

def main():
    s3_storage_class = ['glacier', 'onezone_ia', 'standard_ia', 'intelligent_tiering', 'deep_archive']
    argument_spec = dict(
        ...
        expiration_days=dict(type='int'), <--- type here in int

The conversion isn't performed for parameters in transtions, because this is a list of dicts:

transitions=dict(type='list', elements='dict'),

When the rule is being built via the build_rule function, the value is retrieved "as is", which can be a string if the value has been templated by Jinja2.

I suggest to force the cast to integer via the following change. This way, irrespective of the type being "rendered" (either native int or str from Jinja2 templating), we always send the correct type:

transition_days = int(module.params.get("transition_days"))

I could be missing something, but in my test, forcing the conversion solved the issue.

Issue Type

Bug Report

Component Name

s3_lifecycle

Ansible Version

$ ansible --version
ansible [core 2.12.10]
  config file = None
  configured module search path = ['/home/rmahroua/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/rmahroua/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.13 (main, May 18 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)]
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /home/rmahroua/.ansible/collections/ansible_collections
Collection            Version
--------------------- -------
amazon.aws            5.1.0  
awx.awx               21.7.0 
community.aws         5.2.0  
community.general     6.0.1  
community.hashi_vault 4.0.0  
google.cloud          1.0.2  

# /usr/local/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    2.3.0  
ansible.netcommon             2.6.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.10.0 
arista.eos                    3.1.0  
awx.awx                       19.4.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.2.0  
cisco.aci                     2.2.0  
cisco.asa                     2.1.0  
cisco.dnac                    6.5.0  
cisco.intersight              1.0.19 
cisco.ios                     2.8.1  
cisco.iosxr                   2.9.0  
cisco.ise                     1.2.1  
cisco.meraki                  2.8.0  
cisco.mso                     1.4.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.9.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 2.6.1  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.3.4  
community.digitalocean        1.20.0 
community.dns                 2.2.0  
community.docker              2.6.0  
community.fortios             1.0.0  
community.general             4.8.3  
community.google              1.0.0  
community.grafana             1.5.0  
community.hashi_vault         2.5.0  
community.hrobot              1.4.0  
community.kubernetes          2.0.1  
community.kubevirt            1.0.0  
community.libvirt             1.1.0  
community.mongodb             1.4.1  
community.mysql               2.3.8  
community.network             3.3.0  
community.okd                 2.2.0  
community.postgresql          1.7.4  
community.proxysql            1.4.0  
community.rabbitmq            1.2.1  
community.routeros            2.1.0  
community.sap                 1.0.0  
community.sap_libs            1.1.0  
community.skydive             1.0.0  
community.sops                1.2.2  
community.vmware              1.18.2 
community.windows             1.10.0 
community.zabbix              1.7.0  
containers.podman             1.9.3  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            4.4.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.17.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.6  
frr.frr                       1.0.4  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.6.0  
hpe.nimble                    1.1.4  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.2.2  
inspur.sm                     1.3.0  
junipernetworks.junos         2.10.0 
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.18.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.20.0
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.0  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   1.6.6  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.30.0 
theforeman.foreman            2.2.0  
vmware.vmware_rest            2.2.0  
vyos.vyos                     2.8.0  
wti.remote                    1.0.4  

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/rmahroua/.local/lib/python3.9/site-packages
Requires: 
Required-by: aws-automation
---
Name: boto3
Version: 1.24.89
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: aws-automation, aws-shell
---
Name: botocore
Version: 1.27.89
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: aws-automation, awscli, boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

Fedora release 34 (Thirty Four)

Steps to Reproduce

object_expire_days: 30
object_storage_class: "glacier"
object_transition_days: 15

###
s3_bucket:
  encryption: "AES256"
  force: false
  lifecycle:
    storage_class: "{{ object_storage_class | default('intelligent_tiering', true) }}"
    transition_days: "{{ object_transition_days | default(30) }}"
    expiration_days: "{{ object_expire_days | default(90) }}"
  name: "test-bucket"

###
- name: Manage S3 bucket lifecycle rules in AWS
  community.aws.s3_lifecycle:
    name: "{{ s3_bucket.name }}"
    state: present
    status: enabled
    expiration_days: "{{ s3_bucket.lifecycle.expiration_days | int }}"
    transitions:
      - storage_class: "{{ s3_bucket.lifecycle.storage_class }}"
        transition_days: "{{ s3_bucket.lifecycle.transition_days | int }}"  

Expected Results

I expect the module to treat my input value as integer instead of string

Actual Results

"msg": "Parameter validation failed:\nInvalid type for parameter LifecycleConfiguration.Rules[0].Transitions[0].Days, value: 5, type: <class 'str'>, valid types: <class 'int'>",

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[markuman]

@Razique thanks for reporting and digging into the code.
Are you willing to provide a PR that fixes the issue?

• bugfix
• changelog fragment
• integration test, that covers this case

[Razique]

Yup!
I might need to update my workstation, haven't pushed changes in a while. I'll try to get to that soon.