s3_lifecycle encounters error KeyError: 'Rules' when there are no lifecycle configuration rules on bucket

[jeremyssc]

Summary

Hi :) ,

When I try to use the community.aws.s3_lifecycle module from either community.aws collection 7.2.0 or 7.2.1 to create a lifecycle configuration rule on a s3 bucket created by openshift data foundation using noobaa on OpenShift 4.15.5 with OpenShift Data Foundation operator version v4.14.5-rhodf. The error happens when the bucket has no lifecycle configuration rule and the line where the error happens in the module is here . It seems like the module isn't handling the case where there is no lifecycle configuration rule.

If I use the awscli to create a lifecycle configuration rule then run my ansible playbook then there is no error and the module functions correctly and creates the rule specified. If I then remove all rules and run the playbook again then I can reproduce the issue

I didn't find something similar in an other issue but maybe I missed one.

Let me know if you need more information and I will do my best
Thanks for your great work,
Jeremy

Issue Type

Bug Report

Component Name

s3_lifecycle

Ansible Version

$ ansible --version
ansible [core 2.15.5]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/myusername/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/myusername/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/myusername/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/myusername/.local/bin/ansible
  python version = 3.9.18 (main, Jan  4 2024, 00:00:00) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] (/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /home/myusername/.ansible/collections/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    7.5.0
community.aws                 7.2.0
community.general             8.5.0
kubernetes.core               3.0.1

# /home/myusername/.local/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    6.5.0
ansible.netcommon             5.2.0
ansible.posix                 1.5.4
ansible.utils                 2.11.0
ansible.windows               1.14.0
arista.eos                    6.1.2
awx.awx                       22.7.0
azure.azcollection            1.18.1
check_point.mgmt              5.1.1
chocolatey.chocolatey         1.5.1
cisco.aci                     2.7.0
cisco.asa                     4.0.2
cisco.dnac                    6.7.5
cisco.intersight              1.0.27
cisco.ios                     4.6.1
cisco.iosxr                   5.0.3
cisco.ise                     2.5.16
cisco.meraki                  2.16.5
cisco.mso                     2.5.0
cisco.nso                     1.0.3
cisco.nxos                    4.4.0
cisco.ucs                     1.10.0
cloud.common                  2.1.4
cloudscale_ch.cloud           2.3.1
community.aws                 6.3.0
community.azure               2.0.0
community.ciscosmb            1.0.6
community.crypto              2.15.1
community.digitalocean        1.24.0
community.dns                 2.6.2
community.docker              3.4.9
community.fortios             1.0.0
community.general             7.5.0
community.google              1.0.0
community.grafana             1.5.4
community.hashi_vault         5.0.0
community.hrobot              1.8.1
community.libvirt             1.3.0
community.mongodb             1.6.3
community.mysql               3.7.2
community.network             5.0.0
community.okd                 2.3.0
community.postgresql          2.4.3
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.10.0
community.sap                 1.0.0
community.sap_libs            1.4.1
community.skydive             1.0.0
community.sops                1.6.6
community.vmware              3.10.0
community.windows             1.13.0
community.zabbix              2.1.0
containers.podman             1.10.3
cyberark.conjur               1.2.2
cyberark.pas                  1.0.23
dellemc.enterprise_sonic      2.2.0
dellemc.openmanage            7.6.1
dellemc.powerflex             1.9.0
dellemc.unity                 1.7.1
f5networks.f5_modules         1.26.0
fortinet.fortimanager         2.2.1
fortinet.fortios              2.3.2
frr.frr                       2.0.2
gluster.gluster               1.0.2
google.cloud                  1.2.0
grafana.grafana               2.2.3
hetzner.hcloud                1.16.0
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.12.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.5.0
inspur.ispim                  1.3.0
inspur.sm                     2.3.0
junipernetworks.junos         5.3.0
kubernetes.core               2.4.0
lowlydba.sqlserver            2.2.1
microsoft.ad                  1.3.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0
netapp.ontap                  22.7.0
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.14.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.1.0
ngine_io.vultr                1.1.3
openstack.cloud               2.1.0
openvswitch.openvswitch       2.1.1
ovirt.ovirt                   3.2.0
purestorage.flasharray        1.21.0
purestorage.flashblade        1.14.0
purestorage.fusion            1.6.0
sensu.sensu_go                1.14.0
servicenow.servicenow         1.0.6
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.33.1
telekom_mms.icinga_director   1.34.1
theforeman.foreman            3.14.0
vmware.vmware_rest            2.3.1
vultr.cloud                   1.10.0
vyos.vyos                     4.1.0
wti.remote                    1.0.5

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/myusername/.local/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.22.10
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.25.10
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /etc/ansible/ansible.cfg

OS / Environment

RHEL 9.3

Steps to Reproduce

- name: Create the bucket for Loki
  kubernetes.core.k8s:
    state: present
    definition:
      apiVersion: objectbucket.io/v1alpha1
      kind: ObjectBucketClaim
      metadata:
        finalizers:
        - objectbucket.io/finalizer
        labels:
          app: noobaa
          bucket-provisioner: openshift-storage.noobaa.io-obc
          noobaa-domain: openshift-storage.noobaa.io
        name: loki-bucket-odf
        namespace: openshift-logging
      spec:
        additionalConfig:
          bucketclass: noobaa-default-bucket-class
        generateBucketName: loki-bucket-odf
        objectBucketName: obc-openshift-logging-loki-bucket-odf
        storageClassName: openshift-storage.noobaa.io

- name: Get configmap resource for loki-bucket-odf to extract the information needed
  register: my_bucketName_register
  kubernetes.core.k8s_info:
    api_version: v1
    kind: configmap
    name: loki-bucket-odf
    namespace: openshift-logging
    wait: true

- name: Set fact containing bucketName
  ansible.builtin.set_fact:
    my_bucketName: "{{ my_bucketName_register['resources'][0]['data']['BUCKET_NAME'] }}"

- name: Set fact containing bucketPort
  ansible.builtin.set_fact:
    my_bucketPort: "{{ my_bucketName_register['resources'][0]['data']['BUCKET_PORT'] }}"

- name: Get secret for the bucket
  register: my_bucketSecret_register
  no_log: true
  kubernetes.core.k8s_info:
    api_version: v1
    kind: Secret
    name: loki-bucket-odf
    namespace: openshift-logging
    wait: true

- name: Set facts containing secret keys for the loki-bucket-odf bucket
  no_log: true
  ansible.builtin.set_fact:
    my_AWS_ACCESS_KEY_ID: "{{ my_bucketSecret_register['resources'][0]['data']['AWS_ACCESS_KEY_ID']  }}"
    my_AWS_SECRET_ACCESS_KEY: "{{ my_bucketSecret_register['resources'][0]['data']['AWS_SECRET_ACCESS_KEY'] }}"

- name: Get route for the host
  register: my_route_register
  kubernetes.core.k8s_info:
    api_version: route.openshift.io/v1
    kind: Route
    name: s3
    namespace: openshift-storage

- name: Set fact for route
  ansible.builtin.set_fact:
    my_route_host: "{{ my_route_register['resources'][0]['spec']['host'] }}"

- name: Set fact containing endpoint using the external route for s3 bucket
  ansible.builtin.set_fact:
    my_route_endpoint: "{{ 'https://' ~ my_route_host ~ ':' ~ my_bucketPort }}"

- name: Apply LifeCycle policy to Loki bucket to apply retention policy
  community.aws.s3_lifecycle:
    name: "{{ my_bucketName }}"
    access_key: "{{ my_AWS_ACCESS_KEY_ID | b64decode }}"
    secret_key: "{{ my_AWS_SECRET_ACCESS_KEY | b64decode }}"
    endpoint_url: "{{ my_route_endpoint }}"
    validate_certs: false
    state: present
    wait: true
    expiration_days:  10
    rule_id: "data_expire_withoutprefix"

Expected Results

I expected the s3_lifecycle module to create the lifecycle configuration rule data_expire_withoutprefix

Actual Results

The s3_lifecyle encounters a module error

TASK [openshift_cluster_logging : Apply LifeCycle policy to Loki bucket to apply retention policy] *********************************************************************
task path: /home/myusername/.ansible/roles/openshift_cluster_logging/tasks/configure.yaml:202
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: myusername
<localhost> EXEC /bin/sh -c 'echo ~myusername&& sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/myusername/.ansible/tmp `"&& mkdir "` echo /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326 `" && echo ansible-tmp-1712773370.9758935-30061-199350360303326="` echo /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326 `" ) && sleep 0'
Loading collection amazon.aws from /home/myusername/.ansible/collections/ansible_collections/amazon/aws
Using module file /home/myusername/.ansible/collections/ansible_collections/community/aws/plugins/modules/s3_lifecycle.py
<localhost> PUT /home/myusername/.ansible/tmp/ansible-local-29286b_nmrcj1/tmp1mn6d951 TO /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/AnsiballZ_s3_lifecycle.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/ /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/AnsiballZ_s3_lifecycle.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3 /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/AnsiballZ_s3_lifecycle.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.nii-eccc-ocp-dev1.nii.ec.gc.ca'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n  warnings.warn(\nTraceback (most recent call last):\n  File \"/home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/AnsiballZ_s3_lifecycle.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/AnsiballZ_s3_lifecycle.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/myusername/.ansible/tmp/ansible-tmp-1712773370.9758935-30061-199350360303326/AnsiballZ_s3_lifecycle.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.s3_lifecycle', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.s3_lifecycle', _modlib_path=modlib_path),\n  File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.s3_lifecycle_payload_5sugr_tw/ansible_community.aws.s3_lifecycle_payload.zip/ansible_collections/community/aws/plugins/modules/s3_lifecycle.py\", line 684, in <module>\n  File \"/tmp/ansible_community.aws.s3_lifecycle_payload_5sugr_tw/ansible_community.aws.s3_lifecycle_payload.zip/ansible_collections/community/aws/plugins/modules/s3_lifecycle.py\", line 678, in main\n  File \"/tmp/ansible_community.aws.s3_lifecycle_payload_5sugr_tw/ansible_community.aws.s3_lifecycle_payload.zip/ansible_collections/community/aws/plugins/modules/s3_lifecycle.py\", line 490, in create_lifecycle_rule\n  File \"/tmp/ansible_community.aws.s3_lifecycle_payload_5sugr_tw/ansible_community.aws.s3_lifecycle_payload.zip/ansible_collections/community/aws/plugins/modules/s3_lifecycle.py\", line 264, in fetch_rules\nKeyError: 'Rules'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[jeremyssc]

I updated the pip package of boto and boto3 since I saw a warning relating to the fact that my versions weren't supported when applying after defining a manual rule but I get the same error message :(

$  pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/myusername/.local/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.34.81
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/myusername/.local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.34.81
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/myusername/.local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

[jeremyssc]

From what I can understand the problem seems to be here and the cause is that we do not check if the key Rules exist and since there is no lifecycle rules then we get nothing back (no json)

Here is the output I get when fetching the lifecycle configuration rules when there is none:

 aws --endpoint https://s3-openshift-storage.apps.ocp-cluster.example.com --no-verify-ssl s3api  get-bucket-lifecycle-configuration --bucket loki-bucket-odf-1234567890
urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp-cluster.example.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

I think adding an if else to check if the key Rules exists would resolve this edge case:

def fetch_rules(client, module, name):
    # Get the bucket's current lifecycle rules
    try:
        current_lifecycle = client.get_bucket_lifecycle_configuration(aws_retry=True, Bucket=name)
	#Check if the key Rules is in directory and return empty list if not in dictionary else proceed normally 
	if "Rules" not in current_lifecycle:
	  current_lifecycle_rules = []
	else:
          current_lifecycle_rules = normalize_boto3_result(current_lifecycle["Rules"])

[fernandodlhdt]

Same issue here (mostly the same use case. I confirmed that if I set a dummy lifecycle rule on the bucket, I can change it after. So the problem is exactly as described, when the bucket has no lifecycle rule set.