The cloudwatchlogs_log_group_info module generates throttling exceptions

[b0tting]

Summary

When running the amazon.aws.cloudwatchlogs_log_group_info module on AWS accounts having more than (roughly) 50 log groups we get ThrottlingExceptions once every 20 calls or so. I noticed that both the describe and the list-tags boto calls in the cloudwatchlogs_log_group_info module have no throttling handling configured and use the default throttling handling policy, ie. none.

Issue Type

Bug Report

Component Name

amazon.aws.cloudwatchlogs_log_group_info

Ansible Version

ansible [core 2.13.13]
  config file = None
  configured module search path = ['/home/rundeck/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /home/rundeck/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0]
  jinja version = 3.1.3
  libyaml = True

Collection Versions

Collection            Version
--------------------- -------
amazon.aws            7.3.0
community.general     8.3.0
nutanix.ncp           1.7.0
servicenow.servicenow 1.0.6

# /usr/local/lib/python3.8/dist-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    3.5.0
ansible.netcommon             3.1.3
ansible.posix                 1.4.0
ansible.utils                 2.8.0
ansible.windows               1.12.0
arista.eos                    5.0.1
awx.awx                       21.10.0
azure.azcollection            1.14.0
check_point.mgmt              2.3.0
chocolatey.chocolatey         1.3.1
cisco.aci                     2.3.0
cisco.asa                     3.1.0
cisco.dnac                    6.6.1
cisco.intersight              1.0.22
cisco.ios                     3.3.2
cisco.iosxr                   3.3.1
cisco.ise                     2.5.9
cisco.meraki                  2.13.0
cisco.mso                     2.1.0
cisco.nso                     1.0.3
cisco.nxos                    3.2.0
cisco.ucs                     1.8.0
cloud.common                  2.1.2
cloudscale_ch.cloud           2.2.3
community.aws                 3.6.0
[...]

AWS SDK versions

WARNING: Package(s) not found: boto
Name: boto3
Version: 1.34.45
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.34.45
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3, awscli

Configuration

(no Ansible configuration)

OS / Environment

NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"

Steps to Reproduce

- name: "Role based get all log groups in {{ selected_region }}"
  amazon.aws.cloudwatchlogs_log_group_info:
    region: "{{ selected_region }}"
    aws_access_key: "{{ aws_access_key }}"
    aws_secret_key: "{{ aws_secret_key }}"
    aws_session_token: "{{ aws_session_token }}"
    log_group_name: "{{ log_group_prefix }}"
  register: log_groups

Expected Results

Should get all log group information and tags without error. Optionally, allow us to add an option to skip getting the tags for each log group would allow engineers to work around this issue.

Actual Results

fatal: [127.0.0.1]: FAILED! => {"boto3_version": "1.34.45", "botocore_version": "1.34.45", "changed": false, "error": {"code": "ThrottlingException", "message": "Rate exceeded"}, "msg": "Unable to describe tags for log group /aws/codebuild/tankmaintenanceplanning-pipeline-tsa: An error occurred (ThrottlingException) when calling the ListTagsLogGroup operation (reached max retries: 4): Rate exceeded"

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[abikouo]

@b0tting the way we prevent throttling issues when making API calls is by implementing exponential backoff.
Could you please give it a try with #2019 and see if this is solving your issue?
Thanks

[b0tting]

Ansible tasks run successfully. I don't have definite proof that this solves the issue, I don't see tasks taking more time than expected (ie. timed backoffs), but I expect this is the solution. Thank you!