Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add AWSRetry backoff logic to route53_zone and route53_info #865

Merged
merged 3 commits into from
Feb 4, 2022
Merged

Add AWSRetry backoff logic to route53_zone and route53_info #865

merged 3 commits into from
Feb 4, 2022

Conversation

marknet15
Copy link
Contributor

SUMMARY

Add AWSRetry backoff logic to route53_zone and route53_info. Currently from time to time I've been hitting AWS throttling errors leading to ansible failures:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded
fatal: [localhost_staging -> 127.0.0.1]: FAILED! => changed=false 
  boto3_version: 1.20.34
  botocore_version: 1.23.34
  error:
    code: Throttling
    message: Rate exceeded
    type: Sender
  msg: 'Could not list current hosted zones: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded'
  response_metadata:
    http_headers:
      connection: close
      content-length: '255'
      content-type: text/xml
      date: Fri, 14 Jan 2022 12:09:35 GMT
      x-amzn-requestid: xxxxxxx
    http_status_code: 400
    max_attempts_reached: true
    request_id: xxxxxxx
    retry_attempts: 4
ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

route53_zone
route53_info

ADDITIONAL INFORMATION

I've added the standard backoff retry logic and split out the paginators.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@marknet15 marknet15 force-pushed the add-backoff-retries branch from 3fcd7b9 to d7a55ef Compare January 28, 2022 17:05
@ansibullbot
Copy link

cc @Etherdaemon @jillr @markuman @minichate @s-hertel @tremble
click here for bot help

@ansibullbot ansibullbot added bug This issue/PR relates to a bug community_review module module needs_triage plugins plugin (any type) traceback labels Jan 28, 2022
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@markuman markuman added backport-2 PR should be backported to the stable-2 branch backport-3 PR should be backported to the stable-3 branch mergeit Merge the PR (SoftwareFactory) labels Feb 4, 2022
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded (gate pipeline).

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit ca1d33f into ansible-collections:main Feb 4, 2022
@patchback
Copy link

patchback bot commented Feb 4, 2022 •
edited
Loading

Backport to stable-2: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply ca1d33f on top of patchback/backports/stable-2/ca1d33ff5549eae5eb40aa7dfab4eb059075f70e/pr-865

Backporting merged PR #865 into main

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it: 
    $ git remote add upstream https://github.com/ansible-collections/community.aws.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code: 
    $ git fetch upstream
$ git checkout -b patchback/backports/stable-2/ca1d33ff5549eae5eb40aa7dfab4eb059075f70e/pr-865 upstream/stable-2
  4. Now, cherry-pick PR Add AWSRetry backoff logic to route53_zone and route53_info #865 contents into that branch: 
    $ git cherry-pick -x ca1d33ff5549eae5eb40aa7dfab4eb059075f70e
    If it'll yell at you with something like fatal: Commit ca1d33ff5549eae5eb40aa7dfab4eb059075f70e is a merge but no -m option was given., add -m 1 as follows intead: 
    $ git cherry-pick -m1 -x ca1d33ff5549eae5eb40aa7dfab4eb059075f70e
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Add AWSRetry backoff logic to route53_zone and route53_info #865 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub: 
    $ git push origin patchback/backports/stable-2/ca1d33ff5549eae5eb40aa7dfab4eb059075f70e/pr-865
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Feb 4, 2022 •
edited
Loading

Backport to stable-3: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-3/ca1d33ff5549eae5eb40aa7dfab4eb059075f70e/pr-865

Backported as #924

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback bot pushed a commit that referenced this pull request Feb 4, 2022
@marknet15 @patchback
Add AWSRetry backoff logic to route53_zone and route53_info (#865)
4ac1c51 
Add AWSRetry backoff logic to route53_zone and route53_info

SUMMARY
Add AWSRetry backoff logic to route53_zone and route53_info. Currently from time to time I've been hitting AWS throttling errors leading to ansible failures:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded
fatal: [localhost_staging -> 127.0.0.1]: FAILED! => changed=false
  boto3_version: 1.20.34
  botocore_version: 1.23.34
  error:
    code: Throttling
    message: Rate exceeded
    type: Sender
  msg: 'Could not list current hosted zones: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded'
  response_metadata:
    http_headers:
      connection: close
      content-length: '255'
      content-type: text/xml
      date: Fri, 14 Jan 2022 12:09:35 GMT
      x-amzn-requestid: xxxxxxx
    http_status_code: 400
    max_attempts_reached: true
    request_id: xxxxxxx
    retry_attempts: 4
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
route53_zone
route53_info
ADDITIONAL INFORMATION
I've added the standard backoff retry logic and split out the paginators.

Reviewed-by: Alina Buzachis <None>
Reviewed-by: Markus Bergholz <[email protected]>
(cherry picked from commit ca1d33f)
@patchback patchback bot mentioned this pull request Feb 4, 2022
Merged
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Feb 4, 2022
@patchback
Add AWSRetry backoff logic to route53_zone and route53_info (#865) (#924
e0e4236 
)

[PR #865/ca1d33ff backport][stable-3] Add AWSRetry backoff logic to route53_zone and route53_info

This is a backport of PR #865 as merged into main (ca1d33f).
SUMMARY
Add AWSRetry backoff logic to route53_zone and route53_info. Currently from time to time I've been hitting AWS throttling errors leading to ansible failures:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded
fatal: [localhost_staging -> 127.0.0.1]: FAILED! => changed=false 
  boto3_version: 1.20.34
  botocore_version: 1.23.34
  error:
    code: Throttling
    message: Rate exceeded
    type: Sender
  msg: 'Could not list current hosted zones: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded'
  response_metadata:
    http_headers:
      connection: close
      content-length: '255'
      content-type: text/xml
      date: Fri, 14 Jan 2022 12:09:35 GMT
      x-amzn-requestid: xxxxxxx
    http_status_code: 400
    max_attempts_reached: true
    request_id: xxxxxxx
    retry_attempts: 4
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
route53_zone
route53_info
ADDITIONAL INFORMATION
I've added the standard backoff retry logic and split out the paginators.
@jatorcasso jatorcasso mentioned this pull request Jun 10, 2022
Merged
abikouo pushed a commit to abikouo/community.aws that referenced this pull request Oct 24, 2023
@jatorcasso
Update dev guidelines - add return values guidelines (ansible-collect…
7f4b4a3 
…ions#865)

Update dev guidelines - add return values guidelines

SUMMARY
As discussed in the last working group meeting. Updating the guidelines to create standards for what _info modules should return.
Items addressed:

more info on how to return tags using boto3_tag_list_to_ansible_dict
info modules should return list of dicts
how to deprecate keys

ISSUE TYPE

Docs Pull Request

COMPONENT NAME
docs/docsite/rst/dev_guidelines.rst

Reviewed-by: Mark Chappell <None>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: Joseph Torcasso <None>
Reviewed-by: Mandar Kulkarni <[email protected]>
Reviewed-by: Markus Bergholz <[email protected]>
Reviewed-by: Jill R <None>
Reviewed-by: Tabah Baridule <[email protected]>
abikouo pushed a commit to abikouo/community.aws that referenced this pull request Oct 24, 2023
@marknet15 @goneri
Add AWSRetry backoff logic to route53_zone and route53_info (ansible-…
de1422d 
…collections#865)

Add AWSRetry backoff logic to route53_zone and route53_info

SUMMARY
Add AWSRetry backoff logic to route53_zone and route53_info. Currently from time to time I've been hitting AWS throttling errors leading to ansible failures:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded
fatal: [localhost_staging -> 127.0.0.1]: FAILED! => changed=false
  boto3_version: 1.20.34
  botocore_version: 1.23.34
  error:
    code: Throttling
    message: Rate exceeded
    type: Sender
  msg: 'Could not list current hosted zones: An error occurred (Throttling) when calling the ListHostedZones operation (reached max retries: 4): Rate exceeded'
  response_metadata:
    http_headers:
      connection: close
      content-length: '255'
      content-type: text/xml
      date: Fri, 14 Jan 2022 12:09:35 GMT
      x-amzn-requestid: xxxxxxx
    http_status_code: 400
    max_attempts_reached: true
    request_id: xxxxxxx
    retry_attempts: 4
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
route53_zone
route53_info
ADDITIONAL INFORMATION
I've added the standard backoff retry logic and split out the paginators.

Reviewed-by: Alina Buzachis <None>
Reviewed-by: Markus Bergholz <[email protected]>

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections@ca1d33f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alinabuzachis alinabuzachis alinabuzachis approved these changes

@markuman markuman markuman approved these changes

Assignees
No one assigned
Labels
backport-2 PR should be backported to the stable-2 branch backport-3 PR should be backported to the stable-3 branch bug This issue/PR relates to a bug community_review mergeit Merge the PR (SoftwareFactory) module module plugins plugin (any type) traceback
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@marknet15 @ansibullbot @markuman @alinabuzachis