Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python-asyncio: update retry factors for actual exponential retries #19337

Merged
merged 2 commits into from
Aug 12, 2024

Conversation

imrehg
Copy link
Contributor

@imrehg imrehg commented Aug 12, 2024

As per the aiohttp-retry library's code1, the timeout is

timeout = self._start_timeout * (self._factor ** attempt)

This means the previous setting with "start_timeout=0.0" would have always just retried right away (0 timeout) regardless of the "factor" value, and also, "factor=0.0" would never have increased the timeout, rather it would have resulted in a 0 timeout regardless of the value of "start_timeout".

This double-zeroing effectively rendered exponential backoff to nothing (or rather "retries" number of retries in quick succession).

The update is a quick fix to set the same default as in aiohttp-retry. In the future this should likely be configurable (through extra Configuration settings perhaps?), as not all APIs are created equal, but this works as a quick fix for making retries more effective.

Connects to #17011

CC: @cbornet @tomplus @krjakbrjak @fa0311 @multani

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • Run the following to build the project and update samples:
    ./mvnw clean package 
    ./bin/generate-samples.sh ./bin/configs/*.yaml
    ./bin/utils/export_docs_generators.sh
    
    (For Windows users, please run the script in Git BASH)
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
  • File the PR against the correct branch: master (upcoming 7.6.0 minor release - breaking changes with fallbacks), 8.0.x (breaking changes without fallbacks)
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Footnotes

  1. https://github.com/inyutin/aiohttp_retry/blob/ba2169891f5b32a5c59e48ca185dd8e68e44ded7/aiohttp_retry/retry_options.py#L38-L65

imrehg added 2 commits August 12, 2024 15:11
As per the `aiohttp-retry` library's code[^1], the timeout is

```python
timeout = self._start_timeout * (self._factor ** attempt)
```

This means the previous setting with "start_timeout=0.0" would have
always just retried right away (0 timeout) regardless of the "factor" value,
and also, "factor=0.0" would never have increased the timeout, rather it
would have resulted in a 0 timeout regardless of the value of "start_timeout".

This double-zeroing effectively rendered exponential backoff to nothing (rather than
"retries" number of retries in quick succession.

The update is a quick fix to set the same default as in `aiohttp-retry`. In
the future this should likely be configurable (through extra Configuration settings perhaps?),
as not all APIs are created equal, but this works as a quick fix for making retries more effective.

[^1]: https://github.com/inyutin/aiohttp_retry/blob/ba2169891f5b32a5c59e48ca185dd8e68e44ded7/aiohttp_retry/retry_options.py#L38-L65
@wing328
Copy link
Member

wing328 commented Aug 12, 2024

lgtm. thanks for the fix

@wing328 wing328 merged commit fdd2dc9 into OpenAPITools:master Aug 12, 2024
23 checks passed
@imrehg imrehg deleted the python-asyncio-parameters branch August 12, 2024 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants