-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove exponential backoff code from all plugins #23
Labels
bug
Something isn't working
Comments
PettitWesley
added a commit
to PettitWesley/amazon-kinesis-firehose-for-fluent-bit
that referenced
this issue
May 12, 2020
This was referenced May 12, 2020
PettitWesley
added a commit
to PettitWesley/amazon-kinesis-firehose-for-fluent-bit
that referenced
this issue
May 15, 2020
PettitWesley
added a commit
that referenced
this issue
Jun 1, 2020
This was referenced Mar 15, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I built the Firehose and CloudWatch plugins, I thought it would be a good idea for them to implement backoff in case of API throttling. That instinct was good, but the execution shows my lack of understanding of Fluent Bit at the time. Fluent Bit has it's own retry mechanisms; a plugin can return FLB_RETRY and let Fluent Bit handle it. More importantly, the backoff mechanism in use currently causes the plugin to sleep- since Fluent Bit is single-threaded, but concurrent, this sleep stops all other plugins, and thus impacts its overall performance.
In practice I suspect few users trigger throttling; however, this is a very easy thing to fix, we just need to remove a bit of code from all the plugins.
The text was updated successfully, but these errors were encountered: