Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flush expired lines before polling input at file source line aggregation #1870

Open
MOZGIII opened this issue Feb 20, 2020 · 0 comments
Open
Labels
source: file Anything `file` source related type: tech debt A code change that does not add user value.

Comments

@MOZGIII
Copy link
Contributor

MOZGIII commented Feb 20, 2020

Something that came up during #1852 review. See #1852 (comment).

We should flush expired before polling inner, otherwise, we don't provide proper backpressure and the expired lines may never even flush at all if the load is super-high.

There should be many more tests for the timeout and expiration logic, and once those are done - we can safely alter details like this.

@binarylogic binarylogic added source: file Anything `file` source related type: tech debt A code change that does not add user value. labels Feb 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
source: file Anything `file` source related type: tech debt A code change that does not add user value.
Projects
None yet
Development

No branches or pull requests

2 participants