Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't update retry state when failed to get ack response. fix #1665 #1686

Merged
merged 3 commits into from
Nov 29, 2017

Conversation

repeatedly
Copy link
Member

Ack failure is not buffer flush error so it should not update retry state.
It create invalid retry state and this causes busy loop.
The problematic node is disabled and its node is excluded at next flush after rollback.
So non-acked chunk should be retried next flush.

To resolve this problem, we add update_retry parameter to Output#rollback_write.
This parameter controls rollback considers retry or not.

Ack failure is not buffer flush error so it should not update retry state.
The problematic node is disabled and its node is excluded at next flush after rollback.

To resolve this problem, we add update_retry parameter to Output#rollback_write.
This parameter controls rollback behaviour considers retry or not.
@mururu
Copy link
Member

mururu commented Sep 11, 2017

I can't fully get what this commit means yet, but I feel that retry strategies should be applied to the set of "commit_write + rollback_write" in the case of async buffered output.

@repeatedly
Copy link
Member Author

I feel that retry strategies should be applied to the set of "commit_write + rollback_write" in the case of async buffered output.

Yeah, generally yes. But out_forward removes problematic node from the active node list, so immediate retry is no problem. If we retry flush with same destination, retry state should be updated.

@GLStephen
Copy link

@repeatedly I'm currently testing this behavior and I will patch my 14.22.rc2 with this and report back

@repeatedly
Copy link
Member Author

Use keyword argument. This is more clear than before.

@repeatedly
Copy link
Member Author

repeatedly commented Nov 28, 2017

@mururu How about this? I will consider changing retry status management but it needs more time to re-desing internal mechanzim. So this temporary fix seems acceptable for out_forward because failed chunk is retried at next flush. If the destination still has a problem, it updates retry status.

@mururu
Copy link
Member

mururu commented Nov 28, 2017

@repeatedly Fair enough. Then it is good to add comments for that.

@repeatedly repeatedly merged commit 1b9cddd into master Nov 29, 2017
@repeatedly repeatedly deleted the fix-busy-flush-in-out_forward branch March 9, 2018 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v0.14
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants