Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flush instead of synced-flush inactive shards #49126

Merged
merged 7 commits into from
Nov 22, 2019

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Nov 14, 2019

With peer recovery retention leases and sequence-number based replica allocation, a regular flush can speed up recovery as a synced-flush. With this change, we will flush instead of synced-flush when a shard becomes inactive.

Depends on #48430
Closes #31965

@dnhatn dnhatn added >enhancement :Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. v8.0.0 v7.6.0 labels Nov 14, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Distributed)

Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will still need the old sync-flush logic in 7.x to handle 6.x/7.x indices without soft-deletes as rolling/full cluster restarts are o.w. going to be painful.

@dnhatn dnhatn removed the v7.6.0 label Nov 19, 2019
@dnhatn
Copy link
Member Author

dnhatn commented Nov 19, 2019

I think we will still need the old sync-flush logic in 7.x to handle 6.x/7.x indices without soft-deletes as rolling/full cluster restarts are o.w. going to be painful.

Yes, we should keep this logic in 7.x.

Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I think the flushing behavior should be backported to 7.x (and the sync-flush behavior kept in 7.x as well for non-soft-deletes indices)

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@dnhatn
Copy link
Member Author

dnhatn commented Nov 22, 2019

Thanks @ywelsch and @henningandersen for reviewing.

@dnhatn dnhatn merged commit 725dda3 into elastic:master Nov 22, 2019
@dnhatn dnhatn deleted the flush-on-inactive branch November 22, 2019 03:25
dnhatn added a commit that referenced this pull request Jan 23, 2020
If all nodes are on 7.6, we prefer to perform a normal flush 
instead of synced flush when a shard becomes inactive.

Backport of #49126
@dnhatn dnhatn added the v7.7.0 label Jan 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. >enhancement v7.7.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flush inactive shards
5 participants