Improve translog corruption detection #47873
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today we do not throw a `TranslogCorruptedException` in certain cases of translog corruption, such as for a corrupted checkpoint file or when an expected file (either checkpoint or translog) is completely missing or truncated. This means that `elasticsearch-shard` will not truncate the translog in those cases. This commit strengthens the translog corruption tests to corrupt and/or delete both translog and checkpoint files, and ensures that a `TranslogCorruptedException` is thrown in all cases. It also sometimes simulates a recovery after a crash while rolling the translog generation, including cases where the rolled checkpoint contains incorrect data. This backports elastic#42980, elastic#42744 and elastic#44217 to 6.8. It also backports elastic#41480 to adjust the tool to check shards regardless of whether there is a corruption marker. Co-authored-by: Henning Andersen <[email protected]>
DaveCTurner
added
>bug
:Distributed Indexing/Store
|
Pinging @elastic/es-distributed (:Distributed/Engine) |
henningandersen
approved these changes
Oct 11, 2019
tomcallahan
removed
the
:Distributed Indexing/Store
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today we do not throw a
TranslogCorruptedExceptionin certain cases oftranslog corruption, such as for a corrupted checkpoint file or when an
expected file (either checkpoint or translog) is completely missing or
truncated. This means that
elasticsearch-shardwill not truncate the translogin those cases.
This commit strengthens the translog corruption tests to corrupt and/or delete
both translog and checkpoint files, and ensures that a
TranslogCorruptedExceptionis thrown in all cases. It also sometimessimulates a recovery after a crash while rolling the translog generation,
including cases where the rolled checkpoint contains incorrect data. This
backports #42980, #42744 and #44217 to 6.8.
It also backports #41480 to adjust the tool to check shards regardless of
whether there is a corruption marker.
Co-authored-by: Henning Andersen [email protected]