[backport] [v23.1.x] tx/compaction: ensure last batch in a segment is not compacted way #13687
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
control batches (tx_commit/tx_abort) are raft_data batches too and may accidentally overwrite previous user raft_data batches with same key. Ensure that doesn't happen (cherry picked from commit 675ed6e)
(cherry picked from commit 46d17bf)
Currently we always discard control batches. This is a problem when the batch being discarded is the last batch in a segment. In some edgecases where there is no user data after the control batch (empty new segment), consumers may be stuck behind LSO and repeatedly trying to poll for data when there is none. This may look like an infinite loop from a consumer POV but its just that there is no data after that offset as it got compacted away. This situation is not possible in a regular self compaction becuase we always retain the last batch. With this commit we use regular compaction process for control batches, so the resulting segment may end up with some of them and guaranteed to retain the last batch. These are just filtered away on the client side but gurantees the consumed offset makes progress. (cherry picked from commit 55d2ae2)
(cherry picked from commit 1fb2e1a)
bharathv
force-pushed
the
v231x-compaction
branch
from
d202fba
to
3132722
Compare
rystsov
approved these changes
Sep 27, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #13609
With this PR, control batches are not discarded but are compacted in the same manner as other data batches. This approach prevents the elimination of the last batch within a segment when it happens to be a control batch. If the final batch in a segment is discarded, and there are no subsequent data batches, it could lead to clients being unable to advance their consumed offset until the LSO, creating the perception of a stuck consumer unable to make progress.
However, this situation is not problematic when the last batch is an aborted data batch. This is because we can guarantee the presence of user consumable batches following it (in the next segment), specifically the abort control batch (or maybe other interleaved committed batches), ensuring continuous consumer progress.
Fixes #13639
Backports Required
Release Notes
Bug Fixes