kvserver/rangefeed: always use time-bound iterators for catchup scans #82450
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
erikgrinaker
force-pushed
the
rangefeed-catchup-tbi
branch
from
1b0e814 to
d66ac88
Compare
This was referenced Jun 5, 2022
This patch always uses time-bound iterators for rangefeed catchup scans. Previously, this was controlled by the default-off cluster setting `kv.rangefeed.catchup_scan_iterator_optimization.enabled`, which has now been removed. These have been used in production by several users, who have seen significant performance improvements without reports of any issues. Release note (performance improvement): Changefeed catchup scans now use time-bound iterators, which improves their performance by avoiding accessing data that is outside the catchup scan time interval. Previously, this was controlled by the default-off cluster setting `kv.rangefeed.catchup_scan_iterator_optimization.enabled`, which has now been removed (it is in effect always enabled).
erikgrinaker
force-pushed
the
rangefeed-catchup-tbi
branch
from
d66ac88 to
8d3f4cf
Compare
stevendanna
approved these changes
Jun 6, 2022
|
Thanks! bors r=stevendanna |
|
Build succeeded: |
|
|
I'd leave it in for 22.1, just to have an escape hatch. Just discovered that TBIs may omit inline values, and I don't know yet whether any of the new span config stuff in 22.1 (which relies on rangefeeds) can run across inline values, which might be problematic: #82453 (comment). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
EDIT: Oops, I completely missed that this had already been defaulted to true in #73473. Got confused by #79727, which I took to mean that it hadn't been flipped yet. This PR essentially just removed the setting then. Sorry for the confusion.
This patch always uses time-bound iterators for rangefeed catchup scans.
Previously, this was controlled by the default-off cluster setting
kv.rangefeed.catchup_scan_iterator_optimization.enabled, which has nowbeen removed. These have been used in production by several users, who
have seen significant performance improvements without reports of any
issues.
Resolves #79728.
Touches #82454.
Release note (performance improvement): Changefeed catchup scans now use
time-bound iterators, which improves their performance by avoiding
accessing data that is outside the catchup scan time interval.
Previously, this was controlled by the default-off cluster setting
kv.rangefeed.catchup_scan_iterator_optimization.enabled, which has nowbeen removed (it is in effect always enabled).