-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kvserver: add garbage collection for range tombstones #76783
Conversation
Realized |
@jbowens One of the changes here is to add an |
4e74115
to
67d71b2
Compare
Ok, I think this should be ready for review now. I added a parameter This optimization could possibly also simply call |
67d71b2
to
33bdf5d
Compare
This patch adds an iteration key type `IterKeyTypePointsWithRanges` for `MVCCIterator` which iterates only over point keys, but surfaces range keys overlapping those points. Release note: None
This patch automatically clears all range keys (at any timestamp) in the `Engine` methods `ClearMVCCRangeAndIntents` and `ClearIterRange`. Range keys are not affected by `ClearRawRange` (which clears raw engine keys) and `ClearMVCCRange` (which is intended for clearing a subset of versions for a single key). This is implemented via a new `Engine.ExperimentalClearMVCCRangeKeys` method which calls through to Pebble's `RangeKeyDelete`, efficiently deleting all range keys in a key span. By extension, the `ClearRange` RPC method now also clears range keys, both when using point deletions and range deletions. Release note: None
This patch adds basic support for MVCC range tombstones in garbage collection. It garbage collects points below a range tombstone, as well as the range tombstones themselves. However, `MVCCStats` do not currently take range tombstones into account for garbage statistics -- this will be addressed separately. Garbage collection below range tombstones does not do anything fancy like dropping a Pebble range tombstone when there are no newer versions above the range tombstone -- it still uses point clears for every GCed key. This can be optimized later. Release note: None
This adds a parameter `ExperimentalRanges` to `GCRequest`, and a corresponding `ExperimentalGCRanges` version gate, which allows GCing large swathes of keys using Pebble range tombstones. This parameter is not yet in use, but is added preemptively to allow garbage collection to make use of it in the future when GCing below an MVCC range tombstone with no keys above it. Since MVCC range tombstones are experimental, this can possibly be added in a 22.1 patch release. Release note: None
33bdf5d
to
4cdd8cd
Compare
storage: add
IterKeyTypePointsWithRanges
forMVCCIterator
This patch adds an iteration key type
IterKeyTypePointsWithRanges
forMVCCIterator
which iterates only over point keys, but surfaces rangekeys overlapping those points.
Release note: None
storage: clear range keys in some
Engine.Clear*
methodsThis patch automatically clears all range keys (at any timestamp) in the
Engine
methodsClearMVCCRangeAndIntents
andClearIterRange
. Rangekeys are not affected by
ClearRawRange
(which clears raw engine keys)and
ClearMVCCRange
(which is intended for clearing a subset ofversions for a single key).
This is implemented via a new
Engine.ExperimentalClearMVCCRangeKeys
method which calls through to Pebble's
RangeKeyDelete
, efficientlydeleting all range keys in a key span.
By extension, the
ClearRange
RPC method now also clears range keys,both when using point deletions and range deletions.
Release note: None
kvserver: add garbage collection for range tombstones
This patch adds basic support for MVCC range tombstones in garbage
collection. It garbage collects points below a range tombstone, as well
as the range tombstones themselves. However,
MVCCStats
do notcurrently take range tombstones into account for garbage statistics --
this will be addressed separately.
Garbage collection below range tombstones does not do anything fancy
like dropping a Pebble range tombstone when there are no newer versions
above the range tombstone -- it still uses point clears for every GCed
key. This can be optimized later.
Resolves #70414.
Release note: None
batcheval: add
ExperimentalRanges
parameter forGCRequest
This adds a parameter
ExperimentalRanges
toGCRequest
, and acorresponding
ExperimentalGCRanges
version gate, which allows GCinglarge swathes of keys using Pebble range tombstones.
This parameter is not yet in use, but is added preemptively to allow
garbage collection to make use of it in the future when GCing below an
MVCC range tombstone with no keys above it. Since MVCC range tombstones
are experimental, this can possibly be added in a 22.1 patch release.
Release note: None