Skip to content

Commit

Permalink
storage: add MVCC point synthesizing iterator
Browse files Browse the repository at this point in the history
This patch adds `pointSynthesizingIter`, an MVCC iterator which wraps an
arbitrary MVCC iterator and synthesizes point keys for range keys at
their start key and where they overlap point keys. It can optionally
synthesize around the SeekGE seek key too, which is useful for point
operations like `MVCCGet` where we may want to return a synthetic
tombstone for an MVCC range tombstone if there is no existing point key.

This will primarily be used to handle MVCC range tombstones in MVCC
scans and gets, as well as during MVCC conflict checks, which allows
much of this logic to remain unchanged and simplified (in particular,
`pebbleMVCCScanner` will not need any changes).

However, this patch does not make use of the iterator yet, since both it
and Pebble will need further performance optimizations for use in hot
paths. For now, correctness is sufficient, and only basic attempts at
performance optimization have been made.

Release note: None
  • Loading branch information
erikgrinaker committed Jun 8, 2022
1 parent a1dd3f3 commit d07bf3a
Show file tree
Hide file tree
Showing 5 changed files with 1,749 additions and 1 deletion.
1 change: 1 addition & 0 deletions pkg/storage/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ go_library(
"pebble_iterator.go",
"pebble_merge.go",
"pebble_mvcc_scanner.go",
"point_synthesizing_iter.go",
"read_as_of_iterator.go",
"replicas_storage.go",
"resource_limiter.go",
Expand Down
5 changes: 4 additions & 1 deletion pkg/storage/mvcc_history_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ import (
// get [t=<name>] [ts=<int>[,<int>]] [resolve [status=<txnstatus>]] k=<key> [inconsistent] [tombstones] [failOnMoreRecent] [localUncertaintyLimit=<int>[,<int>]] [globalUncertaintyLimit=<int>[,<int>]]
// scan [t=<name>] [ts=<int>[,<int>]] [resolve [status=<txnstatus>]] k=<key> [end=<key>] [inconsistent] [tombstones] [reverse] [failOnMoreRecent] [localUncertaintyLimit=<int>[,<int>]] [globalUncertaintyLimit=<int>[,<int>]] [max=<max>] [targetbytes=<target>] [avoidExcess] [allowEmpty]
//
// iter_new [k=<key>] [end=<key>] [prefix] [kind=key|keyAndIntents] [types=pointsOnly|pointsWithRanges|pointsAndRanges|rangesOnly] [maskBelow=<int>[,<int>]]
// iter_new [k=<key>] [end=<key>] [prefix] [kind=key|keyAndIntents] [types=pointsOnly|pointsWithRanges|pointsAndRanges|rangesOnly] [pointSynthesis [emitOnSeekGE]] [maskBelow=<int>[,<int>]]
// iter_seek_ge k=<key> [ts=<int>[,<int>]]
// iter_seek_lt k=<key> [ts=<int>[,<int>]]
// iter_seek_intent_ge k=<key> txn=<name>
Expand Down Expand Up @@ -1027,6 +1027,9 @@ func cmdIterNew(e *evalCtx) error {
MVCCIterator: r.NewMVCCIterator(kind, opts),
closeReader: closeReader,
}
if e.hasArg("pointSynthesis") {
e.iter = newPointSynthesizingIter(e.iter, e.hasArg("emitOnSeekGE"))
}
return nil
}

Expand Down
6 changes: 6 additions & 0 deletions pkg/storage/mvcc_key.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,12 @@ func (k MVCCKey) Next() MVCCKey {
}
}

// Clone returns a copy of the key.
func (k MVCCKey) Clone() MVCCKey {
k.Key = k.Key.Clone()
return k
}

// Compare returns -1 if this key is less than the given key, 0 if they're
// equal, or 1 if this is greater. Comparison is by key,timestamp, where larger
// timestamps sort before smaller ones except empty ones which sort first (like
Expand Down
Loading

0 comments on commit d07bf3a

Please sign in to comment.