storage: add MVCC range tombstone handling for scans and gets

[erikgrinaker]

This is for the mvcc-range-tombstones branch. Only the last commit should be reviewed here -- the former are from #78085 and its ancestors.

This will need performance optimizations both in MVCC and Pebble before being merged, as it's been seen to have a severe performance penalty (up to 40%). For now, focus on correctness.

---

This patch adds MVCC range tombstone handling for scans and gets. In the
basic case, this simply means that point keys below an MVCC range
tombstone are not visible.

When tombstones are requested by the caller, the MVCC range tombstones
themselves are never exposed, to avoid having to explicitly handle these
throughout the codebase. Instead, synthetic MVCC point tombstones are
emitted at the start of MVCC range tombstones and wherever they overlap
a point key (above and below). Additionally, point gets return synthetic
point tombstones even if no existing point key exists.

The point tombstone synthesis is implemented by a new
pointSynthesizingIterator, an MVCCIterator implementation that wraps
an existing MVCCIterator. This allows e.g. pebbleMVCCScanner to
remain unchanged, and automatically take MVCC range tombstones into
account for e.g. conflict/uncertainty checks.

Point tombstone synthesis must be enabled even when the caller has not
requested tombstones, because they must always be taken into account for
conflict/uncertainty checks. However, in these cases we enable range key
masking below the read timestamp, omitting any covered points since
these are no longer needed.

Touches #70412.

Release note: None

[cockroach-teamcity]

This change is 

[nicktrav]

Reviewed 7 of 7 files at r8, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @erikgrinaker, @jbowens, and @sumeerbhola)

---

pkg/storage/testdata/mvcc_histories/range_tombstone_scans, line 6 at r8 (raw file):

#
#  T
#  6                 [e6]     

nit: looks like there's some extra whitespace here.

[sumeerbhola]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @erikgrinaker, @jbowens, and @nicktrav)

---

pkg/storage/point_synthesizing_iter.go, line 21 at r16 (raw file):

// pointSynthesizingIter wraps an MVCCIterator, and synthesizes point tombstones
// for range tombstones above/below existing point keys. It does not emit range

is "range tombstones" the same as "MVCC range tombstone" accomplished via a RANGEKEYSET?
Just calling it "range tombstone" will confuse folks who are thinking of Pebble range tombstones.
I don't have good naming ideas.

---

pkg/storage/point_synthesizing_iter.go, line 23 at r16 (raw file):

// for range tombstones above/below existing point keys. It does not emit range
// keys at all, since these would appear to conflict with the synthesized point
// keys.

how does this honor scans that set FailOnMoreRecent or want to do Uncertainty checking? They need to see a range key even if there is no overlapping point, yes?

[jbowens]

I like the general approach of synthesizing point tombstones.

When the Tombstones option is disabled, range key masking is enabled
to omit point keys below older range tombstones during iteration.
However, point synthesis must still be enabled in this case to account
for future range tombstones in uncertainty checks.

Does this mean point tombstones are synthesized at the start key of the range tombstone in this case?

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @erikgrinaker, and @nicktrav)

[erikgrinaker]

Does this mean point tombstones are synthesized at the start key of the range tombstone in this case?

Not currently -- they're synthesized around point keys that are above the masking range tombstone. Consider e.g. the case where we're masking @2, and have b@1, [a-c)@2, b@3, [a-c)@4. In this case, b@1 is masked, but we'll synthesize point tombstones for b@2 and b@4.

However, we should likely synthesize points at the start of each range key fragment too -- this handles both FailOnMoreRecent for bare range tombstones and tombstone synthesis for point reads where there is no existing point (replaces emitOnSeek). Not sure yet whether this on-start behavior should always be enabled, or if we should make it optional -- will probably start off with always-on.

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @nicktrav, and @sumeerbhola)

---

pkg/storage/point_synthesizing_iter.go, line 21 at r16 (raw file):

Previously, sumeerbhola wrote…

is "range tombstones" the same as "MVCC range tombstone" accomplished via a RANGEKEYSET?
Just calling it "range tombstone" will confuse folks who are thinking of Pebble range tombstones.
I don't have good naming ideas.

Yes, MVCC range tombstones. I don't think it's too confusing -- we already have the same issue with point tombstones and I don't find it too hard to distinguish between Pebble and MVCC point tombstones from context. But I'll be more consistent in specifying MVCC/Pebble tombstones.

---

pkg/storage/point_synthesizing_iter.go, line 23 at r16 (raw file):

Previously, sumeerbhola wrote…

how does this honor scans that set FailOnMoreRecent or want to do Uncertainty checking? They need to see a range key even if there is no overlapping point, yes?

Good point. We should synthesize points at the start/seek key of range tombstones too, which handles both FailOnMoreRecent and Uncertainty for both scans and gets without point keys. This would replace emitOnSeek.

---

pkg/storage/testdata/mvcc_histories/range_tombstone_scans, line 6 at r8 (raw file):

Previously, nicktrav (Nick Travers) wrote…

nit: looks like there's some extra whitespace here.

Thanks, fixed.

[erikgrinaker]

Ok, this should be ready for a high-level review now.

[erikgrinaker]

Replaced by #82045.