Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: use RangeKeyChanged in ReadAsOfIterator #86513

Merged
merged 1 commit into from
Aug 22, 2022

Conversation

erikgrinaker
Copy link
Contributor

This patch uses RangeKeyChanged() to detect range keys in
ReadAsOfIterator, and caches them to improve performance.
It also fixes a bug where the iterator would fail to detect tombstones
with a non-empty MVCCValueHeader.

Resolves #84714.

Release justification: bug fixes and low-risk updates to new functionality

Release note: None

@erikgrinaker erikgrinaker requested review from tbg and msbutler August 20, 2022 16:15
@erikgrinaker erikgrinaker self-assigned this Aug 20, 2022
@erikgrinaker erikgrinaker requested a review from a team as a code owner August 20, 2022 16:15
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@erikgrinaker
Copy link
Contributor Author

@msbutler I was in the area anyway, so took the opportunity to fix this.

@erikgrinaker erikgrinaker force-pushed the readasof-rangekeychanged branch from 881fd7f to 9795d62 Compare August 21, 2022 14:50
This patch uses `RangeKeyChanged()` to detect range keys in
`ReadAsOfIterator`, and caches them to improve performance. It also
fixes a bug where the iterator would fail to detect tombstones with a
non-empty `MVCCValueHeader`.

Release justification: bug fixes and low-risk updates to new functionality

Release note: None
@erikgrinaker erikgrinaker force-pushed the readasof-rangekeychanged branch from 9795d62 to b184a71 Compare August 21, 2022 14:52
@erikgrinaker erikgrinaker requested a review from a team August 22, 2022 11:36
Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @erikgrinaker and @msbutler)


pkg/storage/read_as_of_iterator.go line 149 at r1 (raw file):

			f.newestRangeTombstone = hlc.Timestamp{}
			if hasRange {
				if v, ok := f.iter.RangeKeys().FirstBelow(f.asOf); ok {

I was surprised to find that FirstBelow is really FirstAtOrBelow. It might be worth renaming the method (ditto for FirstAbove) - I'm always worried about one-offs.

Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @erikgrinaker and @msbutler)

Copy link
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bors r=tbg

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @msbutler and @tbg)


pkg/storage/read_as_of_iterator.go line 149 at r1 (raw file):

Previously, tbg (Tobias Grieger) wrote…

I was surprised to find that FirstBelow is really FirstAtOrBelow. It might be worth renaming the method (ditto for FirstAbove) - I'm always worried about one-offs.

Sure, will submit a separate PR in a bit.

Copy link
Collaborator

@msbutler msbutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this!!

Where exactly was the empty MVCC value header bug? Did it relate to how we used to detect tombstones len(f.iter.UnsafeValue()) == 0?

@@ -129,43 +133,40 @@ func (f *ReadAsOfIterator) updateValid() bool {
// advance moves past keys with timestamps later than f.asOf and skips MVCC keys
// whose latest value (subject to f.asOF) has been deleted by a point or range
// tombstone.
Copy link
Collaborator

@msbutler msbutler Aug 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conceptually, we need to pass seeked ==false to get info on f.newestRangeTombstone, since RangeKeyChanged() may be false, yet we still might have a rangekey to worry about, right?

Next time I make edits to this code, I might add a comment that explains this, or push this logic outside of advance and into readAsOfIterator.SeekGE to remove the parameter in advance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Because we're getting an external iterator, whose state is unknown to us, the SeekGE call may land on a range key that the iterator was already positioned on, so we need to explicitly reset the state on any SeekGE call too.

@erikgrinaker
Copy link
Contributor Author

Where exactly was the empty MVCC value header bug? Did it relate to how we used to detect tombstones len(f.iter.UnsafeValue()) == 0?

Correct. Although I suppose this wouldn't fire when we're reading from SSTs, where we currently only have empty value headers. If we add some though, e.g. the import job ID, this would fail.

@craig
Copy link
Contributor

craig bot commented Aug 22, 2022

Build succeeded:

@craig craig bot merged commit 9c7db33 into cockroachdb:master Aug 22, 2022
@erikgrinaker erikgrinaker deleted the readasof-rangekeychanged branch August 23, 2022 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storage: ensure readAsOfIterator caches rangekeys
4 participants