Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: remove support for handling physically interleaved intents #89388

Merged
merged 1 commit into from
Oct 5, 2022

Conversation

sumeerbhola
Copy link
Collaborator

Release note: None

@sumeerbhola sumeerbhola requested a review from a team as a code owner October 5, 2022 14:30
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking this over, good catches as always.

We have a fair bit of other code related to interleaved intents too, would be good to remove it all. Fine with a separate PR though. For example:

// ScanInterleavedIntentsRequest is the request for a ScanInterleavedIntents operation.
// This is a read-only operation that returns all interleaved (non-separated)
// intents found over the request range.
message ScanInterleavedIntentsRequest {
RequestHeader header = 1 [(gogoproto.nullable) = false, (gogoproto.embed) = true];
}

var interleavedIntentCount, separatedIntentCount int

func (db *DB) ScanInterleavedIntents(

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @jbowens and @sumeerbhola)


pkg/storage/intent_interleaving_iter.go line 782 at r1 (raw file):

		// iter could now be at a bare range key that is equal to the intentIter
		// key.
		if err := i.maybeSkipIntentRangeKey(); err != nil {

Good catch, thanks. This seems problematic. Can you pull this out as a separate commit or PR and write a token GA-blocker issue for it, so we can backport to 22.2.0?


pkg/storage/intent_interleaving_iter_test.go line 185 at r1 (raw file):

}

// TODO(erikgrinaker,jackson): enhance TestIntentInterleavingIter with range

Is this not sufficiently covered by the TestMVCCHistories cases, which have a fair bit of intents in them?

Copy link
Collaborator Author

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Created #89428 to point to your list.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @erikgrinaker and @jbowens)


pkg/storage/intent_interleaving_iter.go line 782 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

Good catch, thanks. This seems problematic. Can you pull this out as a separate commit or PR and write a token GA-blocker issue for it, so we can backport to 22.2.0?

I think it cannot occur in practice, so not really a GA-blocker.

  • non-prefix iteration: even though pebble.Iterator.*WithLimit methods claim that the limit enforcement is best-effort, in practice it is not (to make the behavior deterministic for testing inside Pebble's metamorphic test). So if i.iter.Next stepped onto a new roachpb.Key, which is the only case where we could have a range but not a point, then i.intentIterState == pebble.IterAtLimit will also be true (since the previous limit would have been respected).
  • prefix iteration: we cannot step onto a different roachpb.Key in i.iter.Next.

I made the change because I was worried about making an assumption about the Pebble implementation that may break in the future.
I'm a bit wary of backporting since when iterating over many MVCC versions of the same key, this extra call to maybeSkipIntentRangeKey can add cost, and don't want to add more performance work on top of what is already being done by folks for 22.2 regressions. Thoughts?


pkg/storage/intent_interleaving_iter_test.go line 185 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

Is this not sufficiently covered by the TestMVCCHistories cases, which have a fair bit of intents in them?

I took another look at TestMVCCHistories, and realized I had overlooked the iteration testing via the iter-* commands, which is quite comprehensive, so removed this todo.

Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @jbowens and @sumeerbhola)


pkg/storage/intent_interleaving_iter.go line 782 at r1 (raw file):

Previously, sumeerbhola wrote…

I think it cannot occur in practice, so not really a GA-blocker.

  • non-prefix iteration: even though pebble.Iterator.*WithLimit methods claim that the limit enforcement is best-effort, in practice it is not (to make the behavior deterministic for testing inside Pebble's metamorphic test). So if i.iter.Next stepped onto a new roachpb.Key, which is the only case where we could have a range but not a point, then i.intentIterState == pebble.IterAtLimit will also be true (since the previous limit would have been respected).
  • prefix iteration: we cannot step onto a different roachpb.Key in i.iter.Next.

I made the change because I was worried about making an assumption about the Pebble implementation that may break in the future.
I'm a bit wary of backporting since when iterating over many MVCC versions of the same key, this extra call to maybeSkipIntentRangeKey can add cost, and don't want to add more performance work on top of what is already being done by folks for 22.2 regressions. Thoughts?

I see, thanks. In that case I agree, let's not much around with hot path backports this late in the game, and leave this for 23.1.

@sumeerbhola
Copy link
Collaborator Author

bors r=erikgrinaker

@craig
Copy link
Contributor

craig bot commented Oct 5, 2022

Build succeeded:

@craig craig bot merged commit 81586f6 into cockroachdb:master Oct 5, 2022
@sumeerbhola sumeerbhola deleted the iiter_cleanup branch November 18, 2022 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants