Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: enforce no split user keys in CheckOrdering #3085

Merged
merged 1 commit into from
Nov 21, 2023

Conversation

jbowens
Copy link
Collaborator

@jbowens jbowens commented Nov 17, 2023

Adapt CheckOrdering to perform its overlap comparisons using the stricter requirement that adjacent files not contain the same user key. Since we still support the earlier format major versions that predate this requirement, this behavior is gated behind an OrderingInvariants enum variant determined by the active format major version.

Some of the tests that use more recent format major versions still created test cases with split user keys, and they needed to be adjusted.

Informs #3064.

@jbowens jbowens requested review from a team and aadityasondhi November 17, 2023 22:58
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote this because I was growing concerned about the complexity of the ingest codepath and the possibility we might violate this invariant by ingesting a user key into a level it already exists within. As far as I can tell, we're not doing that, but this stronger assertion will ensure it.

Reviewable status: 0 of 20 files reviewed, all discussions resolved (waiting on @aadityasondhi, @RaduBerinde, and @sumeerbhola)

Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 20 of 20 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @aadityasondhi, @jbowens, and @RaduBerinde)


format_major_version.go line 222 at r1 (raw file):

}

func (v FormatMajorVersion) orderingInvariants() manifest.OrderingInvariants {

Maybe add a comment that OrderingInvariants only get stricter as the format major version advances. So it is ok that someone retrieves an invariant and by the time it checks the invariant the FMV has advanced.

Adapt CheckOrdering to perform its overlap comparisons using the stricter
requirement that adjacent files not contain the same user key. Since we still
support the earlier format major versions that predate this requirement, this
behavior is gated behind an OrderingInvariants enum variant determined by the
active format major version.

Some of the tests that use more recent format major versions still created test
cases with split user keys, and they needed to be adjusted.
@jbowens jbowens force-pushed the check-ordering-splituserkeys branch from 6cc49f3 to 1bfc681 Compare November 21, 2023 01:45
Copy link
Collaborator Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: 19 of 20 files reviewed, 1 unresolved discussion (waiting on @aadityasondhi, @RaduBerinde, and @sumeerbhola)


format_major_version.go line 222 at r1 (raw file):

Previously, sumeerbhola wrote…

Maybe add a comment that OrderingInvariants only get stricter as the format major version advances. So it is ok that someone retrieves an invariant and by the time it checks the invariant the FMV has advanced.

Good call, done.

@jbowens jbowens merged commit 407f860 into cockroachdb:master Nov 21, 2023
11 checks passed
@jbowens jbowens deleted the check-ordering-splituserkeys branch November 21, 2023 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants