Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

catalog: add SystemDatabaseSchemaVersion field to DatabaseDescriptor #111404

Merged
merged 2 commits into from
Oct 4, 2023

Conversation

andyyang890
Copy link
Collaborator

@andyyang890 andyyang890 commented Sep 28, 2023

catalog: add SystemDatabaseSchemaVersion field to DatabaseDescriptor

This patch adds a new SystemDatabaseSchemaVersion field to
descpb.DatabaseDescriptor that will be used to store the schema
version of the system database.

Release note: None


upgrades: add helper function to bump SystemDatabaseSchemaVersion

This patch adds a helper function that should be used by upgrades that
create or modify the schema of system tables to bump the schema version
field on the system database descriptor.

Release note: None


Fixes #109039

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@andyyang890 andyyang890 changed the title upgrades: use descs.DB instead of InternalExecutor for migrateTable upgrades: use descs.Txn instead of InternalExecutor for migrateTable Sep 29, 2023
@andyyang890 andyyang890 force-pushed the migrate_table_txn branch 2 times, most recently from 1525bcb to 773258c Compare September 29, 2023 06:16
@andyyang890 andyyang890 changed the title upgrades: use descs.Txn instead of InternalExecutor for migrateTable catalog,upgrades: add internal database version to database descriptor Sep 29, 2023
@andyyang890 andyyang890 force-pushed the migrate_table_txn branch 3 times, most recently from 6928098 to 15875bb Compare September 29, 2023 20:48
@andyyang890 andyyang890 changed the title catalog,upgrades: add internal database version to database descriptor catalog,upgrades: add InternalDatabaseSchemaVersion field to DatabaseDescriptor Sep 29, 2023
@andyyang890 andyyang890 changed the title catalog,upgrades: add InternalDatabaseSchemaVersion field to DatabaseDescriptor catalog: add InternalDatabaseSchemaVersion field to DatabaseDescriptor Sep 29, 2023
@andyyang890 andyyang890 force-pushed the migrate_table_txn branch 2 times, most recently from 21028de to 235f465 Compare September 30, 2023 18:46
@andyyang890 andyyang890 changed the title catalog: add InternalDatabaseSchemaVersion field to DatabaseDescriptor catalog: add SystemDatabaseSchemaVersion field to DatabaseDescriptor Sep 30, 2023
@cockroachdb cockroachdb deleted a comment from blathers-crl bot Sep 30, 2023
@andyyang890 andyyang890 force-pushed the migrate_table_txn branch 5 times, most recently from bc3b422 to b0efa2a Compare October 1, 2023 05:47
@andyyang890 andyyang890 marked this pull request as ready for review October 1, 2023 07:06
@andyyang890 andyyang890 requested a review from a team as a code owner October 1, 2023 07:06
@andyyang890 andyyang890 requested review from dt, knz, rafiss and fqazi October 1, 2023 07:07
pkg/sql/catalog/systemschema/system.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor things. Great work, Andy!

Reviewed 5 of 10 files at r1, 12 of 12 files at r3, 7 of 7 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andyyang890, @knz, and @rafiss)


pkg/sql/catalog/systemschema/system.go line 1059 at r3 (raw file):

// that should be used during bootstrap. It should be bumped up alongside any
// upgrade that creates or modifies the schema of a system table.
var SystemDatabaseSchemaBootstrapVersion = clusterversion.ByKey(clusterversion.V23_2_RegionaLivenessTable)

Should we add something to the bootstrap test so that we make sure people bump this up?


pkg/upgrade/upgrades/schema_changes.go line 428 at r4 (raw file):

		}
		if sv := systemDBDesc.GetSystemDatabaseSchemaVersion(); sv != nil {
			if cs.Version.Less(*sv) {

I think the validation layer will also catch it right before we write. So this might be redundant?

This patch adds a new `SystemDatabaseSchemaVersion` field to
`descpb.DatabaseDescriptor` that will be used to store the schema
version of the system database.

Release note: None
This patch adds a helper function that should be used by upgrades that
create or modify the schema of system tables to bump the schema version
field on the `system` database descriptor.

Release note: None
Copy link
Collaborator Author

@andyyang890 andyyang890 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @fqazi, @knz, and @rafiss)


pkg/sql/catalog/systemschema/system.go line 1059 at r3 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Should we add something to the bootstrap test so that we make sure people bump this up?

Added a new test to check this!


pkg/upgrade/upgrades/schema_changes.go line 428 at r4 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

I think the validation layer will also catch it right before we write. So this might be redundant?

We discussed this offline already but just noting it here for posterity, the check in ValidateSelf is checking that we're setting it to a valid version relative to BinaryMinSupportedVersion and BinaryVersion while this check is making sure we can't decrease the version.

We also discussed moving that check to ValidateSelf but since that operates only on an *dbdesc.immutable (not an *dbdesc.Mutable which has access to the previous descriptor), we decided that was too complex and unnecessary for now.

@andyyang890 andyyang890 requested a review from fqazi October 4, 2023 00:20
Copy link
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 9 of 9 files at r5, 7 of 7 files at r6, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andyyang890, @knz, and @rafiss)


pkg/sql/catalog/systemschema/system.go line 1059 at r3 (raw file):

Previously, andyyang890 (Andy Yang) wrote…

Added a new test to check this!

Great work 😊

@andyyang890
Copy link
Collaborator Author

TFTRs!

bors r=dt,fqazi

@craig
Copy link
Contributor

craig bot commented Oct 4, 2023

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql/catalog: Add a cluster version for system / internal database descriptors
4 participants