Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: fix mixed-version behaviour of create_tenant() #94774

Merged
merged 2 commits into from
Jan 21, 2023

Conversation

postamar
Copy link
Contributor

@postamar postamar commented Jan 5, 2023

Previously, running create_tenant on the latest binary in
a mixed-version cluster would bootstrap the system schema using the
logic from the latest binary, in which all system database migration
upgrade steps have been "baked in".

This might not be compatible with the the tenant cluster's version which
is (correctly) set to the active host cluster version, the idea there
being that tenant clusters versions cannot be greater than host cluster
versions.

This commit fixes this by version-gating the tenant cluster system
schema bootstrapping, and using hard-coded values when bootstrapping
into the old version.

Informs #94773.

Release note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@postamar postamar force-pushed the data-driven-bootstrap branch from d9ba4d3 to ffc6d7e Compare January 5, 2023 20:58
@postamar postamar changed the title bootstrap: add string representation of initial values sql: fix mixed-version behaviour of create_tenant() Jan 5, 2023
@postamar postamar force-pushed the data-driven-bootstrap branch 3 times, most recently from bfd95b3 to 37d302c Compare January 6, 2023 17:53
@postamar postamar marked this pull request as ready for review January 6, 2023 18:30
@postamar postamar requested a review from a team January 6, 2023 18:30
@postamar postamar requested review from a team as code owners January 6, 2023 18:30
@postamar postamar requested a review from rafiss January 6, 2023 18:30
Copy link
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 5 files at r1, 8 of 8 files at r2, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss)

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @fqazi and @postamar)


pkg/jobs/registry.go line 533 at r4 (raw file):

			ctx, "job-row-update", txn, updateStmt, jobID, jobType.String(),
		); err != nil {
			if pgerror.GetPGCode(err) != pgcode.UndefinedColumn {

consider using select column_name from [show columns from system.jobs] where column_name = 'job_type' rather than executing the query and checking for an error. it should be possible to use pg_catalog also

or if we have the system.jobs descriptor, we could use the FindColumnWithName function


pkg/upgrade/upgrades/permanent_upgrades.go line 71 at r4 (raw file):

		ctx, "addRootToAdminRole", nil /* txn */, updateMembership, username.AdminRole, username.RootUser, username.AdminRoleID, username.RootUserID)
	if err != nil {
		if pgerror.GetPGCode(err) == pgcode.UndefinedColumn {

similar to above, could use show columns here, or inspect the system.role_members descriptor

@postamar postamar added the do-not-merge bors won't merge a PR with this label. label Jan 9, 2023
@postamar postamar force-pushed the data-driven-bootstrap branch from aac5192 to 3db94bb Compare January 12, 2023 15:24
@postamar
Copy link
Contributor Author

Thanks for the suggestion @rafiss, I ended up using a pg_catalog table which should not incur any extra overhead.

@postamar postamar force-pushed the data-driven-bootstrap branch from 3db94bb to 8722605 Compare January 12, 2023 16:03
Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I quite like this. @dt does it please you? What integration testing should accompany this?

//
// This string literal was generated by copying over the expected results
// for the TestInitialValuesToString test in the release-22.2 branch.
nonSystem222 = `[{"key":""}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's this first entry about?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keys without values are split keys and this one is the split we define at the tenant prefix, but in this datum the prefix is stripped away leaving the empty string.

pkg/sql/tenant.go Outdated Show resolved Hide resolved
pkg/upgrade/upgrades/permanent_upgrades.go Outdated Show resolved Hide resolved
@postamar postamar force-pushed the data-driven-bootstrap branch from 8722605 to 6753acb Compare January 18, 2023 20:40
@postamar postamar removed the do-not-merge bors won't merge a PR with this label. label Jan 18, 2023
@postamar postamar force-pushed the data-driven-bootstrap branch 3 times, most recently from 86e28eb to 9bddb8a Compare January 19, 2023 14:28
Marius Posta added 2 commits January 20, 2023 06:05
This commit adds InitialValuesToString and InitialValuesFromString which
convert KV datums and KV split keys from a MetadataSchema to a string
representation and vice-versa. The tenant prefix is omitted from the
string representation.

This functionality exists to allow bootstrapping clusters using the
values and splits as generated by an earlier cockroach binary.

Informs cockroachdb#94773.

Release note: None
Previously, running create_tenant on the latest binary in
a mixed-version cluster would bootstrap the system schema using the
logic from the latest binary, in which all system database migration
upgrade steps have been "baked in".

This might not be compatible with the the tenant cluster's version which
is (correctly) set to the active host cluster version, the idea there
being that tenant clusters versions cannot be greater than host cluster
versions.

This commit fixes this by version-gating the tenant cluster system
schema bootstrapping, and using hard-coded values when bootstrapping
into the old version.

Informs cockroachdb#94773.

Release note: None
@postamar postamar force-pushed the data-driven-bootstrap branch from 9bddb8a to 08f2dc4 Compare January 20, 2023 11:28
@postamar
Copy link
Contributor Author

Thanks for the reviews. I'm going to bors this now before it grows stale.

bors r+

@craig
Copy link
Contributor

craig bot commented Jan 20, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Jan 20, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Jan 21, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Jan 21, 2023

Build succeeded:

@craig craig bot merged commit 1982295 into cockroachdb:master Jan 21, 2023
@postamar postamar deleted the data-driven-bootstrap branch January 21, 2023 01:47
healthy-pod pushed a commit to healthy-pod/cockroach that referenced this pull request Jan 26, 2023
In cockroachdb#94774, we didn't intend to change the TLV and it
is still supposed to be equal to the SLV. Changing the
way we set it however has caused TLV and SLV to be different
if a test uses two binaries, and only one of them was built
with `crdb_test` due to how we add `1e6` to versions in test
binaries.

Closes cockroachdb#95648

Epic: none
Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants