-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
73537: sql: insert missing public schema entry migration r=ajwerner a=RichardJCai When restoring a database, a namespace entry for the public schema was not created. Release note: None 73614: kvserver: alter balanceScore result to classify stores into 3 buckets r=aayushshah15 a=aayushshah15 In #65379 we changed `balanceScore()` to classify stores in a cluster into 4 buckets: underfull, less-than-mean, more-than-mean and overfull. This allowed for the possibility of thrashing around the mean (i.e. replicas could ping-pong between stores classified as less-than-mean and more-than-mean). This patch moves balanceScore back to its previous incarnation, which only divided resulting stores into 3 buckets. Release justification: Fixes regression introduced in a previous patch Release note: None 73754: keys,*: adopt SystemIDChecker r=postamar a=postamar This commit removes many references to `keys` package constants 49, 50 and 54, and replaces them with functions that take a SystemIDChecker. Existing functionality should remain unchanged. This is a prerequisite to making the system descriptor ID space dynamic. Release note: None 73767: backupccl: remove stitching queue file count ceiling r=dt a=adityamaru This change removes the maxQueueSize that limited the number of files that could be buffered in the queue when merging SSTs during a backup. The efficacy and need of this cap is not evident, and we already have a byte limit on how large the queue can grow. This reduces the number of variables that need to be tuned to achieve more optimal file merging behaviour. Informs: #73815 Release note: None Co-authored-by: richardjcai <[email protected]> Co-authored-by: Aayush Shah <[email protected]> Co-authored-by: Marius Posta <[email protected]> Co-authored-by: Aditya Maru <[email protected]>
- Loading branch information
Showing
93 changed files
with
748 additions
and
406 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
93 changes: 93 additions & 0 deletions
93
pkg/ccl/backupccl/insert_missing_public_schema_namespace_entry_restore_test.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| // Copyright 2021 The Cockroach Authors. | ||
| // | ||
| // Licensed as a CockroachDB Enterprise file under the Cockroach Community | ||
| // License (the "License"); you may not use this file except in compliance with | ||
| // the License. You may obtain a copy of the License at | ||
| // | ||
| // https://github.com/cockroachdb/cockroach/blob/master/licenses/CCL.txt | ||
|
|
||
| package backupccl_test | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "testing" | ||
|
|
||
| "github.com/cockroachdb/cockroach/pkg/base" | ||
| "github.com/cockroachdb/cockroach/pkg/clusterversion" | ||
| "github.com/cockroachdb/cockroach/pkg/keys" | ||
| "github.com/cockroachdb/cockroach/pkg/kv" | ||
| "github.com/cockroachdb/cockroach/pkg/server" | ||
| "github.com/cockroachdb/cockroach/pkg/sql/catalog/catalogkeys" | ||
| "github.com/cockroachdb/cockroach/pkg/sql/catalog/descpb" | ||
| "github.com/cockroachdb/cockroach/pkg/testutils" | ||
| "github.com/cockroachdb/cockroach/pkg/testutils/sqlutils" | ||
| "github.com/cockroachdb/cockroach/pkg/testutils/testcluster" | ||
| "github.com/cockroachdb/cockroach/pkg/util/leaktest" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func TestInsertMissingPublicSchemaNamespaceEntry(t *testing.T) { | ||
| defer leaktest.AfterTest(t)() | ||
| ctx := context.Background() | ||
| dir, cleanup := testutils.TempDir(t) | ||
| defer cleanup() | ||
| tc := testcluster.StartTestCluster(t, 3, base.TestClusterArgs{ | ||
| ServerArgs: base.TestServerArgs{ | ||
| ExternalIODir: dir, | ||
| Knobs: base.TestingKnobs{ | ||
| Server: &server.TestingKnobs{ | ||
| DisableAutomaticVersionUpgrade: 1, | ||
| BinaryVersionOverride: clusterversion.ByKey(clusterversion.InsertPublicSchemaNamespaceEntryOnRestore - 1), | ||
| }, | ||
| }, | ||
| }, | ||
| }) | ||
| defer tc.Stopper().Stop(ctx) | ||
|
|
||
| db := tc.ServerConn(0) | ||
| defer db.Close() | ||
| sqlDB := sqlutils.MakeSQLRunner(tc.Conns[0]) | ||
|
|
||
| // Mimic a restore where the public schema system.namespace entries are | ||
| // missing. | ||
| sqlDB.Exec(t, `CREATE DATABASE db1`) | ||
| sqlDB.Exec(t, `CREATE TABLE db1.t()`) | ||
| sqlDB.Exec(t, `CREATE SCHEMA db1.s`) | ||
| sqlDB.Exec(t, `CREATE DATABASE db2`) | ||
| sqlDB.Exec(t, `CREATE TABLE db2.t(x INT)`) | ||
| sqlDB.Exec(t, `INSERT INTO db2.t VALUES (1), (2)`) | ||
| sqlDB.Exec(t, `CREATE SCHEMA db2.s`) | ||
| sqlDB.Exec(t, `CREATE TABLE db2.s.t(x INT)`) | ||
| sqlDB.Exec(t, `INSERT INTO db2.s.t VALUES (1), (2)`) | ||
|
|
||
| var db1ID, db2ID descpb.ID | ||
| row := sqlDB.QueryRow(t, `SELECT id FROM system.namespace WHERE name = 'db1'`) | ||
| row.Scan(&db1ID) | ||
| row = sqlDB.QueryRow(t, `SELECT id FROM system.namespace WHERE name = 'db2'`) | ||
| row.Scan(&db2ID) | ||
|
|
||
| // Remove system.namespace entries for the public schema for the two | ||
| // databases. | ||
| err := tc.Servers[0].DB().Txn(ctx, func(ctx context.Context, txn *kv.Txn) error { | ||
| codec := keys.SystemSQLCodec | ||
| b := txn.NewBatch() | ||
| b.Del(catalogkeys.MakeSchemaNameKey(codec, db1ID, `public`)) | ||
| b.Del(catalogkeys.MakeSchemaNameKey(codec, db2ID, `public`)) | ||
| return txn.Run(ctx, b) | ||
| }) | ||
| require.NoError(t, err) | ||
|
|
||
| // Verify that there are no system.namespace entries for the public schema for | ||
| // the two databases. | ||
| sqlDB.CheckQueryResults(t, fmt.Sprintf(`SELECT id FROM system.namespace WHERE name = 'public' AND "parentID"=%d`, db1ID), [][]string{}) | ||
| sqlDB.CheckQueryResults(t, fmt.Sprintf(`SELECT id FROM system.namespace WHERE name = 'public' AND "parentID"=%d`, db2ID), [][]string{}) | ||
|
|
||
| // Kick off migration by upgrading to the new version. | ||
| _ = sqlDB.Exec(t, `SET CLUSTER SETTING version = $1`, | ||
| clusterversion.ByKey(clusterversion.InsertPublicSchemaNamespaceEntryOnRestore).String()) | ||
|
|
||
| sqlDB.CheckQueryResults(t, fmt.Sprintf(`SELECT id FROM system.namespace WHERE name = 'public' AND "parentID"=%d`, db1ID), [][]string{{"29"}}) | ||
| sqlDB.CheckQueryResults(t, fmt.Sprintf(`SELECT id FROM system.namespace WHERE name = 'public' AND "parentID"=%d`, db2ID), [][]string{{"29"}}) | ||
|
|
||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.