-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
catalog, catalogkv: add and use descriptor builder #61429
Conversation
postamar
commented
Mar 3, 2021
•
edited
Loading
edited
7499e1e
to
151dd95
Compare
ae77b36
to
3c3e135
Compare
3dd8fbd
to
1cbd4ba
Compare
pkg/ccl/backupccl/backup_test.go
Outdated
@@ -2364,9 +2364,9 @@ CREATE TABLE d.t1 (x d.farewell); | |||
RESTORE DATABASE d FROM 'nodelocal://0/rev-history-backup' | |||
AS OF SYSTEM TIME %s | |||
`, ts1)) | |||
sqlDB.ExpectErr(t, `pq: type "d.public.farewell" already exists`, | |||
sqlDB.ExpectErr(t, `pq: type "farewell" already exists`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This message change and others like it are caused by having unified the descriptor collision logic in CheckObjectCollision
in catalogkv
.
return desc | ||
mkTable := func(descriptor tbDesc) catalog.Descriptor { | ||
descProto := tabledesc.NewBuilder(&descriptor).BuildImmutable().DescriptorProto() | ||
return catalogkv.NewBuilderWithMVCCTimestamp(descProto, ts1).BuildImmutable() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not necessary to use NewBuilderWithMVCCTimestamp
here, strictly speaking, but it's the nice way of maybe-setting a modification time.
descs = append(descs, db) | ||
} | ||
return catalog.ValidateSelfAndCrossReferences(ctx, bdg, descs...) | ||
return nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked into this in detail while chasing an unrelated bug and realized that there's no point to these checks here because the validation-on-write checks kick in very soon after.
|
||
var updatedPrivileges *descpb.PrivilegeDescriptor | ||
|
||
) (updatedPrivileges *descpb.PrivilegeDescriptor, err error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this is a bug, we still have to update privileges on system tables when doing a cluster restore.
var privDesc *descpb.PrivilegeDescriptor | ||
b := catalogkv.NewBuilder(descriptor) | ||
if b == nil { | ||
return "" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This happens when descriptor.Union == nil
.
} | ||
return nil | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I replaced this with CheckObjectCollision
because it's the same thing.
@@ -914,15 +914,15 @@ func newTestSpec( | |||
switch format.Format { | |||
case roachpb.IOFileFormat_CSV: | |||
descr = descForTable(ctx, t, | |||
"CREATE TABLE simple (i INT PRIMARY KEY, s text )", 10, 20, NoFKs) | |||
"CREATE TABLE simple (i INT PRIMARY KEY, s text )", 100, 200, NoFKs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These descriptor IDs were in the system range and this caused privilege validation failures.
@@ -223,7 +224,7 @@ func (m *randomStreamClient) getDescriptorAndNamespaceKVForTableID( | |||
IngestionDatabaseID, | |||
tableID, | |||
fmt.Sprintf(RandomStreamSchemaPlaceholder, tableName), | |||
&descpb.PrivilegeDescriptor{}, | |||
descpb.NewDefaultPrivilegeDescriptor(security.RootUserName()), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that we don't insta-upgrade our privilege descriptors when validating, we have to be more careful when crafting test cases.
if err != nil { | ||
return err | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needed to be moved a tiny bit further up or else there would be cross-reference validation failures!
pkg/sql/alter_table_set_schema.go
Outdated
@@ -80,7 +80,7 @@ func (p *planner) AlterTableSetSchema( | |||
for _, dependent := range tableDesc.DependedOnBy { | |||
if !dependent.ByID { | |||
return nil, p.dependentViewError( | |||
ctx, tableDesc.TypeName(), tableDesc.Name, | |||
ctx, string(tableDesc.TypeName()), tableDesc.Name, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TypeName()
now returns a catalog.DescriptorType
which aliases string
, for extra type safety.
@@ -802,7 +802,11 @@ func TruncateInterleavedIndexes( | |||
// All the data chunks have been removed. Now also removed the | |||
// zone configs for the dropped indexes, if any. | |||
if err := db.Txn(ctx, func(ctx context.Context, txn *kv.Txn) error { | |||
return RemoveIndexZoneConfigs(ctx, txn, execCfg, table.GetParentID(), indexes) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm certain that table.GetParentID()
instead of table.GetID()
is a bug here but found no way to test it. It only involves zone configs on interleaved indexes so I'm not super motivated to do more.
if err != nil { | ||
return err | ||
} | ||
return RemoveIndexZoneConfigs(ctx, txn, execCfg, freshTableDesc, indexes) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed the signature of RemoveIndexZoneConfigs
to take a catalog.TableDescriptor
because otherwise it loads it from storage using a catalogkv api call which causes cross-reference validation check failures in indexTruncateInTxn
.
This is another of these new failures which cropped up after I raised the validation level for mutable descs.
indexDescs []descpb.IndexDescriptor, | ||
) error { | ||
if !execCfg.Codec.ForSystemTenant() { | ||
// Tenants are agnostic to zone configs. | ||
return nil | ||
} | ||
desc, err := catalogkv.GetDescriptorByID(ctx, txn, execCfg.Codec, tableID, | ||
catalogkv.Mutable, catalogkv.TableDescriptorKind, true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return errors.AssertionFailedf("found no owner for system %s with ID=%d", | ||
objectType, id) | ||
return errors.AssertionFailedf("found no owner for %s%s with ID=%d", | ||
maybeSystem, objectType, id) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some reason the error messages here describe everything as "system" even when they're not system tables or anything.
@@ -1423,12 +1428,12 @@ var ( | |||
}, | |||
NextIndexID: 2, | |||
Privileges: descpb.NewCustomSuperuserPrivilegeDescriptor( | |||
descpb.SystemAllowedPrivileges[keys.ReplicationStatsTableID], security.NodeUserName()), | |||
descpb.SystemAllowedPrivileges[keys.ProtectedTimestampsMetaTableID], security.NodeUserName()), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bad copy pasta
pkg/sql/opt/optbuilder/scope.go
Outdated
@@ -35,7 +35,7 @@ import ( | |||
// been bound within the current scope as columnProps. Variables bound in the | |||
// parent scope are also visible in this scope. | |||
// | |||
// See builder.go for more details. | |||
// See table_desc_builder.go for more details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll fix this
} else if err != nil { | ||
return err | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check is entirely redundant.
@@ -569,7 +567,7 @@ func (p *planner) UnsafeDeleteDescriptor(ctx context.Context, descID int64, forc | |||
mut, err := p.Descriptors().GetMutableDescriptorByID(ctx, id, p.txn) | |||
var forceNoticeString string // for the event | |||
if err != nil { | |||
if !errors.Is(err, catalog.ErrDescriptorNotFound) && force { | |||
if force { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think force
is enough here. The other test was failing because ErrDescriptorNotFound was being returned by a failing back-reference check.
desc, err = lCtx.getTableByID(desc.GetID()) | ||
if err != nil { | ||
return "", err | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously this used to work without this because we were shallow-copying on descriptor construction. Deep-copy makes this necessary.
{ "privileges": 2, "user_proto": "newuser1" }, | ||
{ "privileges": 2, "user_proto": "newuser2" } | ||
{ "privileges": 2, "user_proto": "newuser2" }, | ||
{ "privileges": 2, "user_proto": "root" } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apparently order matters here.
@@ -103,7 +103,7 @@ func TestDescriptorRepairOrphanedDescriptors(t *testing.T) { | |||
// the dance of adding back a parent database in order to drop the table. | |||
require.NoError(t, crdb.ExecuteTx(ctx, db, nil, func(tx *gosql.Tx) error { | |||
if _, err := tx.Exec( | |||
"SELECT crdb_internal.unsafe_delete_descriptor($1);", | |||
"SELECT crdb_internal.unsafe_delete_descriptor($1, true);", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Required due to more stringent validation-on-read.
dg catalog.DescGetter, | ||
desc *descpb.TableDescriptor, | ||
skipFKsWithNoMatchingTable bool, | ||
) (changes PostDeserializationTableDescriptorChanges, err error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved as-is from structured.go, along with dependent functions
@@ -0,0 +1,91 @@ | |||
// Copyright 2021 The Cockroach Authors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved all of this out of catalogkv.go
, only big new thing is fromKeyValue
method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 148 of 153 files at r1.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @dt and @postamar)
pkg/ccl/backupccl/backup_test.go, line 2367 at r1 (raw file):
Previously, postamar (Marius Posta) wrote…
This message change and others like it are caused by having unified the descriptor collision logic in
CheckObjectCollision
incatalogkv
.
aye, not great though; now the error can be pretty ambiguous.
pkg/ccl/backupccl/restore_job.go, line 436 at r1 (raw file):
Previously, postamar (Marius Posta) wrote…
I looked into this in detail while chasing an unrelated bug and realized that there's no point to these checks here because the validation-on-write checks kick in very soon after.
nice.
pkg/ccl/backupccl/restore_job.go, line 2069 at r1 (raw file):
Previously, postamar (Marius Posta) wrote…
I believe this is a bug, we still have to update privileges on system tables when doing a cluster restore.
say more?
pkg/ccl/backupccl/targets.go, line 194 at r1 (raw file):
// Collect the prior IDs of table descriptors, as the ID may have been // changed during truncate.
can you add a prior to 20.2
after truncate. We no longer change the IDs.
pkg/ccl/changefeedccl/schemafeed/schema_feed.go, line 624 at r1 (raw file):
b := catalogkv.NewBuilderWithMVCCTimestamp(&desc, k.Timestamp) if b != nil && (b.TypeName() == catalog.Table || b.TypeName() == catalog.Database) {
should this second one be catalog.Type
?
pkg/sql/show_create.go, line 263 at r1 (raw file):
Previously, postamar (Marius Posta) wrote…
Previously this used to work without this because we were shallow-copying on descriptor construction. Deep-copy makes this necessary.
oh man
pkg/sql/catalog/descriptor.go, line 56 at r1 (raw file):
// TypeName returns a symbol identifying the type of the descriptor // built by this builder. TypeName() DescriptorType
nit: how about you call this method DescriptorType()
?
pkg/sql/catalog/descriptor.go, line 60 at r1 (raw file):
// RunPostDeserializationChanges attempts to perform post-deserialization // changes to the descriptor being built. RunPostDeserializationChanges(ctx context.Context, dg DescGetter) error
can you comment on the contract with the DescGetter
?
pkg/sql/catalog/descriptor.go, line 109 at r1 (raw file):
GetPrivileges() *descpb.PrivilegeDescriptor TypeName() DescriptorType
how about calling this DescriptorType()
too?
pkg/sql/catalog/catalogkv/catalogkv.go, line 176 at r1 (raw file):
// builderFromKeyValue is a utility function for descriptorFromKeyValue which // unmarshals the proto and checks that it exists and that it matches the
nit: extra space
pkg/sql/catalog/catalogkv/unwrap_validation_test.go, line 65 at r1 (raw file):
desc, err := m.GetDesc(context.Background(), id) require.NoErrorf(t, err, "id: %d", id) require.NotNil(t, desc, "id: %d", id)
NotNilf
?
pkg/sql/catalog/descpb/descriptor.go, line 200 at r1 (raw file):
// FromDescriptorWithMVCCTimestamp is a replacement for // Get(Table|Database|TypeSchema)() methods which seeks to ensure that clients
Type|Schema
?
pkg/sql/catalog/descpb/privilege.go, line 302 at r1 (raw file):
Previously, postamar (Marius Posta) wrote…
For some reason the error messages here describe everything as "system" even when they're not system tables or anything.
I'm sure it's just code rot.
pkg/sql/catalog/descpb/privilege_test.go, line 353 at r1 (raw file):
rootWrongPrivilegesErr := "user root must have exactly SELECT, GRANT " + "privileges on s?y?s?t?e?m? ?table with ID=.*"
why not just (system )?
pkg/sql/catalog/systemschema/system.go, line 1431 at r1 (raw file):
Previously, postamar (Marius Posta) wrote…
bad copy pasta
🤦
pkg/sql/logictest/testdata/logic_test/enums, line 28 at r1 (raw file):
SELECT * FROM db2.t statement error pq: type "t" already exists
this highlights something interesting. Getting the error handling right here so that the right level understands enough context to construct a detailed error is not currently very easy.
pkg/sql/rowexec/tablereader.go, line 91 at r1 (raw file):
tr.maxTimestampAge = time.Duration(spec.MaxTimestampAgeNanos) tableDesc := tabledesc.NewBuilder(&spec.Table).BuildImmutableTable()
so, I feel like the copying behavior of the builder is going to hurt here. We end up calling this right here a lot. We should fix that so that instead we look up a cached descriptor but this new set of allocations per table reader most gives me pause to backporting this.
1cbd4ba
to
5a83477
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 153 files at r1, 87 of 87 files at r2.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @dt and @postamar)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review. Although we discussed things verbally I'll still respond to the comments here for the record.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner, @dt, and @postamar)
pkg/ccl/backupccl/backup_test.go, line 2367 at r1 (raw file):
Previously, ajwerner wrote…
aye, not great though; now the error can be pretty ambiguous.
Fixed by having CheckObjectCollision
take a tree.ObjectName
instead.
pkg/ccl/backupccl/restore_job.go, line 2069 at r1 (raw file):
Previously, ajwerner wrote…
say more?
To clarify: the existing behaviour was correct except when restoring to the temporary system DB. Those backed-up tables came from system tables and as such may not have r+w permissions for superusers, causing validation to freak out.
We never noticed this until now because r+w permissions for superusers used to be reset by the privilege descriptor surgery in ValidateSelf.
This change here grants those tables the same permissions as restoreTempSystemDB
which is good enough, the contents of these restored tables get inserted into the real system tables later on.
pkg/ccl/backupccl/targets.go, line 194 at r1 (raw file):
Previously, ajwerner wrote…
can you add a
prior to 20.2
after truncate. We no longer change the IDs.
Done.
pkg/ccl/changefeedccl/schemafeed/schema_feed.go, line 624 at r1 (raw file):
Previously, ajwerner wrote…
should this second one be
catalog.Type
?
Fixed.
pkg/sql/catalog/descriptor.go, line 56 at r1 (raw file):
Previously, ajwerner wrote…
nit: how about you call this method
DescriptorType()
?
Done.
pkg/sql/catalog/descriptor.go, line 60 at r1 (raw file):
Previously, ajwerner wrote…
can you comment on the contract with the
DescGetter
?
Done.
pkg/sql/catalog/descriptor.go, line 109 at r1 (raw file):
Previously, ajwerner wrote…
how about calling this
DescriptorType()
too?
Done.
pkg/sql/catalog/catalogkv/catalogkv.go, line 176 at r1 (raw file):
Previously, ajwerner wrote…
nit: extra space
Done.
pkg/sql/catalog/catalogkv/unwrap_validation_test.go, line 65 at r1 (raw file):
Previously, ajwerner wrote…
NotNilf
?
Done.
pkg/sql/catalog/descpb/descriptor.go, line 200 at r1 (raw file):
Previously, ajwerner wrote…
Type|Schema
?
Done.
pkg/sql/catalog/descpb/privilege_test.go, line 353 at r1 (raw file):
Previously, ajwerner wrote…
why not just
(system )?
Done
pkg/sql/rowexec/tablereader.go, line 91 at r1 (raw file):
Previously, ajwerner wrote…
so, I feel like the copying behavior of the builder is going to hurt here. We end up calling this right here a lot. We should fix that so that instead we look up a cached descriptor but this new set of allocations per table reader most gives me pause to backporting this.
I ran the following microbenchmark: BenchmarkKV/SQL
with the kv
table modified to gratuitously fatten its descriptor. I did this by adding an extra 10 stored computed columns and an extra 10 secondary indexes. Using the master
branch as a control I found the difference in allocs/op to not be significant. I was honestly expecting it to be worse than that. My working hypothesis right now is that the cost of these deep-copy allocations is amortized by the other allocations we already do when creating a tabledesc.immutable.
bors r+ |
Release justification: bug fixes and low-risk updates to new functionality. This commit is, at heart, a refactor of how catalog.Descriptor objects are created; either when read from storage via the catalogkv API or when constructed from an in-memory descpb.*Descriptor struct. The original motivation for this change was the fact that previously, calling the descriptor validation methods could sometimes modify the descriptor itself. Specifically, ValidateSelf would perform "privilege surgery" and modify the privileges descriptor. This appears to have been around since version 2 and is, in fact, responsible for a few bugs, which this commit also fixes. Taking a broader perspective it appears necessary that all such descriptor surgeries should be performed together when read from storage regardless of descriptor subtype (table, db, type, schema). These surgeries are typically no-ops: table format upgrade, foreign key representation upgrade, etc. Considering that catalog.Descriptor objects can be created either from storage or from memory, this involves: 1. unifying the constructors in the (table|db|type|schema)desc packages, 2. unifying the descriptor unwrapping logic in catalogkv. To address (1) this commit introduces catalog.DescriptorBuilder, a builder interface which replaces the NewImmutable, NewExistingMutable, NewCreatedMutable, etc. constructors. The builder does the same thing as these except it uses a deep copy of the proto descriptor instead of a shallow copy. This has, in itself, uncovered a few minor bugs. Still, the main reason for the existence of this builder object is that we can now perform all post-deserialization descriptor changes in a method to be called before building the catalog.Descriptor object proper. This commit also changes how we update the descriptor modification time using the MVCC timestamp. This is done at builder creation time, or, in cases where we stick to descpb structs, using a new descp.FromDescriptorWithMVCCTimestamp function which replaces descpb.TableFromDescriptor and the descpb.Descriptor.GetTable, GetDatabase, GetType and GetSchema methods. These methods are now all forbidden from casual use by a new linter check (it used to be just GetTable). To address (2) this commit uses the new builder to unify all descriptor wrapping logic in catalogkv into a new descriptorFromKeyValue function. This function is what backs the catalogkv API now. This commit also changes the API somewhat, the old GetDescriptorFromID function with its many flag arguments is gone in favour of many lightweight public getter functions (complementing existing ones) which all delegate to a common private getDescriptorByID function. The most important side benefit to doing all this is that we now no longer skip cross-reference validation when getting mutable descriptors through the catalogkv API. Cross-reference validation is now the default, and the exceptions to this are few: - The catalogkv DescGetter implemementation, aka the oneLevelUncachedDescGetter, does not perform these checks, because they, in turn, require a DescGetter. In theory we could get around this infinite recursion with caching but right now there's no point. - GetAllDescriptorsUnvalidated explicitly performs no validation at all. Doing all of this has uncovered a few bugs, the most serious of which is that the protected_ts_meta system table didn't have the correct permissions applied. Furthermore this commit also unifies namespace collision detection logic in catalogkv. A side-effect of this is that error messages for PG codes 42710 and 42P07 contain qualified names more often than before. Release note (bug fix): Fixed privileges for system.protected_ts_meta.
5a83477
to
15ee387
Compare
Canceled. |
rebased to fix merge conflict |
bors r+ |
Build succeeded: |