Skip to content

Commit

Permalink
Merge #95622 #95865 #95999 #96115 #96202 #96235 #96328 #96366
Browse files Browse the repository at this point in the history 
95622: backupccl,storage: add logs around manifest handling and ExportRequest pagination r=stevendanna a=adityamaru

backupccl: add logging to backup manifest handling

Release note: None

storage: log the ExportRequest pagination reason

Release note: None

Epic: None

95865: cmd/roachtest: adapt disk-stall detection roachtest r=nicktrav,erikgrinaker a=jbowens

Move the existing disk-stall/* roachtests under disk-stall/fuse/* (for the FUSE
filesystem approach to stalling) and skip them for now. Currently, they're not
capable of stalling the disk longer 50us (see #95886), which makes them
unreliable at exercising stalls.

Add two new roachtests, disk-stall/dmsetup and disk-stall/cgroup that use
dmsetup and cgroup bandwidth restrctions respectively to reliably induce a
write stall for an indefinite duration.

Informs #94373.
Epic: None
Release note: None

95999: multitenant: add multitenant/shared-process/basic roachtest r=stevendanna a=msbutler

This patch introduces a simple roachtest that runs in a shared-process tenant.
This test imports a 500 tpcc workload (about 30 GB of replicated data), and
runs the workload for 10 minutes. The test is run on a 4 node, 4vcpu cluster
with local ssds.

A future patch could complicate the test by running schema changes or other
bulk operations.

Fixes #95990

Release note: None

96115: schemachanger: Implement `DROP CONSTRAINT` in declarative schema changer r=Xiang-Gu a=Xiang-Gu

This PR implements `ALTER TABLE t DROP CONSTRAINT cons_name` in declarative schema changer.

Supported constraints include Checks, FK, and UniqueWithoutIndex.

Dropping PK or Unique constraints will fall back to legacy schema changer, which in turn spits out an "not supported yet" error.

Epic: None

96202: opt: inverted-index accelerate filters of the form j->0 @> '{"b": "c"} r=Shivs11 a=Shivs11

Previously, the optimizer did not plan inverted index scans for filters
having an integer as the index for the fetch value in a filter alongside
the "contains" or the "contained by" operator.

To address this, we now build JSON arrays from fetch value expressions
with integer indexes. From these JSON arrays, inverted spans are built
for constraining scans over inverted indexes. With these changes chains
of both integer and string fetch value operators are now supported
alongside the "contains" and the "contained by" operators.
(e.g., j->0 `@>` '{"b": "c"}' and j->0 <@ '{"b": "c"}').

Epic: [CRDB-3301](https://cockroachlabs.atlassian.net/browse/CRDB-3301)
Fixes: #94667

Release note (performance improvement): The optimizer now plans
inverted index scans for queries that filter by JSON fetch value
operators (->) with integer indices alongside the "contains" or
the "contained by" operators, e.g, json_col->0 `@>` '{"b": "c"}'
or json_col->0 <@ '{"b": "c"}'

96235: sem/tree: add support for producing vectorized data from strings r=cucaroach a=cucaroach

tree.ValueHandler exposes raw machine type hooks that are used by
vec_handler to build coldata.Vec's.

Epic: CRDB-18892
Informs: #91831
Release note: None


96328: udf: allow strict UDF with no arguments r=DrewKimball a=DrewKimball

This patch fixes the case when a strict UDF (returns null on null input) has no arguments. Previously, attempting to call such a function would result in `ERROR: reflect: call of reflect.Value.Pointer on zero Value`.

Fixes #96326

Release note: None

96366: release: skip nil GitHub events r=celiala a=rail

Previously, we referenced `*event.Event`, but in some cases the event objects are `nil`.

This PR skips the nil GitHub event objects.

Epic: none
Release note: None

Co-authored-by: adityamaru <[email protected]>
Co-authored-by: Jackson Owens <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: Xiang Gu <[email protected]>
Co-authored-by: Shivam Saraf <[email protected]>
Co-authored-by: Tommy Reilly <[email protected]>
Co-authored-by: Drew Kimball <[email protected]>
Co-authored-by: Rail Aliiev <[email protected]>
  • Loading branch information
@adityamaru @jbowens
@msbutler @Xiang-Gu @Shivs11 @cucaroach @DrewKimball @rail
9 people committed Feb 1, 2023
9 parents d6a32ed + 17bb2b5 + 2e8f273 + 33a01d2 + 2428a43 + 3ed8616 + 0927174 + e08a60b + 0fc745a commit 08a188a
Show file tree
Hide file tree
Showing 76 changed files with 3,919 additions and 1,071 deletions.
6 changes: 3 additions & 3 deletions pkg/bench/rttanalysis/testdata/benchmark_expectations
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ exp,benchmark
15,AlterTableDropColumn/alter_table_drop_1_column
15,AlterTableDropColumn/alter_table_drop_2_columns
15,AlterTableDropColumn/alter_table_drop_3_columns
9,AlterTableDropConstraint/alter_table_drop_1_check_constraint
9,AlterTableDropConstraint/alter_table_drop_2_check_constraints
9,AlterTableDropConstraint/alter_table_drop_3_check_constraints
11,AlterTableDropConstraint/alter_table_drop_1_check_constraint
11,AlterTableDropConstraint/alter_table_drop_2_check_constraints
11,AlterTableDropConstraint/alter_table_drop_3_check_constraints
9,AlterTableSplit/alter_table_split_at_1_value
13,AlterTableSplit/alter_table_split_at_2_values
17,AlterTableSplit/alter_table_split_at_3_values
Expand Down
10 changes: 9 additions & 1 deletion pkg/ccl/backupccl/backupinfo/manifest_handling.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -164,21 +164,29 @@ func ReadBackupManifestFromStore(
// If we did not find `BACKUP_METADATA` we look for the
// `BACKUP_MANIFEST` file as it is possible the backup was created by a
// pre-23.1 node.
log.VInfof(ctx, 2, "could not find BACKUP_METADATA, falling back to BACKUP_MANIFEST")
backupManifest, backupManifestMemSize, backupManifestErr := ReadBackupManifest(ctx, mem, exportStore,
backupbase.BackupManifestName, encryption, kmsEnv)
if backupManifestErr != nil {
if !errors.Is(backupManifestErr, cloud.ErrFileDoesNotExist) {
return backuppb.BackupManifest{}, 0, err
return backuppb.BackupManifest{}, 0, backupManifestErr
}

// If we did not find a `BACKUP_MANIFEST` we look for a `BACKUP` file as
// it is possible the backup was created by a pre-20.1 node.
//
// TODO(adityamaru): Remove this logic once we disallow restores beyond
// the binary upgrade compatibility window.
log.VInfof(ctx, 2, "could not find BACKUP_MANIFEST, falling back to BACKUP")
oldBackupManifest, oldBackupManifestMemSize, oldBackupManifestErr := ReadBackupManifest(ctx, mem, exportStore,
backupbase.BackupOldManifestName, encryption, kmsEnv)
if oldBackupManifestErr != nil {
if errors.Is(oldBackupManifestErr, cloud.ErrFileDoesNotExist) {
log.VInfof(ctx, 2, "could not find any of the supported backup metadata files")
return backuppb.BackupManifest{}, 0,
errors.Wrapf(oldBackupManifestErr, "could not find BACKUP manifest file in any of the known locations: %s, %s, %s",
backupbase.BackupMetadataName, backupbase.BackupManifestName, backupbase.BackupOldManifestName)
}
return backuppb.BackupManifest{}, 0, oldBackupManifestErr
} else {
// We found a `BACKUP` manifest file.
Expand Down
15 changes: 15 additions & 0 deletions pkg/ccl/schemachangerccl/backup_base_generated_test.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 11 additions & 11 deletions pkg/ccl/schemachangerccl/testdata/explain/drop_database_multiregion_primary_region
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,21 +65,21 @@ Schema change plan for DROP DATABASE ‹multi_region_test_db› CASCADE;
│ │ ├── PUBLIC → VALIDATED PrimaryIndex:{DescID: 108, IndexID: 1, ConstraintID: 1}
│ │ └── PUBLIC → ABSENT IndexName:{DescID: 108, Name: table_regional_by_table_pkey, IndexID: 1}
│ └── 15 Mutation operations
│ ├── MarkDescriptorAsDropped {"DescriptorID":104}
│ ├── MarkDescriptorAsDropped {"DescriptorID":105}
│ ├── RemoveSchemaParent {"Parent":{"ParentDatabaseID":104,"SchemaID":105}}
│ ├── MarkDescriptorAsDropped {"DescriptorID":106}
│ ├── RemoveObjectParent {"ObjectID":106,"ParentSchemaID":105}
│ ├── MarkDescriptorAsDropped {"DescriptorID":107}
│ ├── RemoveObjectParent {"ObjectID":107,"ParentSchemaID":105}
│ ├── MarkDescriptorAsDropped {"DescriptorID":108}
│ ├── RemoveObjectParent {"ObjectID":108,"ParentSchemaID":105}
│ ├── RemoveBackReferenceInTypes {"BackReferencedDescriptorID":108}
│ ├── DrainDescriptorName {"Namespace":{"DescriptorID":104,"Name":"multi_region_tes..."}}
│ ├── MarkDescriptorAsDropped {"DescriptorID":105}
│ ├── RemoveSchemaParent {"Parent":{"ParentDatabaseID":104,"SchemaID":105}}
│ ├── MarkDescriptorAsDropped {"DescriptorID":104}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":105,"Name":"public"}}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":106,"Name":"crdb_internal_re...","SchemaID":105}}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":107,"Name":"_crdb_internal_r...","SchemaID":105}}
│ └── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":108,"Name":"table_regional_b...","SchemaID":105}}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":108,"Name":"table_regional_b...","SchemaID":105}}
│ └── DrainDescriptorName {"Namespace":{"DescriptorID":104,"Name":"multi_region_tes..."}}
├── PreCommitPhase
│ ├── Stage 1 of 2 in PreCommitPhase
│ │ ├── 53 elements transitioning toward ABSENT
Expand Down Expand Up @@ -194,25 +194,25 @@ Schema change plan for DROP DATABASE ‹multi_region_test_db› CASCADE;
│ │ ├── PUBLIC → ABSENT PrimaryIndex:{DescID: 108, IndexID: 1, ConstraintID: 1}
│ │ └── PUBLIC → ABSENT IndexName:{DescID: 108, Name: table_regional_by_table_pkey, IndexID: 1}
│ └── 26 Mutation operations
│ ├── MarkDescriptorAsDropped {"DescriptorID":104}
│ ├── RemoveDatabaseRoleSettings {"DatabaseID":104}
│ ├── MarkDescriptorAsDropped {"DescriptorID":105}
│ ├── RemoveSchemaParent {"Parent":{"ParentDatabaseID":104,"SchemaID":105}}
│ ├── MarkDescriptorAsDropped {"DescriptorID":106}
│ ├── RemoveObjectParent {"ObjectID":106,"ParentSchemaID":105}
│ ├── MarkDescriptorAsDropped {"DescriptorID":107}
│ ├── RemoveObjectParent {"ObjectID":107,"ParentSchemaID":105}
│ ├── MarkDescriptorAsDropped {"DescriptorID":108}
│ ├── RemoveObjectParent {"ObjectID":108,"ParentSchemaID":105}
│ ├── RemoveBackReferenceInTypes {"BackReferencedDescriptorID":108}
│ ├── DrainDescriptorName {"Namespace":{"DescriptorID":104,"Name":"multi_region_tes..."}}
│ ├── MarkDescriptorAsDropped {"DescriptorID":105}
│ ├── RemoveSchemaParent {"Parent":{"ParentDatabaseID":104,"SchemaID":105}}
│ ├── RemoveColumnNotNull {"ColumnID":1,"TableID":108}
│ ├── MarkDescriptorAsDropped {"DescriptorID":104}
│ ├── RemoveDatabaseRoleSettings {"DatabaseID":104}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":105,"Name":"public"}}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":106,"Name":"crdb_internal_re...","SchemaID":105}}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":107,"Name":"_crdb_internal_r...","SchemaID":105}}
│ ├── DrainDescriptorName {"Namespace":{"DatabaseID":104,"DescriptorID":108,"Name":"table_regional_b...","SchemaID":105}}
│ ├── RemoveColumnNotNull {"ColumnID":1,"TableID":108}
│ ├── MakeDeleteOnlyColumnAbsent {"ColumnID":4294967295,"TableID":108}
│ ├── MakeDeleteOnlyColumnAbsent {"ColumnID":4294967294,"TableID":108}
│ ├── DrainDescriptorName {"Namespace":{"DescriptorID":104,"Name":"multi_region_tes..."}}
│ ├── MakeDeleteOnlyColumnAbsent {"ColumnID":1,"TableID":108}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104,"Initialize":true}
│ ├── SetJobStateOnDescriptor {"DescriptorID":105,"Initialize":true}
Expand Down
Loading

0 comments on commit 08a188a

Please sign in to comment.