Skip to content

Commit

Permalink
sql/schemachanger: dropped index and views should be cleaned together
Browse files Browse the repository at this point in the history 
Previously, during dropped indexes and views that
depended on them did not have any proper dependency
order enforced. This could lead to improper event
log entries / wrong transactional behaviour. This
patch, ensures that views are cleaned once the index
becomes invisible.

Release note: None
  • Loading branch information
@fqazi
fqazi committed Dec 15, 2022
1 parent c276451 commit 482d84a
Show file tree
Hide file tree
Showing 163 changed files with 2,971 additions and 3,177 deletions.
3 changes: 1 addition & 2 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_1_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,8 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
└── 7 Mutation operations
└── 6 Mutation operations
├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
├── LogEvent {"TargetStatus":1}
├── MakeIndexAbsent {"IndexID":2,"TableID":104}
├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
Expand Down
3 changes: 1 addition & 2 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_2_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,9 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ └── 6 Mutation operations
│ └── 5 Mutation operations
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── MakeIndexAbsent {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
Expand Down
3 changes: 1 addition & 2 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_3_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,9 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ └── 6 Mutation operations
│ └── 5 Mutation operations
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── MakeIndexAbsent {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
Expand Down
3 changes: 1 addition & 2 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_4_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,9 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ └── 6 Mutation operations
│ └── 5 Mutation operations
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── MakeIndexAbsent {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
Expand Down
8 changes: 4 additions & 4 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_5_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ └── 5 Mutation operations
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ └── 6 Mutation operations
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":2,"TableID":104}
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
└── Stage 2 of 2 in PostCommitNonRevertiblePhase
Expand All @@ -35,8 +36,7 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ ├── PUBLIC → ABSENT IndexData:{DescID: 104, IndexID: 2}
│ ├── DELETE_ONLY → ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ └── PUBLIC → ABSENT IndexData:{DescID: 104, IndexID: 3}
└── 7 Mutation operations
├── LogEvent {"TargetStatus":1}
└── 6 Mutation operations
├── MakeIndexAbsent {"IndexID":2,"TableID":104}
├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
Expand Down
8 changes: 4 additions & 4 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_6_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ └── 5 Mutation operations
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ └── 6 Mutation operations
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":2,"TableID":104}
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
└── Stage 2 of 2 in PostCommitNonRevertiblePhase
Expand All @@ -35,8 +36,7 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ ├── PUBLIC → ABSENT IndexData:{DescID: 104, IndexID: 2}
│ ├── DELETE_ONLY → ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ └── PUBLIC → ABSENT IndexData:{DescID: 104, IndexID: 3}
└── 7 Mutation operations
├── LogEvent {"TargetStatus":1}
└── 6 Mutation operations
├── MakeIndexAbsent {"IndexID":2,"TableID":104}
├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
Expand Down
6 changes: 3 additions & 3 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index.rollback_7_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 1, IndexID: 3}
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 2, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ └── 5 Mutation operations
│ └── 6 Mutation operations
│ ├── LogEvent {"TargetStatus":1}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":2,"TableID":104}
│ ├── SetIndexName {"IndexID":2,"Name":"crdb_internal_in...","TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
Expand All @@ -35,8 +36,7 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ ├── PUBLIC → ABSENT IndexData:{DescID: 104, IndexID: 2}
│ ├── DELETE_ONLY → ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ └── PUBLIC → ABSENT IndexData:{DescID: 104, IndexID: 3}
└── 7 Mutation operations
├── LogEvent {"TargetStatus":1}
└── 6 Mutation operations
├── MakeIndexAbsent {"IndexID":2,"TableID":104}
├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
Expand Down
25 changes: 5 additions & 20 deletions pkg/ccl/schemachangerccl/testdata/explain_verbose/create_index.rollback_1_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,10 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 1 of 7;
│ │ rule: "index removed before garbage collection"
│ │
│ ├── • IndexName:{DescID: 104, Name: id1, IndexID: 2}
│ │ PUBLIC → ABSENT
│ │ │ PUBLIC → ABSENT
│ │ │
│ │ └── • Precedence dependency from DELETE_ONLY SecondaryIndex:{DescID: 104, IndexID: 2, ConstraintID: 0, TemporaryIndexID: 3, SourceIndexID: 1}
│ │ rule: "index no longer public before index name"
│ │
│ ├── • TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ │ │ DELETE_ONLY → ABSENT
Expand Down Expand Up @@ -106,31 +109,13 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 1 of 7;
│ └── • skip PUBLIC → ABSENT operations
│ rule: "skip index-column removal ops on index removal"
│
└── • 7 Mutation operations
└── • 6 Mutation operations
│
├── • SetIndexName
│ IndexID: 2
│ Name: crdb_internal_index_2_name_placeholder
│ TableID: 104
│
├── • LogEvent
│ Element:
│ SecondaryIndex:
│ indexId: 2
│ sourceIndexId: 1
│ tableId: 104
│ temporaryIndexId: 3
│ EventBase:
│ Authorization:
│ UserName: root
│ Statement: CREATE INDEX ‹id1› ON ‹defaultdb›.public.‹t1› (‹id›, ‹name›) STORING
│ (‹money›) PARTITION BY LIST (‹id›) (PARTITION ‹p1› VALUES IN (‹1›))
│ StatementTag: CREATE INDEX
│ TargetMetadata:
│ SourceElementID: 1
│ SubWorkID: 1
│ TargetStatus: 1
│
├── • MakeIndexAbsent
│ IndexID: 2
│ TableID: 104
Expand Down
25 changes: 5 additions & 20 deletions pkg/ccl/schemachangerccl/testdata/explain_verbose/create_index.rollback_2_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,10 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 2 of 7;
│ │ │ rule: "skip index-column removal ops on index removal"
│ │ │
│ │ ├── • IndexName:{DescID: 104, Name: id1, IndexID: 2}
│ │ │ PUBLIC → ABSENT
│ │ │ │ PUBLIC → ABSENT
│ │ │ │
│ │ │ └── • Precedence dependency from DELETE_ONLY SecondaryIndex:{DescID: 104, IndexID: 2, ConstraintID: 0, TemporaryIndexID: 3, SourceIndexID: 1}
│ │ │ rule: "index no longer public before index name"
│ │ │
│ │ ├── • TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ │ │ │ WRITE_ONLY → DELETE_ONLY
Expand Down Expand Up @@ -88,7 +91,7 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 2 of 7;
│ │ └── • skip PUBLIC → ABSENT operations
│ │ rule: "skip index-column removal ops on index removal"
│ │
│ └── • 6 Mutation operations
│ └── • 5 Mutation operations
│ │
│ ├── • SetIndexName
│ │ IndexID: 2
Expand All @@ -99,24 +102,6 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 2 of 7;
│ │ IndexID: 3
│ │ TableID: 104
│ │
│ ├── • LogEvent
│ │ Element:
│ │ SecondaryIndex:
│ │ indexId: 2
│ │ sourceIndexId: 1
│ │ tableId: 104
│ │ temporaryIndexId: 3
│ │ EventBase:
│ │ Authorization:
│ │ UserName: root
│ │ Statement: CREATE INDEX ‹id1› ON ‹defaultdb›.public.‹t1› (‹id›, ‹name›) STORING
│ │ (‹money›) PARTITION BY LIST (‹id›) (PARTITION ‹p1› VALUES IN (‹1›))
│ │ StatementTag: CREATE INDEX
│ │ TargetMetadata:
│ │ SourceElementID: 1
│ │ SubWorkID: 1
│ │ TargetStatus: 1
│ │
│ ├── • MakeIndexAbsent
│ │ IndexID: 2
│ │ TableID: 104
Expand Down
25 changes: 5 additions & 20 deletions pkg/ccl/schemachangerccl/testdata/explain_verbose/create_index.rollback_3_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,10 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 3 of 7;
│ │ │ rule: "skip index-column removal ops on index removal"
│ │ │
│ │ ├── • IndexName:{DescID: 104, Name: id1, IndexID: 2}
│ │ │ PUBLIC → ABSENT
│ │ │ │ PUBLIC → ABSENT
│ │ │ │
│ │ │ └── • Precedence dependency from DELETE_ONLY SecondaryIndex:{DescID: 104, IndexID: 2, ConstraintID: 0, TemporaryIndexID: 3, SourceIndexID: 1}
│ │ │ rule: "index no longer public before index name"
│ │ │
│ │ ├── • TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ │ │ │ WRITE_ONLY → DELETE_ONLY
Expand Down Expand Up @@ -88,7 +91,7 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 3 of 7;
│ │ └── • skip PUBLIC → ABSENT operations
│ │ rule: "skip index-column removal ops on index removal"
│ │
│ └── • 6 Mutation operations
│ └── • 5 Mutation operations
│ │
│ ├── • SetIndexName
│ │ IndexID: 2
Expand All @@ -99,24 +102,6 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 3 of 7;
│ │ IndexID: 3
│ │ TableID: 104
│ │
│ ├── • LogEvent
│ │ Element:
│ │ SecondaryIndex:
│ │ indexId: 2
│ │ sourceIndexId: 1
│ │ tableId: 104
│ │ temporaryIndexId: 3
│ │ EventBase:
│ │ Authorization:
│ │ UserName: root
│ │ Statement: CREATE INDEX ‹id1› ON ‹defaultdb›.public.‹t1› (‹id›, ‹name›) STORING
│ │ (‹money›) PARTITION BY LIST (‹id›) (PARTITION ‹p1› VALUES IN (‹1›))
│ │ StatementTag: CREATE INDEX
│ │ TargetMetadata:
│ │ SourceElementID: 1
│ │ SubWorkID: 1
│ │ TargetStatus: 1
│ │
│ ├── • MakeIndexAbsent
│ │ IndexID: 2
│ │ TableID: 104
Expand Down
25 changes: 5 additions & 20 deletions pkg/ccl/schemachangerccl/testdata/explain_verbose/create_index.rollback_4_of_7
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,10 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 4 of 7;
│ │ │ rule: "skip index-column removal ops on index removal"
│ │ │
│ │ ├── • IndexName:{DescID: 104, Name: id1, IndexID: 2}
│ │ │ PUBLIC → ABSENT
│ │ │ │ PUBLIC → ABSENT
│ │ │ │
│ │ │ └── • Precedence dependency from DELETE_ONLY SecondaryIndex:{DescID: 104, IndexID: 2, ConstraintID: 0, TemporaryIndexID: 3, SourceIndexID: 1}
│ │ │ rule: "index no longer public before index name"
│ │ │
│ │ ├── • TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 1, SourceIndexID: 1}
│ │ │ │ WRITE_ONLY → DELETE_ONLY
Expand Down Expand Up @@ -88,7 +91,7 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 4 of 7;
│ │ └── • skip PUBLIC → ABSENT operations
│ │ rule: "skip index-column removal ops on index removal"
│ │
│ └── • 6 Mutation operations
│ └── • 5 Mutation operations
│ │
│ ├── • SetIndexName
│ │ IndexID: 2
Expand All @@ -99,24 +102,6 @@ EXPLAIN (ddl, verbose) rollback at post-commit stage 4 of 7;
│ │ IndexID: 3
│ │ TableID: 104
│ │
│ ├── • LogEvent
│ │ Element:
│ │ SecondaryIndex:
│ │ indexId: 2
│ │ sourceIndexId: 1
│ │ tableId: 104
│ │ temporaryIndexId: 3
│ │ EventBase:
│ │ Authorization:
│ │ UserName: root
│ │ Statement: CREATE INDEX ‹id1› ON ‹defaultdb›.public.‹t1› (‹id›, ‹name›) STORING
│ │ (‹money›) PARTITION BY LIST (‹id›) (PARTITION ‹p1› VALUES IN (‹1›))
│ │ StatementTag: CREATE INDEX
│ │ TargetMetadata:
│ │ SourceElementID: 1
│ │ SubWorkID: 1
│ │ TargetStatus: 1
│ │
│ ├── • MakeIndexAbsent
│ │ IndexID: 2
│ │ TableID: 104
Expand Down
Loading

0 comments on commit 482d84a

Please sign in to comment.