-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
opt: unnecessary columns are added to the input of update operations #57097
Comments
This commit fixes several bugs that are present when using foreign key cascading updates and partial indexes. Previously, the optimizer was not synthesizing partial index DEL columns for FK cascading updates. As a result, a partial index on a child table could become inconsistent with the rows in the primary index, ultimately resulting in incorrect query results. The optbuilder has been refactored to project these columns and to reduce the complexity of doing so. As a result, partial index PUT and DEL columns are now projected in the same expression, rather than the DEL columns being projected as far down in the expression tree as possible. Additionally, the execution engine was unable to handle extraneous columns that can be added as input to FK cascading updates. These extraneous columns would be incorrectly interpreted as synthesized partial index columns. This commit works around this issue by passing a set of partial indexes that columns have been synthesized for from the optimizer to the execution engine. The longer term fix is to not produce these columns (see issue cockroachdb#57097). Fixes cockroachdb#57085 Fixes cockroachdb#57084 Release justification: This is a critical bug fix to a new feature, partial indexes. Release note (bug fix): Previously, updating parent table of a foreign key relationship with cascading updates could cause errors or inconsistencies in partial indexes for child tables with partial indexes. The inconsistency of partial indexes could result in incorrect query results. This has been fixed.
Below is the output from build-cascades. I think it's just the result of doing the join and we didn't bother projecting away what we don't need..
|
Previously, the optimizer was not synthesizing partial index DEL columns for FK cascading updates and deletes. As a result, a cascading `UPDATE` could corrupt a child table's partial index, ultimately resulting in incorrect query results. A cascading `DELETE` would not corrupt partial indexes, but unnecessary `DEL` operations would be issued on the partial index. The optbuilder has been refactored so that these columns are correctly projected. Both PUT and DEL columns are now projected in the same function, `mutationBuilder.projectPartialIndexCols`. This function is called from principal functions in the optbuilder where CHECK constraint columns are also projected, like `mutationBuilder.buildUpdate`. In theory this should make it harder in the future to omit these necessary projections. Additionally, the execution engine was unable to handle extraneous columns that can be added as input to FK cascading updates. These extraneous columns would be incorrectly interpreted as synthesized partial index columns. This commit works around this issue by slicing the source values with an upper bound in `upateNode.processSourceRow`. The longer term fix is to not produce these columns (see issue cockroachdb#57097). Fixes cockroachdb#57085 Fixes cockroachdb#57084 Release justification: This is a critical bug fix to a new feature, partial indexes. Release note (bug fix): A bug has been fixed that caused errors or corrupted partial indexes of child tables in foreign key relationships with cascading `UPDATE`s and `DELETE`s. The corrupt partial indexes could result in incorrect query results. Any partial indexes on child tables of foreign key relationships with `ON DELETE CASCADE` or `ON UPDATE CASCADE` actions may be corrupt and should be dropped and re-created. This bug was introduce in version 20.2.
Previously, the optimizer was not synthesizing partial index DEL columns for FK cascading updates and deletes. As a result, a cascading `UPDATE` could corrupt a child table's partial index, ultimately resulting in incorrect query results. A cascading `DELETE` would not corrupt partial indexes, but unnecessary `DEL` operations would be issued on the partial index. The optbuilder has been refactored so that these columns are correctly projected. Both PUT and DEL columns are now projected in the same function, `mutationBuilder.projectPartialIndexCols`. This function is called from principal functions in the optbuilder where CHECK constraint columns are also projected, like `mutationBuilder.buildUpdate`. In theory this should make it harder in the future to omit these necessary projections. Additionally, the execution engine was unable to handle extraneous columns that can be added as input to FK cascading updates. These extraneous columns would be incorrectly interpreted as synthesized partial index columns. This commit works around this issue by slicing the source values with an upper bound in `upateNode.processSourceRow`. The longer term fix is to not produce these columns (see issue cockroachdb#57097). Fixes cockroachdb#57085 Fixes cockroachdb#57084 Release justification: This is a critical bug fix to a new feature, partial indexes. Release note (bug fix): A bug has been fixed that caused errors or corrupted partial indexes of child tables in foreign key relationships with cascading `UPDATE`s and `DELETE`s. The corrupt partial indexes could result in incorrect query results. Any partial indexes on child tables of foreign key relationships with `ON DELETE CASCADE` or `ON UPDATE CASCADE` actions may be corrupt and should be dropped and re-created. This bug was introduce in version 20.2.
Previously, the optimizer was not synthesizing partial index DEL columns for FK cascading updates and deletes. As a result, a cascading `UPDATE` could corrupt a child table's partial index, ultimately resulting in incorrect query results. A cascading `DELETE` would not corrupt partial indexes, but unnecessary `DEL` operations would be issued on the partial index. The optbuilder has been refactored so that these columns are correctly projected. There are now three functions for projecting PUT columns, DEL columns, and both PUT and DEL columns, each ensuring that the input scopes are non-nil. These three functions are called from principal functions in the optbuilder where CHECK constraint columns are also projected, like `mutationBuilder.buildUpdate`. In theory this should make it harder in the future to omit these necessary projections. Additionally, the execution engine was unable to handle extraneous columns that can be added as input to FK cascading updates. These extraneous columns would be incorrectly interpreted as synthesized partial index columns. This commit works around this issue by slicing the source values with an upper bound in `updateNode.processSourceRow`. The longer term fix is to not produce these columns (see issue cockroachdb#57097). Fixes cockroachdb#57085 Fixes cockroachdb#57084 Release justification: This is a critical bug fix to a new feature, partial indexes. Release note (bug fix): A bug has been fixed that caused errors or corrupted partial indexes of child tables in foreign key relationships with cascading `UPDATE`s and `DELETE`s. The corrupt partial indexes could result in incorrect query results. Any partial indexes on child tables of foreign key relationships with `ON DELETE CASCADE` or `ON UPDATE CASCADE` actions may be corrupt and should be dropped and re-created. This bug was introduce in version 20.2.
Previously, the optimizer was not synthesizing partial index DEL columns for FK cascading updates and deletes. As a result, a cascading `UPDATE` could corrupt a child table's partial index, ultimately resulting in incorrect query results. A cascading `DELETE` would not corrupt partial indexes, but unnecessary `DEL` operations would be issued on the partial index. The optbuilder has been refactored so that these columns are correctly projected. There are now three functions for projecting PUT columns, DEL columns, and both PUT and DEL columns, each ensuring that the input scopes are non-nil. These three functions are called from principal functions in the optbuilder where CHECK constraint columns are also projected, like `mutationBuilder.buildUpdate`. In theory this should make it harder in the future to omit these necessary projections. Additionally, the execution engine was unable to handle extraneous columns that can be added as input to FK cascading updates. These extraneous columns would be incorrectly interpreted as synthesized partial index columns. This commit works around this issue by slicing the source values with an upper bound in `updateNode.processSourceRow`. The longer term fix is to not produce these columns (see issue cockroachdb#57097). Fixes cockroachdb#57085 Fixes cockroachdb#57084 Release justification: This is a critical bug fix to a new feature, partial indexes. Release note (bug fix): A bug has been fixed that caused errors or corrupted partial indexes of child tables in foreign key relationships with cascading `UPDATE`s and `DELETE`s. The corrupt partial indexes could result in incorrect query results. Any partial indexes on child tables of foreign key relationships with `ON DELETE CASCADE` or `ON UPDATE CASCADE` actions may be corrupt and should be dropped and re-created. This bug was introduce in version 20.2.
57100: opt: fix FK cascades to child tables with partial indexes r=mgartner a=mgartner Previously, the optimizer was not synthesizing partial index DEL columns for FK cascading updates and deletes. As a result, a cascading `UPDATE` could corrupt a child table's partial index, ultimately resulting in incorrect query results. A cascading `DELETE` would not corrupt partial indexes, but unnecessary `DEL` operations would be issued on the partial index. The optbuilder has been refactored so that these columns are correctly projected. There are now three functions for projecting PUT columns, DEL columns, and both PUT and DEL columns, each ensuring that the input scopes are non-nil. These three functions are called from principal functions in the optbuilder where CHECK constraint columns are also projected, like `mutationBuilder.buildUpdate`. In theory this should make it harder in the future to omit these necessary projections. Additionally, the execution engine was unable to handle extraneous columns that can be added as input to FK cascading updates. These extraneous columns would be incorrectly interpreted as synthesized partial index columns. This commit works around this issue by slicing the source values with an upper bound in `updateNode.processSourceRow`. The longer term fix is to not produce these columns (see issue #57097). Fixes #57085 Fixes #57084 Release justification: This is a critical bug fix to a new feature, partial indexes. Release note (bug fix): A bug has been fixed that caused errors or corrupted partial indexes of child tables in foreign key relationships with cascading `UPDATE`s and `DELETE`s. The corrupt partial indexes could result in incorrect query results. Any partial indexes on child tables of foreign key relationships with `ON DELETE CASCADE` or `ON UPDATE CASCADE` actions may be corrupt and should be dropped and re-created. This bug was introduce in version 20.2. 57323: authors: add Paul Kernfeld to authors r=kernfeld-cockroach a=kernfeld-cockroach Release note: None 57324: authors: add ricky to authors r=rickystewart a=rickystewart Release note: None Co-authored-by: Marcus Gartner <[email protected]> Co-authored-by: Paul Kernfeld <[email protected]> Co-authored-by: Ricky Stewart <[email protected]>
We have marked this issue as stale because it has been inactive for |
In some situations, exec builder is adding unnecessary columns to mutation inputs here.
To reproduce the issue (I've used print statements here for those who don't have a debugger setup):
updateNode.processSource
function inpkg/sql/update.go
:TESTFLAGS='-v'
:These are printed when the cascading FK update rows are processed. There are 3 fetch cols and 1 update col, but
sourceVals
includes an unnecessary fifth value. AFAICT this value (1
in this case) is never used.Jira issue: CRDB-2865
The text was updated successfully, but these errors were encountered: