-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
alter table of parent geo-partitioned table affects select on child #9547
Comments
The issue is because altering the partitioned table results in adding a column to the partitioned table, and then recursively adding a column for all the child tables. This is done correctly at the PG metadata side.
The fix is to add a column at the YB side for all the partitions as well in the above code. |
When can we expect this to be released? |
…ect on child Summary: Failing example: ``` -- Add column to parent partitioned table. yugabyte@yugabyte=# alter table transactions add column foo varchar(30); ALTER TABLE -- Column is added to child partition in PG metadata. yugabyte@yugabyte=# \d transactions_eu; Table "public.transactions_eu" Column | Type | Collation | Nullable | Default ---------------+-----------------------------+-----------+----------+--------- user_id | integer | | not null | geo_partition | character varying | | not null | foo | character varying(30) | | | Partition of: transactions FOR VALUES IN ('EU', 'EMEA') -- Select data from child localhost:5433 yugabyte@yugabyte=# select * from transactions_eu; ERROR: Invalid argument: Invalid column number 8 ``` The above issue is happening because ALTER on parent partitioned table recursively reflects the ALTER operation on the child partitions by the PG layer. However, this does not happen on the DocDB side. This leads to mismatch in the YSQL and DocDB metadata for the child partition tables following an ALTER, resulting in failure of future DQL and DML operations. Test Plan: ybd --scb --sj --java-test org.yb.pgsql.TestPgRegressPartitions Reviewers: myang Reviewed By: myang Subscribers: ena, myang, yql Differential Revision: https://phabricator.dev.yugabyte.com/D14685
…ects select on child Summary: Failing example: ``` -- Add column to parent partitioned table. yugabyte@yugabyte=# alter table transactions add column foo varchar(30); ALTER TABLE -- Column is added to child partition in PG metadata. yugabyte@yugabyte=# \d transactions_eu; Table "public.transactions_eu" Column | Type | Collation | Nullable | Default ---------------+-----------------------------+-----------+----------+--------- user_id | integer | | not null | geo_partition | character varying | | not null | foo | character varying(30) | | | Partition of: transactions FOR VALUES IN ('EU', 'EMEA') -- Select data from child localhost:5433 yugabyte@yugabyte=# select * from transactions_eu; ERROR: Invalid argument: Invalid column number 8 ``` The above issue is happening because ALTER on parent partitioned table recursively reflects the ALTER operation on the child partitions by the PG layer. However, this does not happen on the DocDB side. This leads to mismatch in the YSQL and DocDB metadata for the child partition tables following an ALTER, resulting in failure of future DQL and DML operations. Test Plan: ybd --scb --sj --java-test org.yb.pgsql.TestPgRegressPartitions Reviewers: myang Reviewed By: myang Subscribers: ena, myang, yql Differential Revision: https://phabricator.dev.yugabyte.com/D14685
…le affects select on child Summary: Failing example: ``` -- Add column to parent partitioned table. yugabyte@yugabyte=# alter table transactions add column foo varchar(30); ALTER TABLE -- Column is added to child partition in PG metadata. yugabyte@yugabyte=# \d transactions_eu; Table "public.transactions_eu" Column | Type | Collation | Nullable | Default ---------------+-----------------------------+-----------+----------+--------- user_id | integer | | not null | geo_partition | character varying | | not null | foo | character varying(30) | | | Partition of: transactions FOR VALUES IN ('EU', 'EMEA') -- Select data from child localhost:5433 yugabyte@yugabyte=# select * from transactions_eu; ERROR: Invalid argument: Invalid column number 8 ``` The above issue is happening because ALTER on parent partitioned table recursively reflects the ALTER operation on the child partitions by the PG layer. However, this does not happen on the DocDB side. This leads to mismatch in the YSQL and DocDB metadata for the child partition tables following an ALTER, resulting in failure of future DQL and DML operations. Original Diff: https://phabricator.dev.yugabyte.com/D14685 Original Commit: ff176a5 Conflict resolution: Since the number of tests has become too high to fit into an acceptable timeout, the tests were split into multiple tests before the original commit was pushed into master. However commit ccabdf7 that introduces this split cannot be back ported to 2.6 as it requires other commits before it. Adding the tests introduced by this commit further increases the possibility of the test failing due to timeout. Hence in this commit, introduce a new file TestYbPgPartitions and only run the tests introduced by YB into its schedule. This simple split does not require any previous commits and will also not run into timeout issues. Starting 2.8 release onwards, the split tests are present, so there will not be any divergence from 2.8 onwards. Test Plan: Jenkins: rebase: 2.6 ybd --scb --sj --java-test org.yb.pgsql.TestPgRegressPartitions ybd --scb --sj --java-test org.yb.pgsql.TestYbPgPartitions Reviewers: myang Reviewed By: myang Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D15823
…able affects select on child Summary: Failing example: ``` -- Add column to parent partitioned table. yugabyte@yugabyte=# alter table transactions add column foo varchar(30); ALTER TABLE -- Column is added to child partition in PG metadata. yugabyte@yugabyte=# \d transactions_eu; Table "public.transactions_eu" Column | Type | Collation | Nullable | Default ---------------+-----------------------------+-----------+----------+--------- user_id | integer | | not null | geo_partition | character varying | | not null | foo | character varying(30) | | | Partition of: transactions FOR VALUES IN ('EU', 'EMEA') -- Select data from child localhost:5433 yugabyte@yugabyte=# select * from transactions_eu; ERROR: Invalid argument: Invalid column number 8 ``` The above issue is happening because ALTER on parent partitioned table recursively reflects the ALTER operation on the child partitions by the PG layer. However, this does not happen on the DocDB side. This leads to mismatch in the YSQL and DocDB metadata for the child partition tables following an ALTER, resulting in failure of future DQL and DML operations. Original Diff: https://phabricator.dev.yugabyte.com/D14685 Original Commit: ff176a5 Test Plan: Jenkins: rebase: 2.12 ybd --scb --sj --java-test org.yb.pgsql.TestPgRegressPartitions Reviewers: myang Reviewed By: myang Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D15820
…le affects select on child Summary: Failing example: ``` -- Add column to parent partitioned table. yugabyte@yugabyte=# alter table transactions add column foo varchar(30); ALTER TABLE -- Column is added to child partition in PG metadata. yugabyte@yugabyte=# \d transactions_eu; Table "public.transactions_eu" Column | Type | Collation | Nullable | Default ---------------+-----------------------------+-----------+----------+--------- user_id | integer | | not null | geo_partition | character varying | | not null | foo | character varying(30) | | | Partition of: transactions FOR VALUES IN ('EU', 'EMEA') -- Select data from child localhost:5433 yugabyte@yugabyte=# select * from transactions_eu; ERROR: Invalid argument: Invalid column number 8 ``` The above issue is happening because ALTER on parent partitioned table recursively reflects the ALTER operation on the child partitions by the PG layer. However, this does not happen on the DocDB side. This leads to mismatch in the YSQL and DocDB metadata for the child partition tables following an ALTER, resulting in failure of future DQL and DML operations. Original Diff: https://phabricator.dev.yugabyte.com/D14685 Original Commit: ff176a5 Test Plan: ybd --scb --sj --java-test org.yb.pgsql.TestPgRegressPartitions Reviewers: myang Reviewed By: myang Subscribers: dfelsing, yql Differential Revision: https://phabricator.dev.yugabyte.com/D16021
ALTER TABLE on the parent geo-partitioned table affects SELECT on the child table.
The following example uses the schema from the documentation.
The text was updated successfully, but these errors were encountered: