Support Table and Index partitioning in Babelfish #399
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sumit Jaiswal <[email protected]>
Signed-off-by: Sumit Jaiswal <[email protected]>
…umiji-partitioning-support
Deepesh125
requested changes
Jul 8, 2024
…umiji-partitioning-support
…umiji-partitioning-support
…umiji-partitioning-support
Signed-off-by: Sumit Jaiswal <[email protected]>
rishabhtanwar29
approved these changes
Jul 12, 2024
…umiji-partitioning-support
Deepesh125
requested changes
Jul 15, 2024
Signed-off-by: Sumit Jaiswal <[email protected]>
Deepesh125
approved these changes
Jul 22, 2024
…umiji-partitioning-support
…umiji-partitioning-support
KushaalShroff
approved these changes
Jul 25, 2024
KushaalShroff
merged commit b670a08
into
babelfish-for-postgresql:BABEL_4_X_DEV__PG_16_X
2 checks passed
KushaalShroff
pushed a commit
to babelfish-for-postgresql/babelfish_extensions
that referenced
this pull request
Jul 25, 2024
This commit introduces Support for Table and Index Partitioning in Babelfish, enabling users to partition large tables and indexes using partition schemes. Key Changes: 1. Partitioned Table Support: - Added syntax support for creating partitioned tables using the `CREATE TABLE ... on PARTITION_SCHEME (column)` statement. - Implemented support for partitioned table using partition scheme, where child partitions are implicitly created by looking up the metadata of the partition function and partitioning column values. - Restricted InsertBulk operations for partitioned tables as they are not supported yet. - Restricted users from modifying Babelfish partitioned tables to attach, detach, or modify partitions of these tables to maintain data consistency in the Babelfish catalog. Existing users who have created partitioned tables from the PostgreSQL endpoint can continue to modify, attach, and detach partitions as usual. - Introduced sys.babelfish_partition_depend catalog to track the dependency between partitioned tables and the partition schemes used to create them. The catalog maintains consistent table name even after RENAME or DROP TABLE operations. 2. Partitioned Index Support: - Added syntax support for creating partitioned indexes using the `CREATE INDEX ... ON PARTITION_SCHEME (column)` statement. - Implemented support for creating aligned partitioned indexes, where the index partitioning scheme aligns with the underlying table partitioning scheme. - Restricted the ability to create non-aligned partitioned indexes with custom partition schemes. 3. System Catalog Views: - Modified system catalog views to exclude listing child partitions for partitioned tables and indexes. - Modified system catalog views to list the partitioned table along with regular tables. - These changes ensures that only the parent partitioned object is listed in the relevant system views Signed-off-by: Sumit Jaiswal [[email protected]](mailto:[email protected]) Engine PR: babelfish-for-postgresql/postgresql_modified_for_babelfish#399 Issues Resolved Task: BABEL-4968, BABEL-4969
sharathbp
pushed a commit
to amazon-aurora/babelfish_extensions
that referenced
this pull request
Aug 20, 2024
…gresql#2702) This commit introduces Support for Table and Index Partitioning in Babelfish, enabling users to partition large tables and indexes using partition schemes. Key Changes: 1. Partitioned Table Support: - Added syntax support for creating partitioned tables using the `CREATE TABLE ... on PARTITION_SCHEME (column)` statement. - Implemented support for partitioned table using partition scheme, where child partitions are implicitly created by looking up the metadata of the partition function and partitioning column values. - Restricted InsertBulk operations for partitioned tables as they are not supported yet. - Restricted users from modifying Babelfish partitioned tables to attach, detach, or modify partitions of these tables to maintain data consistency in the Babelfish catalog. Existing users who have created partitioned tables from the PostgreSQL endpoint can continue to modify, attach, and detach partitions as usual. - Introduced sys.babelfish_partition_depend catalog to track the dependency between partitioned tables and the partition schemes used to create them. The catalog maintains consistent table name even after RENAME or DROP TABLE operations. 2. Partitioned Index Support: - Added syntax support for creating partitioned indexes using the `CREATE INDEX ... ON PARTITION_SCHEME (column)` statement. - Implemented support for creating aligned partitioned indexes, where the index partitioning scheme aligns with the underlying table partitioning scheme. - Restricted the ability to create non-aligned partitioned indexes with custom partition schemes. 3. System Catalog Views: - Modified system catalog views to exclude listing child partitions for partitioned tables and indexes. - Modified system catalog views to list the partitioned table along with regular tables. - These changes ensures that only the parent partitioned object is listed in the relevant system views Signed-off-by: Sumit Jaiswal [[email protected]](mailto:[email protected]) Engine PR: babelfish-for-postgresql/postgresql_modified_for_babelfish#399 Issues Resolved Task: BABEL-4968, BABEL-4969
roshan0708
pushed a commit
to amazon-aurora/postgresql_modified_for_babelfish
that referenced
this pull request
Oct 15, 2024
…or-postgresql#399) To support partitioned tables and indexes in Babelfish, this commit introduces the following changes: Babelfish stores the creation date and original name of the table in the reloption, whereas PostgreSQL does not allow the usage of reloption for partitioned tables. This commit enables the usage of reloption for Babelfish partitioned tables to store the creation date and original name. This information will be utilized by various system views, such as information_schema_tsql.tables, sys.tables, etc., to display the table creation date and original name. Babelfish supports column ordering and NULLs ordering in table constraints while PostgreSQL does not and to handle dump/restore of this we dump the underlying index of a UNIQUE/PRIMARY KEY constraint and make use of ALTER TABLE ADD CONSTRAINT USING syntax to support column ordering and NULLs ordering in table constraints. PostgreSQL disallows creation of an constraint using index like this for partitioned table so we will need to bypass that check during restore of Babelfish databases. Signed-off-by: Sumit Jaiswal [email protected]
roshan0708
pushed a commit
to amazon-aurora/postgresql_modified_for_babelfish
that referenced
this pull request
Oct 18, 2024
…or-postgresql#399) To support partitioned tables and indexes in Babelfish, this commit introduces the following changes: Babelfish stores the creation date and original name of the table in the reloption, whereas PostgreSQL does not allow the usage of reloption for partitioned tables. This commit enables the usage of reloption for Babelfish partitioned tables to store the creation date and original name. This information will be utilized by various system views, such as information_schema_tsql.tables, sys.tables, etc., to display the table creation date and original name. Babelfish supports column ordering and NULLs ordering in table constraints while PostgreSQL does not and to handle dump/restore of this we dump the underlying index of a UNIQUE/PRIMARY KEY constraint and make use of ALTER TABLE ADD CONSTRAINT USING syntax to support column ordering and NULLs ordering in table constraints. PostgreSQL disallows creation of an constraint using index like this for partitioned table so we will need to bypass that check during restore of Babelfish databases. Signed-off-by: Sumit Jaiswal [email protected]
roshan0708
pushed a commit
to amazon-aurora/postgresql_modified_for_babelfish
that referenced
this pull request
Oct 18, 2024
…or-postgresql#399) To support partitioned tables and indexes in Babelfish, this commit introduces the following changes: Babelfish stores the creation date and original name of the table in the reloption, whereas PostgreSQL does not allow the usage of reloption for partitioned tables. This commit enables the usage of reloption for Babelfish partitioned tables to store the creation date and original name. This information will be utilized by various system views, such as information_schema_tsql.tables, sys.tables, etc., to display the table creation date and original name. Babelfish supports column ordering and NULLs ordering in table constraints while PostgreSQL does not and to handle dump/restore of this we dump the underlying index of a UNIQUE/PRIMARY KEY constraint and make use of ALTER TABLE ADD CONSTRAINT USING syntax to support column ordering and NULLs ordering in table constraints. PostgreSQL disallows creation of an constraint using index like this for partitioned table so we will need to bypass that check during restore of Babelfish databases. Signed-off-by: Sumit Jaiswal [email protected]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
To support partitioned tables and indexes in Babelfish, this commit introduces the following changes:
Babelfish stores the creation date and original name of the table in the reloption, whereas PostgreSQL does not allow the usage of reloption for partitioned tables. This commit enables the usage of reloption for Babelfish partitioned tables to store the creation date and original name. This information will be utilized by various system views, such as information_schema_tsql.tables, sys.tables, etc., to display the table creation date and original name.
Babelfish supports column ordering and NULLs ordering in table constraints while PostgreSQL does not and to handle dump/restore of this we dump the underlying index of a UNIQUE/PRIMARY KEY constraint and make use of ALTER TABLE ADD CONSTRAINT USING syntax to support column ordering and NULLs ordering in table constraints. PostgreSQL disallows creation of an constraint using index like this for partitioned table so we will need to bypass that check during restore of Babelfish databases.
Signed-off-by: Sumit Jaiswal [email protected]
Extension PR: babelfish-for-postgresql/babelfish_extensions#2702
Issues Resolved
Task: BABEL-4968, BABEL-4969
Check List
By submitting this pull request, I confirm that my contribution is under the terms of the PostgreSQL license, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.