[DocDB] xCluster: New table added to a colocated database could stop getting replicated after a t-server restart #23188
Assignees
Labels
Comments
lingamsandeep
added
area/docdb
yugabyte-ci
added
kind/bug
yugabyte-ci
added
priority/high
lingamsandeep
added a commit
that referenced
this issue
Jul 18, 2024
…f processing CHANGE_METADATA_OP in xCluster ClusterConfig. Summary: Problem: When a new table is added to a colocated database, the table needs to be created on both xCluster source and target with the same colocation_id. As part of processing the ChangeMetadataOp for the AddTable operation received from the source, the target creates a mapping of source->target schema versions of the newly created table. However, this is not getting persisted in ClusterConfig if the source/target schema versions are default values. As a result while replication may work immediately after setup, upon a restart of the T-server, the mapping of source-target schema versions may be lost and replication may stall until another schema change happens on the source. Fix: Fix is to detect the non-existent colocation_id correctly and persisting the ClusterConfig. Test Plan: ybt xcluster_ysql_colocated-test XClusterYsqlColocatedTest.DatabaseReplication The test was failing without the fix and is passing with the fix. Reviewers: hsunder, xCluster Reviewed By: hsunder Subscribers: ybase Differential Revision: https://phorge.dev.yugabyte.com/D36552
1 task
lingamsandeep
added a commit
that referenced
this issue
Jul 18, 2024
…covered as part of processing CHANGE_METADATA_OP in xCluster ClusterConfig. Summary: Original commit: 9f82c01 / D36552 Problem: When a new table is added to a colocated database, the table needs to be created on both xCluster source and target with the same colocation_id. As part of processing the ChangeMetadataOp for the AddTable operation received from the source, the target creates a mapping of source->target schema versions of the newly created table. However, this is not getting persisted in ClusterConfig if the source/target schema versions are default values. As a result while replication may work immediately after setup, upon a restart of the T-server, the mapping of source-target schema versions may be lost and replication may stall until another schema change happens on the source. Fix: Fix is to detect the non-existent colocation_id correctly and persisting the ClusterConfig. Test Plan: ybt xcluster_ysql_colocated-test XClusterYsqlColocatedTest.DatabaseReplication The test was failing without the fix and is passing with the fix. Reviewers: hsunder, xCluster Reviewed By: hsunder Subscribers: ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36699
jasonyb
pushed a commit
that referenced
this issue
Jul 19, 2024
Summary: a64fbfc [#23120] YSQL: Do not wait for a safe snapshot in serializable read only deferrable mode. 9f82c01 [#23188] DocDB: Persist new colocated_id mapping discovered as part of processing CHANGE_METADATA_OP in xCluster ClusterConfig. 1522522 Update third-party dependencies to remove CentOS 7 cd7410c 2.18.1.1 release notes (#23241) 550458d [#23047] docdb: Fix cotable ids in flushed frontier at restore 225ddfe [PLAT-14700] Make node-agent error message on installation to be more precise 9c9a059 [PLAT-14563] Fixing disk mounting logic to use mount points defined in the config 84eefbd [DOC-368] Azure workload identity docs changes (#22881) 781af0d [#23183] xCluster: Move Setup, Bootstrap, Alter and Delete Target Replication functions out of CatalogManager 1d646b1 [docs] Add ysql_output_buffer_size to yb-tserver config reference. (#23233) 2d95dd2 Fix the issue yaml parsing dd9e85b [PLAT-14534]Add regex match for GCP Instance template d3bba18 update diagram (#23245) 78b317c [/PLAT-14708] Fix JSON field name in TaskInfo query ac9164b [#23173] DocDB: Allow large bytes to be passed to RateLimiter Test Plan: Jenkins: rebase: pg15-cherrypicks Reviewers: jason, tfoucher Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36723
lingamsandeep
added a commit
that referenced
this issue
Jul 20, 2024
…vered as part of processing CHANGE_METADATA_OP in xCluster ClusterConfig. Summary: Original commit: 9f82c01 / D36552 Problem: When a new table is added to a colocated database, the table needs to be created on both xCluster source and target with the same colocation_id. As part of processing the ChangeMetadataOp for the AddTable operation received from the source, the target creates a mapping of source->target schema versions of the newly created table. However, this is not getting persisted in ClusterConfig if the source/target schema versions are default values. As a result while replication may work immediately after setup, upon a restart of the T-server, the mapping of source-target schema versions may be lost and replication may stall until another schema change happens on the source. Fix: Fix is to detect the non-existent colocation_id correctly and persisting the ClusterConfig. Test Plan: ybt xcluster_ysql_colocated-test XClusterYsqlColocatedTest.DatabaseReplication The test was failing without the fix and is passing with the fix. Reviewers: hsunder, xCluster Reviewed By: hsunder Subscribers: ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36700
lingamsandeep
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-12129
Description
When a new table is added to a colocated database, the table needs to be created on both xCluster source and target with the same colocation_id.
As part of processing the ChangeMetadataOp for the AddTable operation received from the source, the target creates a mapping of source->target schema versions of the newly created table. However, this is not getting persisted in ClusterConfig. As a result while replication may work immediately after setup, upon a restart of the T-server, the mapping of source-target schema versions may be lost and replication may stall until another schema change happens on the source.
Issue Type
kind/bug
Warning: Please confirm that this issue does not contain any sensitive information
The text was updated successfully, but these errors were encountered: