[YSQL] Feature to temporarily disable DDL catalog version bump #23786

kai-franz · 2024-09-03T21:46:28Z

Description

Currently, most DDLs bump the catalog version, forcing all backend processes to stop and refresh their cache.

Sometimes, when we are doing many DDLs in one script, we'd like to be able to have some way of bumping the catalog version once for all the DDLs instead of once per DDL (assuming it is safe to do so).

There is a similar existing feature (SET yb_make_next_ddl_statement_nonbreaking = true;) which makes the next DDL non-breaking.

We would like to have something like SET yb_make_next_ddl_statement_nonincrementing = true;.

Issue Type

kind/new-feature

Warning: Please confirm that this issue does not contain any sensitive information

I confirm this issue does not contain any sensitive information.

The text was updated successfully, but these errors were encountered:

Summary: In a recent customer issue investigation, the customer performed an application upgrade, which involves running many DDLs (100+) consecutively. Many of these DDLs cause catalog version to increment. As a result, all the active connections kept doing catalog cache refreshes continuously that led to PG memory spike and latency spike. It will be beneficial to reduce the number of DDLs that increment the catalog version. After analyzing customer's DDL statements that have caused catalog version to increment, they can be classified as 3 categories: (1) create index statements that build indexes on a newly created table (2) create new partition statements and link them to the existing parent partition (3) create new table but referencing an existing table via foreign key For (1), because the table itself is newly created, there is no need for the create index statement to increment the catalog version. To make things worse, a create index statement by default runs concurrently, and increments the catalog version by 3. In this particular customer's situation, more than half of the DDLs belong to category (1). One solution is to combine create statement and the create index statement together. Using an example found online: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text); CREATE TABLE yugabyte=# create unique index foo_idx on foo using btree (code, label); NOTICE: index method "btree" was replaced with "lsm" in YugabyteDB CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 4 | 1 13252 | 1 | 1 (5 rows) ``` By create table and then create a unique index, the catalog version was incremented by 3. On the other hand, we can combine them into one create table statement: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text, yugabyte(# constraint foo_uq unique (code, label)); select *CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` We can see that there was no catalog version increment. However it is not always possible to do these rewrite. The create index statement does allow more options, such as specifying ASC, DESC which isn't allowed when declaring a uniqe constraint in the create table statement itself. It is in such a case we introduce the new GUC `yb_make_next_ddl_statement_nonincrementing` to help. This is an auto-reset gflag that is done similar to the existing GUC `yb_make_next_ddl_statement_nonbreaking`. When set to true, it will suppress the next DDL statement from incrementing both the `current_version` and the `last_breaking_version`. Note that in order for the GUC to work for create index statement, we need to use the nonconcurrent keyword. By default the create index statement runs concurrently and its algorithm involves incrementing the catalog version by 3 to work correctly. A create index noncurrently only bumps up the catalog version by 1. For a newly created table, because it is empty, it is safe and correct to run create index nonconcurrently. For example, ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 2 | 1 13252 | 1 | 1 (5 rows) ``` With the new GUC `yb_make_next_ddl_statement_nonincrementing`, the catalog version will stay the same: ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# set yb_make_next_ddl_statement_nonincrementing = true; SET yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` A new unit test is added. Jira: DB-12689 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: svc_phabricator, yql Differential Revision: https://phorge.dev.yugabyte.com/D37915

…nting Summary: In a recent customer issue investigation, the customer performed an application upgrade, which involves running many DDLs (100+) consecutively. Many of these DDLs cause catalog version to increment. As a result, all the active connections kept doing catalog cache refreshes continuously that led to PG memory spike and latency spike. It will be beneficial to reduce the number of DDLs that increment the catalog version. After analyzing customer's DDL statements that have caused catalog version to increment, they can be classified as 3 categories: (1) create index statements that build indexes on a newly created table (2) create new partition statements and link them to the existing parent partition (3) create new table but referencing an existing table via foreign key For (1), because the table itself is newly created, there is no need for the create index statement to increment the catalog version. To make things worse, a create index statement by default runs concurrently, and increments the catalog version by 3. In this particular customer's situation, more than half of the DDLs belong to category (1). One solution is to combine create statement and the create index statement together. Using an example found online: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text); CREATE TABLE yugabyte=# create unique index foo_idx on foo using btree (code, label); NOTICE: index method "btree" was replaced with "lsm" in YugabyteDB CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 4 | 1 13252 | 1 | 1 (5 rows) ``` By create table and then create a unique index, the catalog version was incremented by 3. On the other hand, we can combine them into one create table statement: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text, yugabyte(# constraint foo_uq unique (code, label)); select *CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` We can see that there was no catalog version increment. However it is not always possible to do these rewrite. The create index statement does allow more options, such as specifying ASC, DESC which isn't allowed when declaring a uniqe constraint in the create table statement itself. It is in such a case we introduce the new GUC `yb_make_next_ddl_statement_nonincrementing` to help. This is an auto-reset gflag that is done similar to the existing GUC `yb_make_next_ddl_statement_nonbreaking`. When set to true, it will suppress the next DDL statement from incrementing both the `current_version` and the `last_breaking_version`. Note that in order for the GUC to work for create index statement, we need to use the nonconcurrent keyword. By default the create index statement runs concurrently and its algorithm involves incrementing the catalog version by 3 to work correctly. A create index noncurrently only bumps up the catalog version by 1. For a newly created table, because it is empty, it is safe and correct to run create index nonconcurrently. For example, ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 2 | 1 13252 | 1 | 1 (5 rows) ``` With the new GUC `yb_make_next_ddl_statement_nonincrementing`, the catalog version will stay the same: ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# set yb_make_next_ddl_statement_nonincrementing = true; SET yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` A new unit test is added. Jira: DB-12689 Original commit: 063dbe5 / D37915 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: yql, svc_phabricator Differential Revision: https://phorge.dev.yugabyte.com/D38101

Summary: In a recent customer issue investigation, the customer performed an application upgrade, which involves running many DDLs (100+) consecutively. Many of these DDLs cause catalog version to increment. As a result, all the active connections kept doing catalog cache refreshes continuously that led to PG memory spike and latency spike. It will be beneficial to reduce the number of DDLs that increment the catalog version. After analyzing customer's DDL statements that have caused catalog version to increment, they can be classified as 3 categories: (1) create index statements that build indexes on a newly created table (2) create new partition statements and link them to the existing parent partition (3) create new table but referencing an existing table via foreign key For (1), because the table itself is newly created, there is no need for the create index statement to increment the catalog version. To make things worse, a create index statement by default runs concurrently, and increments the catalog version by 3. In this particular customer's situation, more than half of the DDLs belong to category (1). One solution is to combine create statement and the create index statement together. Using an example found online: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text); CREATE TABLE yugabyte=# create unique index foo_idx on foo using btree (code, label); NOTICE: index method "btree" was replaced with "lsm" in YugabyteDB CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 4 | 1 13252 | 1 | 1 (5 rows) ``` By create table and then create a unique index, the catalog version was incremented by 3. On the other hand, we can combine them into one create table statement: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text, yugabyte(# constraint foo_uq unique (code, label)); select *CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` We can see that there was no catalog version increment. However it is not always possible to do these rewrite. The create index statement does allow more options, such as specifying ASC, DESC which isn't allowed when declaring a uniqe constraint in the create table statement itself. It is in such a case we introduce the new GUC `yb_make_next_ddl_statement_nonincrementing` to help. This is an auto-reset gflag that is done similar to the existing GUC `yb_make_next_ddl_statement_nonbreaking`. When set to true, it will suppress the next DDL statement from incrementing both the `current_version` and the `last_breaking_version`. Note that in order for the GUC to work for create index statement, we need to use the nonconcurrent keyword. By default the create index statement runs concurrently and its algorithm involves incrementing the catalog version by 3 to work correctly. A create index noncurrently only bumps up the catalog version by 1. For a newly created table, because it is empty, it is safe and correct to run create index nonconcurrently. For example, ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 2 | 1 13252 | 1 | 1 (5 rows) ``` With the new GUC `yb_make_next_ddl_statement_nonincrementing`, the catalog version will stay the same: ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# set yb_make_next_ddl_statement_nonincrementing = true; SET yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` A new unit test is added. Jira: DB-12689 Original commit: 063dbe5 / D37915 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: yql, svc_phabricator Differential Revision: https://phorge.dev.yugabyte.com/D38128

Summary: f97d7d5 [#23770] [#23797] [#23837] YSQL: Fix most non-regress tests when run with Connection Manager ee2b108 [docs] reverted PR 23909 changes (#23941) bd80f4e [#23924] DocDB: Address recent flakiness of PgGetLockStatusTest.TestGetWaitStart Excluded: f0a5db7 [#20908] YSQL: Introduce interface to optimize index non-key column updates 6556498 [#23897] xClusterDDLRepl: Support create table with partition by primary key e2b1d28 [#23363] Changing the default gflags of YSQL memory configuration. e717f43 [#23925] DocDB: Address recent regression of test PgWaitQueuesTest.MultiTabletFairness 09b7702 [#23940] YSQL: Replace copyright string from YugaByteDB to YugabyteDB add83ef [#23947] Update callhome URL to use https 69db717 Update faq page (#23704) Excluded: 063dbe5 [#23786] YSQL: yb_make_next_ddl_statement_nonincrementing Excluded: a913524 [#23859] YSQL: Remove redundant Bitmap Scan filters on partial indexes 41f5afd [PLAT-15097]: Delete System platform DB table while disabling ysql d6bbf59 [#23890] docdb: Add filtering for bootstrap intent iterators based on min_replay_txn_start_ht 0b37479 [#23770] YSQL: Stabalize TestPgExplainAnalyze#testExplainAnalyzeOptions the test with ysql connection manager Test Plan: Jenkins: rebase: pg15-cherrypicks Reviewers: jason, tfoucher Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D38123

…nonincrementing Summary: ``` - guc.c: - yb_plpgsql_disable_prefetch_in_for_query: - pg15 commit 91a265a touches GUC yb_plpgsql_disable_prefetch_in_for_query (QUERY_TUNING to QUERY_TUNING_METHOD) - master commit 063dbe5 adds GUC yb_make_next_ddl_statement_nonincrementing above that - adjacent lines conflict ``` In a recent customer issue investigation, the customer performed an application upgrade, which involves running many DDLs (100+) consecutively. Many of these DDLs cause catalog version to increment. As a result, all the active connections kept doing catalog cache refreshes continuously that led to PG memory spike and latency spike. It will be beneficial to reduce the number of DDLs that increment the catalog version. After analyzing customer's DDL statements that have caused catalog version to increment, they can be classified as 3 categories: (1) create index statements that build indexes on a newly created table (2) create new partition statements and link them to the existing parent partition (3) create new table but referencing an existing table via foreign key For (1), because the table itself is newly created, there is no need for the create index statement to increment the catalog version. To make things worse, a create index statement by default runs concurrently, and increments the catalog version by 3. In this particular customer's situation, more than half of the DDLs belong to category (1). One solution is to combine create statement and the create index statement together. Using an example found online: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text); CREATE TABLE yugabyte=# create unique index foo_idx on foo using btree (code, label); NOTICE: index method "btree" was replaced with "lsm" in YugabyteDB CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 4 | 1 13252 | 1 | 1 (5 rows) ``` By create table and then create a unique index, the catalog version was incremented by 3. On the other hand, we can combine them into one create table statement: ``` yugabyte=# create table foo ( yugabyte(# id serial primary key, yugabyte(# code integer, yugabyte(# label text, yugabyte(# constraint foo_uq unique (code, label)); select *CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` We can see that there was no catalog version increment. However it is not always possible to do these rewrite. The create index statement does allow more options, such as specifying ASC, DESC which isn't allowed when declaring a uniqe constraint in the create table statement itself. It is in such a case we introduce the new GUC `yb_make_next_ddl_statement_nonincrementing` to help. This is an auto-reset gflag that is done similar to the existing GUC `yb_make_next_ddl_statement_nonbreaking`. When set to true, it will suppress the next DDL statement from incrementing both the `current_version` and the `last_breaking_version`. Note that in order for the GUC to work for create index statement, we need to use the nonconcurrent keyword. By default the create index statement runs concurrently and its algorithm involves incrementing the catalog version by 3 to work correctly. A create index noncurrently only bumps up the catalog version by 1. For a newly created table, because it is empty, it is safe and correct to run create index nonconcurrently. For example, ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 2 | 1 13252 | 1 | 1 (5 rows) ``` With the new GUC `yb_make_next_ddl_statement_nonincrementing`, the catalog version will stay the same: ``` yugabyte=# create table foo(id int); CREATE TABLE yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) yugabyte=# set yb_make_next_ddl_statement_nonincrementing = true; SET yugabyte=# create index nonconcurrently id_idx on foo(id); CREATE INDEX yugabyte=# select * from pg_yb_catalog_version; db_oid | current_version | last_breaking_version --------+-----------------+----------------------- 1 | 1 | 1 13248 | 1 | 1 13249 | 1 | 1 13251 | 1 | 1 13252 | 1 | 1 (5 rows) ``` A new unit test is added. Jira: DB-12689 Original commit: 063dbe5 / D37915 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: jason, tfoucher Reviewed By: jason Subscribers: yql, svc_phabricator Differential Revision: https://phorge.dev.yugabyte.com/D38141

…AdminVariables Summary: The yb_make_next_ddl_statement_nonincrementing has PGC_SUSET and needs to be added to YbDbAdminVariables in order for yb_db_admin role to set it. Otherwise, we see permission denied error for yb_db_admin: ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; ERROR: permission denied to set parameter "yb_make_next_ddl_statement_nonincrementing" ``` I added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables, and the above error is gone ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; SET ``` Jira: DB-12689 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D38135

…rementing to YbDbAdminVariables Summary: The yb_make_next_ddl_statement_nonincrementing has PGC_SUSET and needs to be added to YbDbAdminVariables in order for yb_db_admin role to set it. Otherwise, we see permission denied error for yb_db_admin: ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; ERROR: permission denied to set parameter "yb_make_next_ddl_statement_nonincrementing" ``` I added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables, and the above error is gone ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; SET ``` Jira: DB-12689 Original commit: 2beb872 / D38135 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D38173

…menting to YbDbAdminVariables Summary: The yb_make_next_ddl_statement_nonincrementing has PGC_SUSET and needs to be added to YbDbAdminVariables in order for yb_db_admin role to set it. Otherwise, we see permission denied error for yb_db_admin: ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; ERROR: permission denied to set parameter "yb_make_next_ddl_statement_nonincrementing" ``` I added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables, and the above error is gone ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; SET ``` Jira: DB-12689 Original commit: 2beb872 / D38135 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D38175

…rementing to YbDbAdminVariables Summary: The yb_make_next_ddl_statement_nonincrementing has PGC_SUSET and needs to be added to YbDbAdminVariables in order for yb_db_admin role to set it. Otherwise, we see permission denied error for yb_db_admin: ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; ERROR: permission denied to set parameter "yb_make_next_ddl_statement_nonincrementing" ``` I added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables, and the above error is gone ``` yugabyte=# set role yb_db_admin; SET yugabyte=> set yb_make_next_ddl_statement_nonincrementing to true; SET ``` Jira: DB-12689 Original commit: 2beb872 / D38135 Test Plan: ./yb_build.sh --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.NonIncrementingDDLMode Reviewers: kfranz Reviewed By: kfranz Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D38174

Summary: 6b34652 [doc] Update badge colors (#23985) 6dcfce6 [PLAT-15297][dr] GET /configs fails with java.lang.IllegalStateException: DrConfig iTest-dr(uuid) does not have any corresponding xCluster config 7995311 [#23695] docdb: Improve help strings for tablet replica limit flags. 7659151 [DOC-479] Voyager 1.8.1 updates (#23983) 56c9cc9 [PLAT-15282]: fix NPE on k8s operator backup/restore status update c8776b5 [docs] SSL/TLS configuration with hashicorp vault YSQL plugin (#23979) Excluded: d3dd4f7 Bumping version to 2.25.0.0 on branch master Excluded: 0dbe7d6 [#23312] YSQL: Fix backup/restore for colocated tables without tablespace information. 2beb872 [#23786] YSQL: add yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables 28d0d8e [#24011] build: fix ./yb_build.sh --gcc13 9feb6e8 [#24012] YSQL: Replace deprecated function shared_ptr::unique to fix macos compiler error 2cc1480 [PLAT-15349] Fix update_lb_config automated NLB updates af4c46d [#23741] docdb: Add parameter to run PITR tests with clone instead 0103dfc [PLAT-14459] Continuous backups for s3 e6bb890 [PLAT-15192] [PLAT-15191] Document node-agent-provision.yaml 1552321 [PLAT-15205] YBA callhome improvements ba6acbd [PLAT-15342] Install Node Agent as root level unit for ynp 2478a50 [PLAT-13058] Remove references to public key filepath, since we don't need ssh keys for node agent based provisioning aa14f76 [#23975] YSQL, ASH: Fix TestYbAsh#testPgAuxInfo 885b72b [#23960] DocDB: Fix WriteBuffer crash in move ctor Excluded: f4a002a [#23314] YSQL: Fix variable name to yb_ for ATPrepSetTableSpace 29c56ed [PLAT-15369][YBA CLI]Backup list-increments doesn't support pretty/json outputs 784160c [PLAT-15370][YBA CLI]backup list table output contains base_backup_uuid and backup_uuid b32222d [PLAT-15280][PLAT-14753] - Pg parity improvements and other ui fixes c6521cf [PLAT-15368][YBA CLI]Backup list print "No more backup present" after printing all the backups in -o pretty/json - pg_catalog_version-test.cc - NonIncrementingDDLMode: YB master commit 2beb872 sets the role to yb_db_admin, which due to new behavior in upstream PG commit b073c3ccd06e4cb845e121387a43faa8c68a7b62 (documented in https://www.postgresql.org/docs/15/ddl-schemas.html#DDL-SCHEMAS-PATTERNS) causes the following unintended test failure: ERROR: permission denied for schema public. Add a workaround to grant create permission to the role now set. Test Plan: Jenkins: urgent, rebase: pg15-cherrypicks Reviewers: jason, jenkins-bot Differential Revision: https://phorge.dev.yugabyte.com/D38220

kai-franz added area/ysql Yugabyte SQL (YSQL) status/awaiting-triage Issue awaiting triage labels Sep 3, 2024

yugabyte-ci added kind/new-feature This is a request for a completely new feature priority/medium Medium priority issue labels Sep 3, 2024

sushantrmishra removed the status/awaiting-triage Issue awaiting triage label Sep 4, 2024

yugabyte-ci added status/awaiting-triage Issue awaiting triage and removed status/awaiting-triage Issue awaiting triage labels Sep 4, 2024

sushantrmishra assigned myang2021 Sep 9, 2024

sushantrmishra added 2.20 Backport Required 2024.1 Backport Required labels Sep 9, 2024

yugabyte-ci added the 2.20.8_blocker label Sep 19, 2024

myang2021 closed this as completed Oct 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[YSQL] Feature to temporarily disable DDL catalog version bump #23786

[YSQL] Feature to temporarily disable DDL catalog version bump #23786

kai-franz commented Sep 3, 2024 •

edited by jira bot

Loading

[YSQL] Feature to temporarily disable DDL catalog version bump #23786

[YSQL] Feature to temporarily disable DDL catalog version bump #23786

Comments

kai-franz commented Sep 3, 2024 • edited by jira bot Loading

Description

Issue Type

Warning: Please confirm that this issue does not contain any sensitive information

kai-franz commented Sep 3, 2024 •

edited by jira bot

Loading