[BACKPORT pg15-cherrypicks] all: Bulk port from master - 105

Summary:
 ead90cc [#23645] docdb: Fix tests timing out on TSAN after 15786f3
 0a6a31e [doc] Fix BNL flag defaults (#23945)
 54793c8 [#22925] docdb: Persist tserver registry entries to sys catalog
 fbef568 [PLAT-15378][localProvider][dr] Deflake testDrConfigSetup local provider test
 64ac031 [#23978] xCluster: set up sequences_data stream(s) on target universe
 8d228a8 [#23923] YSQL: Fix DDL atomicity check failure
 903d793 [PLAT-15328] Configure cgroup for non rhel9 machines as part of provision
 Excluded: 5dc71ea [#23882] YSQL: Improve cache re-invalidation for alter table commands
 1e70024 [DOC-480] CDC metric description and voyager minor fixes (#24028)
 7a4b409 [#23700] CDCSDK: Use leader epoch instead of leader term in table removal bg task
 4d922ca [#23922] docdb: Handle colocated tablets correctly in tablet limit checks.
 487bc77 [PLAT-15158 Update replication frequency tooltip
 2059eee [#24001] docdb: Replace tablet in tablegroup manager on repartition of colocated table
 90d4e93 [#24020] DocDB: Vector LSM
 294b7bb [PLAT-14435]Fix args parsing in failure detection py script
 Excluded: 872b59e [#23542] YSQL: Add new YSQL function yb_servers_metrics() to fetch metrics such as cpu/memory usage from all nodes in cluster
 252717b [PLAT-12263] G-Flag upgrade fails for tmp_dir if Rolling restart used

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, jenkins-bot

Differential Revision: https://phorge.dev.yugabyte.com/D38266

docs/content/preview/api/ysql/the-sql-language/statements/cmd_analyze.md

@@ -15,17 +15,20 @@ type: docs
 
 ANALYZE collects statistics about the contents of tables in the database, and stores the results in the `pg_statistic` system catalog. These statistics help the query planner to determine the most efficient execution plans for queries.
 
+The statistics are also used by the YugabyteDB [cost-based optimizer](../../../../../reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model) (CBO) to create optimal execution plans for queries. When run on up-to-date statistics, CBO provides performance improvements and can reduce or eliminate the need to use hints or modify queries to optimize query execution.
+
+{{< warning title="Run ANALYZE manually" >}}
+Currently, YugabyteDB doesn't run a background job like PostgreSQL autovacuum to analyze the tables. To collect or update statistics, run the ANALYZE command manually.
+
+If you have enabled CBO, you must run ANALYZE on user tables after data load for the CBO to create optimal execution plans.
+{{< /warning >}}
+
 The YugabyteDB implementation is based on the framework provided by PostgreSQL, which requires the storage layer to provide a random sample of rows of a predefined size. The size is calculated based on a number of factors, such as the included columns' data types.
 
-{{< note title="Note" >}}
+{{< note title="Large tables" >}}
 The sampling algorithm is not currently optimized for large tables. It may take several minutes to collect statistics from a table containing millions of rows of data.
 {{< /note >}}
 
-{{< note title="Note" >}}
-Currently, YugabyteDB doesn't run a background job like PostgreSQL's autovacuum to analyze the tables.
-To collect or update statistics, run the ANALYZE command manually.
-{{< /note >}}
-
 ## Syntax
 
 {{%ebnf%}}

docs/content/preview/explore/change-data-capture/using-logical-replication/monitor.md

@@ -76,7 +76,7 @@ Provide information about CDC service in YugabyteDB.
 | Metric name | Type | Description |
 | :---- | :---- | :---- |
 | cdcsdk_change_event_count | `long` | The number of records sent by the CDC Service. |
-| cdcsdk_traffic_sent | `long` | The number of milliseconds since the connector has read and processed the most recent event. |
+| cdcsdk_traffic_sent | `long` | Total traffic sent, in bytes. |
 | cdcsdk_event_lag_micros | `long` | The LAG metric is calculated by subtracting the timestamp of the latest record in the WAL of a tablet from the last record sent to the CDC connector. |
 | cdcsdk_expiry_time_ms | `long` | The time left to read records from WAL is tracked by the Stream Expiry Time (ms). |
 

docs/content/preview/explore/change-data-capture/using-yugabytedb-grpc-replication/cdc-monitor.md

@@ -59,7 +59,7 @@ Provide information about CDC service in YugabyteDB.
 | Metric name | Type | Description |
 | :---- | :---- | :---- |
 | cdcsdk_change_event_count | `long` | The Change Event Count metric shows the number of records sent by the CDC Service. |
-| cdcsdk_traffic_sent | `long` | The number of milliseconds since the connector has read and processed the most recent event. |
+| cdcsdk_traffic_sent | `long` | Total traffic sent, in bytes. |
 | cdcsdk_event_lag_micros | `long` | The LAG metric is calculated by subtracting the timestamp of the latest record in the WAL of a tablet from the last record sent to the CDC connector. |
 | cdcsdk_expiry_time_ms | `long` | The time left to read records from WAL is tracked by the Stream Expiry Time (ms). |
 

docs/content/preview/explore/ysql-language-features/join-strategies.md

@@ -211,11 +211,11 @@ The query plan would be similar to the following:
 
 In the case of nested loop joins, the inner table is accessed multiple times, once for each outer table row. This leads to multiple RPC requests across the different nodes in the cluster, making this join strategy very slow as the outer table gets larger.
 
-To reduce the number of requests sent across the nodes during the Nested loop join, YugabyteDB adds an optimization to batch multiple keys of the outer table into one RPC request. This batch size can be controlled using the YSQL configuration parameter [yb_bnl_batch_size](../../../reference/configuration/yb-tserver/#yb-bnl-batch-size), which defaults to `1` (which effectively means that the feature is `OFF`). The suggested value for this variable is `1024`.
+To reduce the number of requests sent across the nodes during the Nested loop join, YugabyteDB adds an optimization to batch multiple keys of the outer table into one RPC request. This batch size can be controlled using the YSQL configuration parameter [yb_bnl_batch_size](../../../reference/configuration/yb-tserver/#yb-bnl-batch-size), which defaults to `1024` (the suggested value for this variable).
 
 If `yb_bnl_batch_size` is greater than `1`, the optimizer will try to adopt the batching optimization when other join strategies are not fit for the current query.
 
-To enable the query planner's use of BNL, use the YB-TServer flag, [yb_enable_batchednl](../../../reference/configuration/yb-tserver/#yb-enable-batchednl).
+To enable the query planner's use of BNL, use the YSQL configuration parameter [yb_enable_batchednl](../../../reference/configuration/yb-tserver/#yb-enable-batchednl).
 
 To fetch all scores of students named `Natasha` who have scored more than `70` in any subject using Batched nested loop join, you would execute the following:
 

docs/content/preview/explore/ysql-language-features/postgresql-compatibility.md

@@ -50,14 +50,17 @@ Conversely, if you are using EPCM on a universe, you cannot set any of the featu
 | Wait-on-conflict | [enable_wait_queues](../../../reference/configuration/yb-tserver/#enable-wait-queues) | 2.20 | 2024.1 |
 | Cost-based optimizer | [yb_enable_base_scans_cost_model](../../../reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model) | 2024.1 | |
 | Batch nested loop join | [yb_enable_batchednl](../../../reference/configuration/yb-tserver/#yb-enable-batchednl) | 2.20 | 2024.1 |
-| Efficient communication<br>between PostgreSQL and DocDB | [pg_client_use_shared_memory](../../../reference/configuration/yb-tserver/#pg-client-use-shared-memory) | 2024.1 | | Yes |
 | Ascending indexing by default | [yb_use_hash_splitting_by_default](../../../reference/configuration/yb-tserver/#yb-use-hash-splitting-by-default) | 2024.1 | |
-| YugabyteDB bitmap scan | [yb_enable_bitmapscan](../../../reference/configuration/yb-tserver/#yb-enable-bitmapscan) | 2024.1.3<br>(Planned) | |
-| Parallel query | | Planned | |
+
+| Planned Feature | Flag/Configuration Parameter | EA |
+| :--- | :--- | :--- |
+| YugabyteDB bitmap scan | [yb_enable_bitmapscan](../../../reference/configuration/yb-tserver/#yb-enable-bitmapscan) | 2024.1.3 |
+| Efficient communication<br>between PostgreSQL and DocDB | [pg_client_use_shared_memory](../../../reference/configuration/yb-tserver/#pg-client-use-shared-memory) | 2024.2 |
+| Parallel query | | Planned |
 
 ### Released
 
-The following features are currently available.
+The following features are currently available in EPCM.
 
 #### Read committed
 
@@ -91,20 +94,14 @@ To learn more about concurrency control in YugabyteDB, see [Concurrency control]
 
 #### Batched nested loop join
 
-Configuration parameter: `yb_bnl_batch_size=1024`
+Configuration parameter: `yb_enable_batchednl=true`
 
 Batched nested loop join (BNLJ) is a join execution strategy that improves on nested loop joins by batching the tuples from the outer table into a single request to the inner table. By using batched execution, BNLJ helps reduce the latency for query plans that previously used nested loop joins. BNLJ provides improved performance parity.
 
 {{<lead link="../join-strategies/">}}
 To learn more about join strategies in YugabyteDB, see [Join strategies](../../../architecture/transactions/concurrency-control/).
 {{</lead>}}
 
-#### Efficient communication between PostgreSQL and DocDB
-
-Configuration parameter: `pg_client_use_shared_memory=true`
-
-Enable more efficient communication between YB-TServer and PostgreSQL using shared memory. This feature provides improved performance parity.
-
 #### Default ascending indexing
 
 Configuration parameter: `yb_use_hash_splitting_by_default=false`
@@ -125,6 +122,12 @@ Configuration parameter: `yb_enable_bitmapscan=true`
 
 Bitmap scans use multiple indexes to answer a query, with only one scan of the main table. Each index produces a "bitmap" indicating which rows of the main table are interesting. Bitmap scans can improve the performance of queries containing AND and OR conditions across several index scans. YugabyteDB bitmap scan provides feature compatibility and improved performance parity. For YugabyteDB relations to use a bitmap scan, the PostgreSQL parameter `enable_bitmapscan` must also be true (the default).
 
+#### Efficient communication between PostgreSQL and DocDB
+
+Configuration parameter: `pg_client_use_shared_memory=true`
+
+Enable more efficient communication between YB-TServer and PostgreSQL using shared memory. This feature provides improved performance parity.
+
 #### Parallel query
 
 Enables the use of PostgreSQL [parallel queries](https://www.postgresql.org/docs/11/parallel-query.html). Using parallel queries, the query planner can devise plans that leverage multiple CPUs to answer queries faster. Parallel query provides feature compatibility and improved performance parity.

docs/content/preview/yugabyte-voyager/migrate/live-fall-forward.md

@@ -1224,5 +1224,5 @@ Refer to [end migration](../../reference/end-migration/) for more details on the
 In addition to the Live migration [limitations](../live-migrate/#limitations), the following additional limitations apply to the fall-forward feature:
 
 - Fall-forward is unsupported with a YugabyteDB cluster running on [YugabyteDB Aeon](../../../yugabyte-cloud).
-- [SSL Connectivity](../../reference/yb-voyager-cli/#ssl-connectivity) is unsupported for export or streaming events from YugabyteDB during `export data from target`.
+- [SSL Connectivity](../../reference/yb-voyager-cli/#ssl-connectivity) is partially supported for export or streaming events from YugabyteDB during `export data from target`. Basic SSL and server authentication via root certificate is supported. Client authentication is not supported.
 - [Export data from target](../../reference/data-migration/export-data/#export-data-from-target) supports DECIMAL/NUMERIC datatypes for YugabyteDB versions 2.20.1.1 and later.

docs/content/preview/yugabyte-voyager/migrate/live-migrate.md

@@ -983,4 +983,4 @@ Refer to [end migration](../../reference/end-migration/) for more details on the
 - Truncating a table on the source database is not taken into account; you need to manually truncate tables on your YugabyteDB cluster.
 - Some Oracle data types are unsupported - User Defined Types (UDT), NCHAR, NVARCHAR, VARRAY, BLOB, CLOB, and NCLOB.
 - Case-sensitive table names or column names are partially supported. YugabyteDB Voyager converts them to case-insensitive names. For example, an "Accounts" table in a source Oracle database is migrated as `accounts` (case-insensitive) to a YugabyteDB database.
-- Tables or column names having more than 30 characters are not supported.
+- For Oracle source databases, table and column names with more than 30 characters are not supported.

docs/content/stable/api/ysql/the-sql-language/statements/cmd_analyze.md

@@ -15,17 +15,20 @@ type: docs
 
 ANALYZE collects statistics about the contents of tables in the database, and stores the results in the `pg_statistic` system catalog. These statistics help the query planner to determine the most efficient execution plans for queries.
 
+The statistics are also used by the YugabyteDB [cost-based optimizer](../../../../../reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model) (CBO) to create optimal execution plans for queries. When run on up-to-date statistics, CBO provides performance improvements and can reduce or eliminate the need to use hints or modify queries to optimize query execution.
+
+{{< warning title="Run ANALYZE manually" >}}
+Currently, YugabyteDB doesn't run a background job like PostgreSQL autovacuum to analyze the tables. To collect or update statistics, run the ANALYZE command manually.
+
+If you have enabled CBO, you must run ANALYZE on user tables after data load for the CBO to create optimal execution plans.
+{{< /warning >}}
+
 The YugabyteDB implementation is based on the framework provided by PostgreSQL, which requires the storage layer to provide a random sample of rows of a predefined size. The size is calculated based on a number of factors, such as the included columns' data types.
 
 {{< note title="Note" >}}
 The sampling algorithm is not currently optimized for large tables. It may take several minutes to collect statistics from a table containing millions of rows of data.
 {{< /note >}}
 
-{{< note title="Note" >}}
-Currently, YugabyteDB doesn't run a background job like PostgreSQL's autovacuum to analyze the tables.
-To collect or update statistics, run the ANALYZE command manually.
-{{< /note >}}
-
 ## Syntax
 
 {{%ebnf%}}

docs/content/stable/explore/change-data-capture/using-logical-replication/monitor.md

@@ -76,7 +76,7 @@ Provide information about CDC service in YugabyteDB.
 | Metric name | Type | Description |
 | :---- | :---- | :---- |
 | cdcsdk_change_event_count | `long` | The number of records sent by the CDC Service. |
-| cdcsdk_traffic_sent | `long` | The number of milliseconds since the connector has read and processed the most recent event. |
+| cdcsdk_traffic_sent | `long` | Total traffic sent, in bytes. |
 | cdcsdk_event_lag_micros | `long` | The LAG metric is calculated by subtracting the timestamp of the latest record in the WAL of a tablet from the last record sent to the CDC connector. |
 | cdcsdk_expiry_time_ms | `long` | The time left to read records from WAL is tracked by the Stream Expiry Time (ms). |
 

docs/content/stable/explore/change-data-capture/using-yugabytedb-grpc-replication/cdc-monitor.md

@@ -57,7 +57,7 @@ Provide information about CDC service in YugabyteDB.
 | Metric name | Type | Description |
 | :---- | :---- | :---- |
 | cdcsdk_change_event_count | `long` | The Change Event Count metric shows the number of records sent by the CDC Service. |
-| cdcsdk_traffic_sent | `long` | The number of milliseconds since the connector has read and processed the most recent event. |
+| cdcsdk_traffic_sent | `long` | Total traffic sent, in bytes. |
 | cdcsdk_event_lag_micros | `long` | The LAG metric is calculated by subtracting the timestamp of the latest record in the WAL of a tablet from the last record sent to the CDC connector. |
 | cdcsdk_expiry_time_ms | `long` | The time left to read records from WAL is tracked by the Stream Expiry Time (ms). |
 

docs/content/stable/explore/ysql-language-features/join-strategies.md

@@ -211,11 +211,11 @@ The query plan would be similar to the following:
 
 In the case of nested loop joins, the inner table is accessed multiple times, once for each outer table row. This leads to multiple RPC requests across the different nodes in the cluster, making this join strategy very slow as the outer table gets larger.
 
-To reduce the number of requests sent across the nodes during the Nested loop join, YugabyteDB adds an optimization to batch multiple keys of the outer table into one RPC request. This batch size can be controlled using the YSQL configuration parameter [yb_bnl_batch_size](../../../reference/configuration/yb-tserver/#yb-bnl-batch-size), which defaults to `1` (which effectively means that the feature is `OFF`). The suggested value for this variable is `1024`.
+To reduce the number of requests sent across the nodes during the Nested loop join, YugabyteDB adds an optimization to batch multiple keys of the outer table into one RPC request. This batch size can be controlled using the YSQL configuration parameter [yb_bnl_batch_size](../../../reference/configuration/yb-tserver/#yb-bnl-batch-size), which defaults to `1024` (the suggested value for this variable).
 
 If `yb_bnl_batch_size` is greater than `1`, the optimizer will try to adopt the batching optimization when other join strategies are not fit for the current query.
 
-To enable the query planner's use of BNL, use the YB-TServer flag, [yb_enable_batchednl](../../../reference/configuration/yb-tserver/#yb-enable-batchednl).
+To enable the query planner's use of BNL, use the YSQL configuration parameter [yb_enable_batchednl](../../../reference/configuration/yb-tserver/#yb-enable-batchednl).
 
 To fetch all scores of students named `Natasha` who have scored more than `70` in any subject using Batched nested loop join, you would execute the following:
 

docs/content/stable/explore/ysql-language-features/postgresql-compatibility.md

@@ -48,14 +48,17 @@ Conversely, if you are using EPCM on a universe, you cannot set any of the featu
 | Wait-on-conflict | [enable_wait_queues](../../../reference/configuration/yb-tserver/#enable-wait-queues) | 2.20 | 2024.1 |
 | Cost-based optimizer | [yb_enable_base_scans_cost_model](../../../reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model) | 2024.1 | |
 | Batch nested loop join | [yb_enable_batchednl](../../../reference/configuration/yb-tserver/#yb-enable-batchednl) | 2.20 | 2024.1 |
-| Efficient communication<br>between PostgreSQL and DocDB | [pg_client_use_shared_memory](../../../reference/configuration/yb-tserver/#pg-client-use-shared-memory) | 2024.1 | | Yes |
 | Ascending indexing by default | [yb_use_hash_splitting_by_default](../../../reference/configuration/yb-tserver/#yb-use-hash-splitting-by-default) | 2024.1 | |
-| YugabyteDB bitmap scan | [yb_enable_bitmapscan](../../../reference/configuration/yb-tserver/#yb-enable-bitmapscan) | 2024.1.3<br>(Planned) | |
-| Parallel query | | Planned | |
+
+| Planned Feature | Flag/Configuration Parameter | EA |
+| :--- | :--- | :--- |
+| YugabyteDB bitmap scan | [yb_enable_bitmapscan](../../../reference/configuration/yb-tserver/#yb-enable-bitmapscan) | 2024.1.3 |
+| Efficient communication<br>between PostgreSQL and DocDB | [pg_client_use_shared_memory](../../../reference/configuration/yb-tserver/#pg-client-use-shared-memory) | 2024.2 |
+| Parallel query | | Planned |
 
 ### Released
 
-The following features are currently available.
+The following features are currently available in EPCM.
 
 #### Read committed
 
@@ -89,20 +92,14 @@ To learn more about concurrency control in YugabyteDB, see [Concurrency control]
 
 #### Batched nested loop join
 
-Configuration parameter: `yb_bnl_batch_size=1024`
+Configuration parameter: `yb_enable_batchednl=true`
 
 Batched nested loop join (BNLJ) is a join execution strategy that improves on nested loop joins by batching the tuples from the outer table into a single request to the inner table. By using batched execution, BNLJ helps reduce the latency for query plans that previously used nested loop joins. BNLJ provides improved performance parity.
 
 {{<lead link="../join-strategies/">}}
 To learn more about join strategies in YugabyteDB, see [Join strategies](../../../architecture/transactions/concurrency-control/).
 {{</lead>}}
 
-#### Efficient communication between PostgreSQL and DocDB
-
-Configuration parameter: `pg_client_use_shared_memory=true`
-
-Enable more efficient communication between YB-TServer and PostgreSQL using shared memory. This feature provides improved performance parity.
-
 #### Default ascending indexing
 
 Configuration parameter: `yb_use_hash_splitting_by_default=false`
@@ -123,6 +120,12 @@ Configuration parameter: `yb_enable_bitmapscan=true`
 
 Bitmap scans use multiple indexes to answer a query, with only one scan of the main table. Each index produces a "bitmap" indicating which rows of the main table are interesting. Bitmap scans can improve the performance of queries containing AND and OR conditions across several index scans. YugabyteDB bitmap scan provides feature compatibility and improved performance parity. For YugabyteDB relations to use a bitmap scan, the PostgreSQL parameter `enable_bitmapscan` must also be true (the default).
 
+#### Efficient communication between PostgreSQL and DocDB
+
+Configuration parameter: `pg_client_use_shared_memory=true`
+
+Enable more efficient communication between YB-TServer and PostgreSQL using shared memory. This feature provides improved performance parity.
+
 #### Parallel query
 
 Enables the use of PostgreSQL [parallel queries](https://www.postgresql.org/docs/11/parallel-query.html). Using parallel queries, the query planner can devise plans that leverage multiple CPUs to answer queries faster. Parallel query provides feature compatibility and improved performance parity.