From 26ee595f892b7176242deb299e7db49439d01536 Mon Sep 17 00:00:00 2001 From: xhe Date: Fri, 18 Mar 2022 15:22:24 +0800 Subject: [PATCH 01/19] *: refine placement documents Signed-off-by: xhe --- TOC.md | 2 +- .../information-schema-placement-policies.md | 72 +++++++++++++++++ .../information-schema-placement-rules.md | 77 ------------------- information-schema/information-schema.md | 2 +- placement-rules-in-sql.md | 57 +++++++++++--- .../sql-statement-alter-placement-policy.md | 35 +++++---- .../sql-statement-create-placement-policy.md | 15 +++- ...-statement-show-create-placement-policy.md | 8 +- .../sql-statement-show-placement-for.md | 22 +++--- 9 files changed, 165 insertions(+), 125 deletions(-) create mode 100644 information-schema/information-schema-placement-policies.md delete mode 100644 information-schema/information-schema-placement-rules.md diff --git a/TOC.md b/TOC.md index a238905c9ec09..c8b23709cf798 100644 --- a/TOC.md +++ b/TOC.md @@ -585,7 +585,7 @@ - [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) - [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) - [`PARTITIONS`](/information-schema/information-schema-partitions.md) - - [`PLACEMENT_RULES`](/information-schema/information-schema-placement-rules.md) + - [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) - [`PROCESSLIST`](/information-schema/information-schema-processlist.md) - [`REFERENTIAL_CONSTRAINTS`](/information-schema/information-schema-referential-constraints.md) - [`SCHEMATA`](/information-schema/information-schema-schemata.md) diff --git a/information-schema/information-schema-placement-policies.md b/information-schema/information-schema-placement-policies.md new file mode 100644 index 0000000000000..2953fe206e1af --- /dev/null +++ b/information-schema/information-schema-placement-policies.md @@ -0,0 +1,72 @@ +--- +title: PLACEMENT_POLICIES +sgmmary: Learn the `PLACEMENT_POLICIES` information_schema table. +--- + +# PLACEMENT_POLICIES + +The `PLACEMENT_POLICIES` table provides information on all explicitly configured [Placement Rules in SQL](/placement-rules-in-sql.md), which means all placement policies. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC placement_policies; +``` + +```sql ++----------------------+---------------+------+-----+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++----------------------+---------------+------+-----+---------+-------+ +| POLICY_ID | bigint(64) | NO | | | | +| CATALOG_NAME | varchar(512) | NO | | | | +| POLICY_NAME | varchar(64) | NO | | | | +| PRIMARY_REGION | varchar(1024) | YES | | | | +| REGIONS | varchar(1024) | YES | | | | +| CONSTRAINTS | varchar(1024) | YES | | | | +| LEADER_CONSTRAINTS | varchar(1024) | YES | | | | +| FOLLOWER_CONSTRAINTS | varchar(1024) | YES | | | | +| LEARNER_CONSTRAINTS | varchar(1024) | YES | | | | +| SCHEDULE | varchar(20) | YES | | | | +| FOLLOWERS | bigint(64) | YES | | | | +| LEARNERS | bigint(64) | YES | | | | ++----------------------+---------------+------+-----+---------+-------+ +12 rows in set (0.00 sec) +``` + +## Examples + +The `PLACEMENT_POLICIES` table only shows explicitly configured rules. To see the canonical version of placement rules (including placement policies attached to tables), use the statement `SHOW PLACEMENT` instead: + +{{< copyable "sql" >}} + +```sql +CREATE TABLE t1 (a INT); +CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1"; +CREATE TABLE t3 (a INT) PLACEMENT POLICY=p1; +SHOW PLACEMENT; -- Includes t3. +SELECT * FROM information_schema.placement_policies; -- Does not include t3. +``` + +```sql +Query OK, 0 rows affected (0.09 sec) + +Query OK, 0 rows affected (0.11 sec) + +Query OK, 0 rows affected (0.08 sec) + ++---------------+------------------------------------------------+------------------+ +| Target | Placement | Scheduling_State | ++---------------+------------------------------------------------+------------------+ +| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | NULL | +| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | PENDING | ++---------------+------------------------------------------------+------------------+ +2 rows in set (0.00 sec) + ++-----------+--------------+-------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ +| POLICY_ID | CATALOG_NAME | POLICY_NAME | PRIMARY_REGION | REGIONS | CONSTRAINTS | LEADER_CONSTRAINTS | FOLLOWER_CONSTRAINTS | LEARNER_CONSTRAINTS | SCHEDULE | FOLLOWERS | LEARNERS | ++-----------+--------------+-------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ +| 1 | def | p1 | us-east-1 | us-east-1 | | | | | | 2 | 0 | ++-----------+--------------+-------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ +1 rows in set (0.00 sec) +``` diff --git a/information-schema/information-schema-placement-rules.md b/information-schema/information-schema-placement-rules.md deleted file mode 100644 index 83d0c43f12299..0000000000000 --- a/information-schema/information-schema-placement-rules.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: PLACEMENT_RULES -summary: Learn the `PLACEMENT_RULES` information_schema table. ---- - -# PLACEMENT_RULES - -The `PLACEMENT_RULES` table provides information on all explicitly configured [Placement Rules in SQL](/placement-rules-in-sql.md), which means all placement policies. - -{{< copyable "sql" >}} - -```sql -USE information_schema; -DESC placement_rules; -``` - -```sql -+----------------------+--------------+------+------+---------+-------+ -| Field | Type | Null | Key | Default | Extra | -+----------------------+--------------+------+------+---------+-------+ -| POLICY_ID | bigint(64) | NO | | NULL | | -| CATALOG_NAME | varchar(512) | NO | | NULL | | -| POLICY_NAME | varchar(5) | YES | | NULL | | -| SCHEMA_NAME | varchar(5) | YES | | NULL | | -| TABLE_NAME | varchar(5) | YES | | NULL | | -| PARTITION_NAME | varchar(5) | YES | | NULL | | -| PRIMARY_REGION | varchar(5) | NO | | NULL | | -| REGIONS | varchar(5) | NO | | NULL | | -| CONSTRAINTS | varchar(5) | NO | | NULL | | -| LEADER_CONSTRAINTS | varchar(5) | NO | | NULL | | -| FOLLOWER_CONSTRAINTS | varchar(5) | NO | | NULL | | -| LEARNER_CONSTRAINTS | varchar(5) | NO | | NULL | | -| SCHEDULE | varchar(20) | NO | | NULL | | -| FOLLOWERS | bigint(64) | NO | | NULL | | -| LEARNERS | bigint(64) | NO | | NULL | | -+----------------------+--------------+------+------+---------+-------+ -15 rows in set (0.00 sec) -``` - -## Examples - -The `PLACEMENT_RULES` table only shows explicitly configured rules. To see the canonical version of placement rules (including placement policies attached to tables), use the statement `SHOW PLACEMENT` instead: - -{{< copyable "sql" >}} - -```sql -CREATE TABLE t1 (a INT); -CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1"; -CREATE TABLE t2 (a INT) PLACEMENT POLICY=p1; -SHOW PLACEMENT; -- Includes t2. -SELECT * FROM information_schema.placement_rules; -- Does not include t2. -``` - -```sql -Query OK, 0 rows affected (0.09 sec) - -Query OK, 0 rows affected (0.11 sec) - -Query OK, 0 rows affected (0.08 sec) - -Query OK, 0 rows affected (0.11 sec) - -+---------------+------------------------------------------------+ -| Target | Placement | -+---------------+------------------------------------------------+ -| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | -| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | -+---------------+------------------------------------------------+ -2 rows in set (0.00 sec) - -+-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ -| POLICY_ID | CATALOG_NAME | POLICY_NAME | SCHEMA_NAME | TABLE_NAME | PARTITION_NAME | PRIMARY_REGION | REGIONS | CONSTRAINTS | LEADER_CONSTRAINTS | FOLLOWER_CONSTRAINTS | LEARNER_CONSTRAINTS | SCHEDULE | FOLLOWERS | LEARNERS | -+-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ -| 3 | def | p1 | NULL | NULL | NULL | us-east-1 | us-east-1 | | | | | | 0 | 0 | -+-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ -2 rows in set (0.00 sec) -``` diff --git a/information-schema/information-schema.md b/information-schema/information-schema.md index a886bb829c54a..803c5d9069cea 100644 --- a/information-schema/information-schema.md +++ b/information-schema/information-schema.md @@ -76,7 +76,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) | A summary of metrics extracted from Prometheus. | | `METRICS_SUMMARY_BY_LABEL` | See `METRICS_SUMMARY` table. | | [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) | Provides the PromQL definitions for tables in `METRICS_SCHEMA`. | -| [`PLACEMENT_RULES`](/information-schema/information-schema-placement-rules.md) | Provides information on all objects that have explicit placement rules assigned. | +| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | Provides information on all objects that have explicit placement rules assigned. | | [`SEQUENCES`](/information-schema/information-schema-sequences.md) | The TiDB implementation of sequences is based on MariaDB. | | [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) | Provides information on slow queries on the current TiDB server. | | [`STATEMENTS_SUMMARY`](/statement-summary-tables.md) | Similar to performance_schema statement summary in MySQL. | diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 6a966702ec4c6..23a747875da1a 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -25,7 +25,7 @@ The detailed user scenarios are as follows: ## Specify placement rules -To specify placement rules, first create a placement policy: +To specify placement rules, first create a placement policy by [`CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-create-placement-policy.md): ```sql CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; @@ -34,16 +34,28 @@ CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east-1" REGIONS="us Then attach the policy to a table or partition using either `CREATE TABLE` or `ALTER TABLE`. Then, the placement rules are specified on the table or the partition: ```sql -CREATE TABLE t1 (a INT) PLACEMENT POLICY myplacementpolicy; +CREATE TABLE t1 (a INT) PLACEMENT POLICY=myplacementpolicy; CREATE TABLE t2 (a INT); -ALTER TABLE t2 PLACEMENT POLICY myplacementpolicy; +ALTER TABLE t2 PLACEMENT POLICY=myplacementpolicy; ``` A placement policy is not associated with any database schema and has the global scope. Therefore, assigning a placement policy does not require any additional privileges over the `CREATE TABLE` privilege. +It is also possible to alter placement policies by [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md), and the changes will propagate to all objects with the corresponding policy. + +```sql +ALTER PLACEMENT POLICY myplacementpolicy FOLLOWERS=5; +``` + +Finally, you can use [`DROP PLACEMENT POLICY`](/sql-statements/sql-statement-drop-placement-policy.md) to drop policies that are not attached to any table or partition: + +```sql +DROP PLACEMENT POLICY myplacementpolicy; +``` + ## View current placement rules -If a table has placement rules attached, you can view the placement rules in the output of `SHOW CREATE TABLE`. To view the definition of the policy available, execute `SHOW CREATE PLACEMENT POLICY`: +If a table has placement rules attached, you can view the placement rules in the output of [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md). To view the definition of the policy available, execute [`SHOW CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-show-create-placement-policy.md): ```sql tidb> SHOW CREATE TABLE t1\G @@ -61,6 +73,26 @@ Create Policy: CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east 1 row in set (0.00 sec) ``` +It is also possible to view definition of placement policies through [`INFORMATION_SCHEMA.PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) table. + +```sql +tidb> select * from information_schema.placement_policies\G +***************************[ 1. row ]*************************** +POLICY_ID | 1 +CATALOG_NAME | def +POLICY_NAME | p1 +PRIMARY_REGION | us-east-1 +REGIONS | us-east-1,us-west-1 +CONSTRAINTS | +LEADER_CONSTRAINTS | +FOLLOWER_CONSTRAINTS | +LEARNER_CONSTRAINTS | +SCHEDULE | +FOLLOWERS | 4 +LEARNERS | 0 +1 row in set +``` + The `information_schema.tables` and `information_schema.partitions` tables also include a column for `tidb_placement_policy_name`, which shows all objects with placement rules attached: ```sql @@ -99,8 +131,12 @@ In addition to the placement options above, you can also use the advance configu | Option Name | Description | | --------------| ------------ | -| `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | -| `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | +| `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | +| `LEADER_CONSTRAINTS` | A list of constraints that only apply to leader. | +| `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | +| `LEARNER_CONSTRAINTS` | A list of constraints that only apply to learners. | +| `LEARNERS` | The number of learners. + | ## Examples @@ -135,6 +171,7 @@ To ensure that enough followers are placed in the primary region (`us-east-1`) s In addition to assigning placement options to tables, you can also assign the options to table partitions. For example: ```sql +CREATE PLACEMENT POLICY p1 FOLLOWERS=5; CREATE PLACEMENT POLICY europe PRIMARY_REGION="eu-central-1" REGIONS="eu-central-1,eu-west-1"; CREATE PLACEMENT POLICY northamerica PRIMARY_REGION="us-east-1" REGIONS="us-east-1"; @@ -142,12 +179,15 @@ SET tidb_enable_list_partition = 1; CREATE TABLE t1 ( country VARCHAR(10) NOT NULL, userdata VARCHAR(100) NOT NULL -) PARTITION BY LIST COLUMNS (country) ( +) PLACEMENT POLICY=p1 PARTITION BY LIST COLUMNS (country) ( PARTITION pEurope VALUES IN ('DE', 'FR', 'GB') PLACEMENT POLICY=europe, PARTITION pNorthAmerica VALUES IN ('US', 'CA', 'MX') PLACEMENT POLICY=northamerica + PARTITION pAsia VALUES IN ('CN', 'KR', 'JP') ); ``` +If partitions have no attached policies, it will try to apply possibly existed policy on the table. For example, `pEurope` will apply `europe` policy, but `pAsia` will apply policy `p1` from table `t1`. If `t1` has no assigned policies, `pAsia` will not apply any policy, too. + ### Set the default placement for a schema You can directly attach the default placement rules to a database schema. This works similar to setting the default character set or collation for a schema. Your specified placement options apply when no other options are specified. For example: @@ -174,7 +214,7 @@ CREATE TABLE t4 (a INT); -- Creates a table t4 with the default policy p3. ALTER PLACEMENT POLICY p3 FOLLOWERS=3; -- The table with policy p3 (t4) will have FOLLOWERS=3. ``` -You can use [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) to change a policy, and the changes will propagate to all objects with the corresponding policy. +Note that this is different from inheritance between partitions and tables, where changing policy of tables will effect their partitions. Tables only inherit the policy of schema when it is created without attaching policies. ### Advanced placement options @@ -212,7 +252,6 @@ In dictionary format, constraints also indicate a number of instances that apply The following known limitations exist in the experimental release of Placement Rules in SQL: -* Dumpling does not support dumping placement policies. See [issue #29371](https://github.com/pingcap/tidb/issues/29371). * TiDB tools, including Backup & Restore (BR), TiCDC, TiDB Lightning, and TiDB Data Migration (DM), do not yet support placement rules. * Temporary tables do not support placement options. * Syntactic sugar rules are permitted for setting `PRIMARY_REGION` and `REGIONS`. In the future, we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE`, and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030). diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index d6983ba3ea7c4..bf611feb1b3bb 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -11,15 +11,10 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. > > If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. -`ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions which use the placement policy will automatically be updated. +`ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions attached this placement policy will automatically be updated. `ALTER PLACEMENT POLICY` _replaces_ the previous policy with the new definition. It does not _merge_ the old policy with the new one. In the following example, `FOLLOWERS=4` is lost when the `ALTER PLACEMENT POLICY` is executed: -```sql -CREATE PLACEMENT POLICY p1 FOLLOWERS=4; -ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; -``` - ## Synopsis ```ebnf+diagram @@ -35,16 +30,23 @@ PlacementOptionList ::= | PlacementOptionList ',' PlacementOption PlacementOption ::= + CommonPlacementOption +| SugarPlacementOption +| AdvancedPlacementOption + +CommonPlacementOption ::= + "FOLLOWERS" EqOpt LengthNum + +SugarPlacementOption ::= "PRIMARY_REGION" EqOpt stringLit | "REGIONS" EqOpt stringLit -| "FOLLOWERS" EqOpt LengthNum -| "VOTERS" EqOpt LengthNum -| "LEARNERS" EqOpt LengthNum | "SCHEDULE" EqOpt stringLit + +AdvancedPlacementOption ::= + "LEARNERS" EqOpt LengthNum | "CONSTRAINTS" EqOpt stringLit | "LEADER_CONSTRAINTS" EqOpt stringLit | "FOLLOWER_CONSTRAINTS" EqOpt stringLit -| "VOTER_CONSTRAINTS" EqOpt stringLit | "LEARNER_CONSTRAINTS" EqOpt stringLit ``` @@ -60,8 +62,9 @@ PlacementOption ::= ```sql CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; -ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; -SHOW CREATE PLACEMENT POLICY p1\G +CREATE TABLE t1 (i INT) PLACEMENT POLICY=p1; -- attach policy p1 to table t1 +ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; -- rules of t1 will be updated automatically +SHOW CREATE PLACEMENT POLICY p1\G; ``` ``` @@ -69,9 +72,9 @@ Query OK, 0 rows affected (0.08 sec) Query OK, 0 rows affected (0.10 sec) -*************************** 1. row *************************** - Policy: p1 -Create Policy: CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4 +***************************[ 1. row ]*************************** +Policy | p1 +Create Policy | CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4 1 row in set (0.00 sec) ``` @@ -84,4 +87,4 @@ This statement is a TiDB extension to MySQL syntax. * [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) -* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file +* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index bd0344ab784e7..4a44c1189f426 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -28,16 +28,23 @@ PlacementOptionList ::= | PlacementOptionList ',' PlacementOption PlacementOption ::= + CommonPlacementOption +| SugarPlacementOption +| AdvancedPlacementOption + +CommonPlacementOption ::= + "FOLLOWERS" EqOpt LengthNum + +SugarPlacementOption ::= "PRIMARY_REGION" EqOpt stringLit | "REGIONS" EqOpt stringLit -| "FOLLOWERS" EqOpt LengthNum -| "VOTERS" EqOpt LengthNum -| "LEARNERS" EqOpt LengthNum | "SCHEDULE" EqOpt stringLit + +AdvancedPlacementOption ::= + "LEARNERS" EqOpt LengthNum | "CONSTRAINTS" EqOpt stringLit | "LEADER_CONSTRAINTS" EqOpt stringLit | "FOLLOWER_CONSTRAINTS" EqOpt stringLit -| "VOTER_CONSTRAINTS" EqOpt stringLit | "LEARNER_CONSTRAINTS" EqOpt stringLit ``` diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index 96c90202b66dc..2ab8151b8d22f 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -30,7 +30,7 @@ PolicyName ::= ```sql CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; CREATE TABLE t1 (a INT) PLACEMENT POLICY=p1; -SHOW CREATE PLACEMENT POLICY p1\G +SHOW CREATE PLACEMENT POLICY p1\G; ``` ``` @@ -38,9 +38,9 @@ Query OK, 0 rows affected (0.08 sec) Query OK, 0 rows affected (0.10 sec) -*************************** 1. row *************************** - Policy: p1 -Create Policy: CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 +***************************[ 1. row ]*************************** +Policy | p1 +Create Policy | CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 1 row in set (0.00 sec) ``` diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index 31b36461df439..758eabff86345 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -37,14 +37,13 @@ ShowPlacementTarget ::= ```sql CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; -use test; ALTER DATABASE test PLACEMENT POLICY=p1; CREATE TABLE t1 (a INT); SHOW PLACEMENT FOR DATABASE test; SHOW PLACEMENT FOR TABLE t1; -SHOW CREATE TABLE t1\G +SHOW CREATE TABLE t1\G; CREATE TABLE t3 (a INT) PARTITION BY RANGE (a) (PARTITION p1 VALUES LESS THAN (10), PARTITION p2 VALUES LESS THAN (20)); -SHOW PLACEMENT FOR TABLE t3 PARTITION p1; +SHOW PLACEMENT FOR TABLE t3 PARTITION p1\G; ``` ``` @@ -52,8 +51,6 @@ Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.00 sec) -Query OK, 0 rows affected (0.00 sec) - Query OK, 0 rows affected (0.01 sec) +---------------+----------------------------------------------------------------------+------------------+ @@ -70,18 +67,17 @@ Query OK, 0 rows affected (0.01 sec) +---------------+-------------+------------------+ 1 row in set (0.00 sec) -*************************** 1. row *************************** - Table: t1 -Create Table: CREATE TABLE `t1` ( +***************************[ 1. row ]*************************** +Table | t1 +Create Table | CREATE TABLE `t1` ( `a` int(11) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin /*T![placement] PLACEMENT POLICY=`p1` */ 1 row in set (0.00 sec) -+----------------------------+-----------------------------------------------------------------------+------------------+ -| Target | Placement | Scheduling_State | -+----------------------------+-----------------------------------------------------------------------+------------------+ -| TABLE test.t3 PARTITION p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,,us-west-1" FOLLOWERS=4 | INPROGRESS | -+----------------------------+-----------------------------------------------------------------------+------------------+ +***************************[ 1. row ]*************************** +Target | TABLE test.t3 PARTITION p1 +Placement | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 +Scheduling_State | PENDING 1 row in set (0.00 sec) ``` From 2d2b7bea9d9a491da457c43fa2691ffbf6898449 Mon Sep 17 00:00:00 2001 From: xhe Date: Fri, 18 Mar 2022 15:36:12 +0800 Subject: [PATCH 02/19] *: reword Signed-off-by: xhe --- information-schema/information-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/information-schema/information-schema.md b/information-schema/information-schema.md index 803c5d9069cea..c6b9050717451 100644 --- a/information-schema/information-schema.md +++ b/information-schema/information-schema.md @@ -76,7 +76,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) | A summary of metrics extracted from Prometheus. | | `METRICS_SUMMARY_BY_LABEL` | See `METRICS_SUMMARY` table. | | [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) | Provides the PromQL definitions for tables in `METRICS_SCHEMA`. | -| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | Provides information on all objects that have explicit placement rules assigned. | +| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | Provides all placement policies. | | [`SEQUENCES`](/information-schema/information-schema-sequences.md) | The TiDB implementation of sequences is based on MariaDB. | | [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) | Provides information on slow queries on the current TiDB server. | | [`STATEMENTS_SUMMARY`](/statement-summary-tables.md) | Similar to performance_schema statement summary in MySQL. | From 3ca9d533923fcec6542360c69af0d03bcf22b8ab Mon Sep 17 00:00:00 2001 From: xhe Date: Fri, 18 Mar 2022 16:14:04 +0800 Subject: [PATCH 03/19] *: improve words, update compatibility Signed-off-by: xhe --- placement-rules-in-sql.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 23a747875da1a..425d6ab148c3b 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -214,7 +214,7 @@ CREATE TABLE t4 (a INT); -- Creates a table t4 with the default policy p3. ALTER PLACEMENT POLICY p3 FOLLOWERS=3; -- The table with policy p3 (t4) will have FOLLOWERS=3. ``` -Note that this is different from inheritance between partitions and tables, where changing policy of tables will effect their partitions. Tables only inherit the policy of schema when it is created without attaching policies. +Note that this is different from inheritance between partitions and tables, where changing policy of tables will affect their partitions. Tables only inherit the policy of schema when it is created without attaching policies, and modify policies of schemas does not affect created tables. ### Advanced placement options @@ -252,7 +252,7 @@ In dictionary format, constraints also indicate a number of instances that apply The following known limitations exist in the experimental release of Placement Rules in SQL: -* TiDB tools, including Backup & Restore (BR), TiCDC, TiDB Lightning, and TiDB Data Migration (DM), do not yet support placement rules. +* TiDB tools, including TiDB Lightning, and TiDB Data Migration (DM), do not yet support placement rules. * Temporary tables do not support placement options. * Syntactic sugar rules are permitted for setting `PRIMARY_REGION` and `REGIONS`. In the future, we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE`, and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030). * TiFlash learners are not configurable through Placement Rules syntax. From 5ff6550143d96e8d0f04a05746809117d1cc01e3 Mon Sep 17 00:00:00 2001 From: xhe Date: Fri, 18 Mar 2022 16:27:36 +0800 Subject: [PATCH 04/19] address comments Signed-off-by: xhe --- information-schema/information-schema-placement-policies.md | 2 +- placement-rules-in-sql.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/information-schema/information-schema-placement-policies.md b/information-schema/information-schema-placement-policies.md index 2953fe206e1af..e6e13197f427e 100644 --- a/information-schema/information-schema-placement-policies.md +++ b/information-schema/information-schema-placement-policies.md @@ -59,7 +59,7 @@ Query OK, 0 rows affected (0.08 sec) | Target | Placement | Scheduling_State | +---------------+------------------------------------------------+------------------+ | POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | NULL | -| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | PENDING | +| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | SCHEDULED | +---------------+------------------------------------------------+------------------+ 2 rows in set (0.00 sec) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 425d6ab148c3b..22211b17596e0 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -252,7 +252,7 @@ In dictionary format, constraints also indicate a number of instances that apply The following known limitations exist in the experimental release of Placement Rules in SQL: -* TiDB tools, including TiDB Lightning, and TiDB Data Migration (DM), do not yet support placement rules. +* TiDB Lightning do not yet support placement rules. * Temporary tables do not support placement options. * Syntactic sugar rules are permitted for setting `PRIMARY_REGION` and `REGIONS`. In the future, we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE`, and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030). * TiFlash learners are not configurable through Placement Rules syntax. From c7ad1bf741dd4970e5c73f80f6558ea80cfc982e Mon Sep 17 00:00:00 2001 From: xhe Date: Fri, 18 Mar 2022 16:30:45 +0800 Subject: [PATCH 05/19] fix lint Signed-off-by: xhe --- placement-rules-in-sql.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 22211b17596e0..3ead5645b28de 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -9,7 +9,7 @@ summary: Learn how to schedule placement of tables and partitions using SQL stat > > Placement Rules in SQL is an experimental feature introduced in v5.3.0. The syntax might change before its GA, and there might also be bugs. If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. -> **Note：** +> **Note:** > > The implementation of *Placement Rules in SQL* relies on the *placement rules feature* of PD. For details, refer to [Configure Placement Rules](/configure-placement-rules.md). In the context of Placement Rules in SQL, *placement rules* might refer to *placement policies* attached to other objects, or to rules that are sent from TiDB to PD. @@ -100,7 +100,7 @@ SELECT * FROM information_schema.tables WHERE tidb_placement_policy_name IS NOT SELECT * FROM information_schema.partitions WHERE tidb_placement_policy_name IS NOT NULL; ``` -Rules that are attached to objects are applied _asynchronously_. To view the current scheduling progress of placement, use [`SHOW PLACEMENT`](/sql-statements/sql-statement-show-placement.md). +Rules that are attached to objects are applied *asynchronously*. To view the current scheduling progress of placement, use [`SHOW PLACEMENT`](/sql-statements/sql-statement-show-placement.md). ## Option reference @@ -132,11 +132,10 @@ In addition to the placement options above, you can also use the advance configu | Option Name | Description | | --------------| ------------ | | `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | -| `LEADER_CONSTRAINTS` | A list of constraints that only apply to leader. | -| `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | -| `LEARNER_CONSTRAINTS` | A list of constraints that only apply to learners. | -| `LEARNERS` | The number of learners. - | +| `LEADER_CONSTRAINTS` | A list of constraints that only apply to leader. | +| `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | +| `LEARNER_CONSTRAINTS` | A list of constraints that only apply to learners. | +| `LEARNERS` | The number of learners. | ## Examples From 77bf0503bcb67c59f139353cdf1888f9d9e04feb Mon Sep 17 00:00:00 2001 From: xhe Date: Fri, 18 Mar 2022 18:58:43 +0800 Subject: [PATCH 06/19] sync with cn doc Signed-off-by: xhe --- information-schema/information-schema-placement-policies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/information-schema/information-schema-placement-policies.md b/information-schema/information-schema-placement-policies.md index e6e13197f427e..c79df52dacf95 100644 --- a/information-schema/information-schema-placement-policies.md +++ b/information-schema/information-schema-placement-policies.md @@ -5,7 +5,7 @@ sgmmary: Learn the `PLACEMENT_POLICIES` information_schema table. # PLACEMENT_POLICIES -The `PLACEMENT_POLICIES` table provides information on all explicitly configured [Placement Rules in SQL](/placement-rules-in-sql.md), which means all placement policies. +The `PLACEMENT_POLICIES` table provides information about all placement policies, refer [Placement Rules in SQL](/placement-rules-in-sql.md). {{< copyable "sql" >}} From 838a4e337a6824d8968bb17b92d1a2a2074a0e6d Mon Sep 17 00:00:00 2001 From: xhe Date: Mon, 21 Mar 2022 14:13:12 +0800 Subject: [PATCH 07/19] sync with cn Signed-off-by: xhe --- .../information-schema-placement-policies.md | 12 ++++++------ placement-rules-in-sql.md | 4 +++- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/information-schema/information-schema-placement-policies.md b/information-schema/information-schema-placement-policies.md index c79df52dacf95..335782bdfc6d4 100644 --- a/information-schema/information-schema-placement-policies.md +++ b/information-schema/information-schema-placement-policies.md @@ -5,7 +5,7 @@ sgmmary: Learn the `PLACEMENT_POLICIES` information_schema table. # PLACEMENT_POLICIES -The `PLACEMENT_POLICIES` table provides information about all placement policies, refer [Placement Rules in SQL](/placement-rules-in-sql.md). +The `PLACEMENT_POLICIES` table provides information on all placement policies, refer [Placement Rules in SQL](/placement-rules-in-sql.md). {{< copyable "sql" >}} @@ -36,16 +36,16 @@ DESC placement_policies; ## Examples -The `PLACEMENT_POLICIES` table only shows explicitly configured rules. To see the canonical version of placement rules (including placement policies attached to tables), use the statement `SHOW PLACEMENT` instead: +The `PLACEMENT_POLICIES` table only shows all placement policies. To view the canonical version of placement rules (including all placement policies and objects assigned placement policies), use the statement `SHOW PLACEMENT` instead: {{< copyable "sql" >}} ```sql -CREATE TABLE t1 (a INT); +CREATE TABLE t1 (a INT); CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1"; CREATE TABLE t3 (a INT) PLACEMENT POLICY=p1; -SHOW PLACEMENT; -- Includes t3. -SELECT * FROM information_schema.placement_policies; -- Does not include t3. +SHOW PLACEMENT; -- Shows all information. Includes table t3. +SELECT * FROM information_schema.placement_policies; -- Only shows placement policies. Does not include t3. ``` ```sql @@ -59,7 +59,7 @@ Query OK, 0 rows affected (0.08 sec) | Target | Placement | Scheduling_State | +---------------+------------------------------------------------+------------------+ | POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | NULL | -| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | SCHEDULED | +| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | PENDING | +---------------+------------------------------------------------+------------------+ 2 rows in set (0.00 sec) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 3ead5645b28de..f6f8f1de05d4b 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -251,7 +251,9 @@ In dictionary format, constraints also indicate a number of instances that apply The following known limitations exist in the experimental release of Placement Rules in SQL: -* TiDB Lightning do not yet support placement rules. +* TiDB tools，including Backup & Restore (BR), TiDB Lightning, TiCDC and TiDB Binlog, does not support placement rules before TiDB 6.0. +* Since TiDB 6.0, Backup & Restore (BR) supports imports and exports of placement rules. +* Since TiDB 6.0, TiCDC and TiDB Binlog will work in clusters assigned placement rules, and will not synchronize rules to the downstream clusters. * Temporary tables do not support placement options. * Syntactic sugar rules are permitted for setting `PRIMARY_REGION` and `REGIONS`. In the future, we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE`, and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030). * TiFlash learners are not configurable through Placement Rules syntax. From fd947f357ce154f60da9f8d2c7be8095b076759b Mon Sep 17 00:00:00 2001 From: xhe Date: Mon, 21 Mar 2022 17:51:11 +0800 Subject: [PATCH 08/19] Update information-schema/information-schema-placement-policies.md Co-authored-by: Ran --- information-schema/information-schema-placement-policies.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/information-schema/information-schema-placement-policies.md b/information-schema/information-schema-placement-policies.md index 335782bdfc6d4..6cd30ecd10cee 100644 --- a/information-schema/information-schema-placement-policies.md +++ b/information-schema/information-schema-placement-policies.md @@ -1,6 +1,7 @@ --- title: PLACEMENT_POLICIES -sgmmary: Learn the `PLACEMENT_POLICIES` information_schema table. +summary: Learn the `PLACEMENT_POLICIES` information_schema table. +aliases: ['/tidb/dev/information-schema-placement-rules'] --- # PLACEMENT_POLICIES From 29816faf9190691c36134ad9b1f4e3b971f6e8cb Mon Sep 17 00:00:00 2001 From: xhe Date: Tue, 22 Mar 2022 15:39:23 +0800 Subject: [PATCH 09/19] variable update Signed-off-by: xhe --- system-variables.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/system-variables.md b/system-variables.md index b2b79b7befba0..05b8368910b38 100644 --- a/system-variables.md +++ b/system-variables.md @@ -294,13 +294,6 @@ This variable is an alias for `last_insert_id`. > > Unlike in MySQL, the `max_execution_time` system variable currently works on all kinds of statements in TiDB, not only restricted to the `SELECT` statement. The precision of the timeout value is roughly 100ms. This means the statement might not be terminated in accurate milliseconds as you specify. -### placement_checks New in v5.3.0 - -- Scope: SESSION | GLOBAL -- Default value: `ON` -- This variable controls whether DDL statements validate [Placement Rules in SQL](/placement-rules-in-sql.md). -- It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if placement rules are violated. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. - ### plugin_dir - Scope: INSTANCE @@ -1417,6 +1410,14 @@ explain select * from t where age=5; - Default value: `ON` - This variable controls whether to enable the [ANALYZE configuration persistence](/statistics.md#persist-analyze-configurations) feature. +### tidb_placement_mode New in v6.0.0 + +- Scope: SESSION | GLOBAL +- Default value: `STRICT` +- Possible values: `STRICT`, `IGNORE` +- This variable controls whether DDL statements ignores [Placement Rules in SQL](/placement-rules-in-sql.md). When the variable is `IGNORE`, all placement rule options will be ignored. +- It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if invalidate placement rules are assigned. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. + ### tidb_pprof_sql_cpu New in v4.0 - Scope: INSTANCE From f68dc3c101cc19fe84d389eee60511385e06d7da Mon Sep 17 00:00:00 2001 From: xhe Date: Tue, 22 Mar 2022 16:56:38 +0800 Subject: [PATCH 10/19] compatibility refine Signed-off-by: xhe --- br/backup-and-restore-tool.md | 2 ++ placement-rules-in-sql.md | 12 +++++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/br/backup-and-restore-tool.md b/br/backup-and-restore-tool.md index 4bf4480bcff23..5584630f889b6 100644 --- a/br/backup-and-restore-tool.md +++ b/br/backup-and-restore-tool.md @@ -132,6 +132,8 @@ The compatibility issues of BR and the TiDB cluster are divided into the followi + BR versions earlier than v5.4.0 do not support recovering `charset=GBK` tables. No version of BR supports recovering `charset=GBK` tables to TiDB clusters earlier than v5.4.0. + + BR does not support tables with [placement rules](/placement-rules-in-sql.md) before v6.0.0. Use BR v6.0.0 and above, with the `--with-tidb-placement-mode=strict/ignore` command line option to backup and restore. When the value is `strict`，BR will try to restore placement rules, otherwise will ignore. + + The KV format might change when some features are enabled or disabled. If these features are not consistently enabled or disabled during backup and restore, compatibility issues might occur. These features are as follows: diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index f6f8f1de05d4b..93243ba585b8b 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -247,13 +247,19 @@ In dictionary format, constraints also indicate a number of instances that apply > > Dictionary and list formats are based on the YAML parser, but the YAML syntax might be incorrectly parsed. For example, `"{+disk=ssd:1,+disk=hdd:2}"` is incorrectly parsed as `'{"+disk=ssd:1": null, "+disk=hdd:1": null}'`. But `"{+disk=ssd: 1,+disk=hdd: 1}"` is correctly parsed as `'{"+disk=ssd": 1, "+disk=hdd": 1}'`. +## Tools Compatibility + +Tool Name | Minimum supported version | Description +--- | --- | --- +Backup & Restore (BR) | 6.0 | Supports imports and exports of placement rules，refer [BR Compatibility](/br/backup-and-restore-tool.md#Compatibility] +TiDB Lightning | Not support yet | +TiCDC | 6.0 | Ignore placement rules +TiDB Binlog | 6.0 | Ignore placement rules, and will not synchronize to the downstream + ## Known limitations The following known limitations exist in the experimental release of Placement Rules in SQL: -* TiDB tools，including Backup & Restore (BR), TiDB Lightning, TiCDC and TiDB Binlog, does not support placement rules before TiDB 6.0. -* Since TiDB 6.0, Backup & Restore (BR) supports imports and exports of placement rules. -* Since TiDB 6.0, TiCDC and TiDB Binlog will work in clusters assigned placement rules, and will not synchronize rules to the downstream clusters. * Temporary tables do not support placement options. * Syntactic sugar rules are permitted for setting `PRIMARY_REGION` and `REGIONS`. In the future, we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE`, and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030). * TiFlash learners are not configurable through Placement Rules syntax. From a96eabdcbf2d1a9547d2de293cde4490c61040f4 Mon Sep 17 00:00:00 2001 From: xhe Date: Tue, 22 Mar 2022 17:13:43 +0800 Subject: [PATCH 11/19] fix CI Signed-off-by: xhe --- placement-rules-in-sql.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 93243ba585b8b..f36be932c00b3 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -249,12 +249,12 @@ In dictionary format, constraints also indicate a number of instances that apply ## Tools Compatibility -Tool Name | Minimum supported version | Description ---- | --- | --- -Backup & Restore (BR) | 6.0 | Supports imports and exports of placement rules，refer [BR Compatibility](/br/backup-and-restore-tool.md#Compatibility] -TiDB Lightning | Not support yet | -TiCDC | 6.0 | Ignore placement rules -TiDB Binlog | 6.0 | Ignore placement rules, and will not synchronize to the downstream +| Tool Name | Minimum supported version | Description | +| --- | --- | --- | +| Backup & Restore (BR) | 6.0 | Supports imports and exports of placement rules，refer [BR Compatibility](/br/backup-and-restore-tool.md#Compatibility] | +| TiDB Lightning | Not compatible yet | | +| TiCDC | 6.0 | Ignore placement rules | +| TiDB Binlog | 6.0 | Ignore placement rules, and will not synchronize to the downstream | ## Known limitations From 44001ab3cf3a07babc84b0200630e0bcc9da7ad0 Mon Sep 17 00:00:00 2001 From: xhe Date: Tue, 22 Mar 2022 17:25:08 +0800 Subject: [PATCH 12/19] sync with cn Signed-off-by: xhe --- br/backup-and-restore-tool.md | 2 +- placement-rules-in-sql.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/br/backup-and-restore-tool.md b/br/backup-and-restore-tool.md index 5584630f889b6..25604cd545ca6 100644 --- a/br/backup-and-restore-tool.md +++ b/br/backup-and-restore-tool.md @@ -132,7 +132,7 @@ The compatibility issues of BR and the TiDB cluster are divided into the followi + BR versions earlier than v5.4.0 do not support recovering `charset=GBK` tables. No version of BR supports recovering `charset=GBK` tables to TiDB clusters earlier than v5.4.0. - + BR does not support tables with [placement rules](/placement-rules-in-sql.md) before v6.0.0. Use BR v6.0.0 and above, with the `--with-tidb-placement-mode=strict/ignore` command line option to backup and restore. When the value is `strict`，BR will try to restore placement rules, otherwise will ignore. + + BR does not support [placement rules](/placement-rules-in-sql.md) before v6.0.0. BR v6.0.0 and above support and introduce command lime option `--with-tidb-placement-mode=strict/ignore` to backup and restore. The default value `strict` will import and validate placement rules, but will ignore all placement rules when the value is `ignore`. + The KV format might change when some features are enabled or disabled. If these features are not consistently enabled or disabled during backup and restore, compatibility issues might occur. diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index f36be932c00b3..cf689ab4f93cd 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -253,7 +253,7 @@ In dictionary format, constraints also indicate a number of instances that apply | --- | --- | --- | | Backup & Restore (BR) | 6.0 | Supports imports and exports of placement rules，refer [BR Compatibility](/br/backup-and-restore-tool.md#Compatibility] | | TiDB Lightning | Not compatible yet | | -| TiCDC | 6.0 | Ignore placement rules | +| TiCDC | 6.0 | Ignore placement rules, and will not synchronize to the downstream | | TiDB Binlog | 6.0 | Ignore placement rules, and will not synchronize to the downstream | ## Known limitations From 54ba1b180c939d107e92d09df2f20407d127e741 Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 10:42:14 +0800 Subject: [PATCH 13/19] remove enable_alter Signed-off-by: xhe --- system-variables.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/system-variables.md b/system-variables.md index 05b8368910b38..090924bb5a715 100644 --- a/system-variables.md +++ b/system-variables.md @@ -652,16 +652,6 @@ Constraint checking is always performed in place for pessimistic transactions (d > - If you have enabled TiDB Binlog, enabling this variable cannot improve the performance. To improve the performance, it is recommended to use [TiCDC](/ticdc/ticdc-overview.md) instead. > - Enabling this parameter only means that one-phase commit becomes an optional mode of transaction commit. In fact, the most suitable mode of transaction commit is determined by TiDB. -### tidb_enable_alter_placement - -> **Warning:** -> -> Currently, Placement Rules in SQL is an experimental feature. It is not recommended that you use it in production environments. - -- Scope: GLOBAL -- Default value: `OFF` -- This variable enables or disables [Placement Rules in SQL](/placement-rules-in-sql.md). - ### tidb_enable_amend_pessimistic_txn New in v4.0.7 - Scope: SESSION | GLOBAL From 275079fc97b08fa4738d96393951e34afed4d70d Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 15:25:15 +0800 Subject: [PATCH 14/19] Update system-variables.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index 090924bb5a715..576f916c0a8e6 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1406,7 +1406,7 @@ explain select * from t where age=5; - Default value: `STRICT` - Possible values: `STRICT`, `IGNORE` - This variable controls whether DDL statements ignores [Placement Rules in SQL](/placement-rules-in-sql.md). When the variable is `IGNORE`, all placement rule options will be ignored. -- It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if invalidate placement rules are assigned. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. +- It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if invalid placement rules are assigned. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. ### tidb_pprof_sql_cpu New in v4.0 From b7a9a7c097e0d6738143907145548104976b96c8 Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 15:27:12 +0800 Subject: [PATCH 15/19] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- br/backup-and-restore-tool.md | 2 +- .../information-schema-placement-policies.md | 6 +++--- information-schema/information-schema.md | 2 +- placement-rules-in-sql.md | 20 +++++++++---------- .../sql-statement-alter-placement-policy.md | 2 +- 5 files changed, 16 insertions(+), 16 deletions(-) diff --git a/br/backup-and-restore-tool.md b/br/backup-and-restore-tool.md index 25604cd545ca6..4ec4e973f1d67 100644 --- a/br/backup-and-restore-tool.md +++ b/br/backup-and-restore-tool.md @@ -132,7 +132,7 @@ The compatibility issues of BR and the TiDB cluster are divided into the followi + BR versions earlier than v5.4.0 do not support recovering `charset=GBK` tables. No version of BR supports recovering `charset=GBK` tables to TiDB clusters earlier than v5.4.0. - + BR does not support [placement rules](/placement-rules-in-sql.md) before v6.0.0. BR v6.0.0 and above support and introduce command lime option `--with-tidb-placement-mode=strict/ignore` to backup and restore. The default value `strict` will import and validate placement rules, but will ignore all placement rules when the value is `ignore`. + + BR does not support [placement rules](/placement-rules-in-sql.md) before v6.0.0. Since v6.0.0, BR supports placement rules and introduces a command-line option `--with-tidb-placement-mode=strict/ignore` to control the backup and restore mode of placement rules. With the default value `strict`, BR imports and validates placement rules, but ignores all placement rules when the value is `ignore`. + The KV format might change when some features are enabled or disabled. If these features are not consistently enabled or disabled during backup and restore, compatibility issues might occur. diff --git a/information-schema/information-schema-placement-policies.md b/information-schema/information-schema-placement-policies.md index 6cd30ecd10cee..19aae3f6b0c90 100644 --- a/information-schema/information-schema-placement-policies.md +++ b/information-schema/information-schema-placement-policies.md @@ -6,7 +6,7 @@ aliases: ['/tidb/dev/information-schema-placement-rules'] # PLACEMENT_POLICIES -The `PLACEMENT_POLICIES` table provides information on all placement policies, refer [Placement Rules in SQL](/placement-rules-in-sql.md). +The `PLACEMENT_POLICIES` table provides information on all placement policies. For details, refer to [Placement Rules in SQL](/placement-rules-in-sql.md). {{< copyable "sql" >}} @@ -45,8 +45,8 @@ The `PLACEMENT_POLICIES` table only shows all placement policies. To view the ca CREATE TABLE t1 (a INT); CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1"; CREATE TABLE t3 (a INT) PLACEMENT POLICY=p1; -SHOW PLACEMENT; -- Shows all information. Includes table t3. -SELECT * FROM information_schema.placement_policies; -- Only shows placement policies. Does not include t3. +SHOW PLACEMENT; -- Shows all information, including table t3. +SELECT * FROM information_schema.placement_policies; -- Only shows placement policies, excluding t3. ``` ```sql diff --git a/information-schema/information-schema.md b/information-schema/information-schema.md index c6b9050717451..f36baae720f6c 100644 --- a/information-schema/information-schema.md +++ b/information-schema/information-schema.md @@ -76,7 +76,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) | A summary of metrics extracted from Prometheus. | | `METRICS_SUMMARY_BY_LABEL` | See `METRICS_SUMMARY` table. | | [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) | Provides the PromQL definitions for tables in `METRICS_SCHEMA`. | -| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | Provides all placement policies. | +| [`PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) | Provides information on all placement policies. | | [`SEQUENCES`](/information-schema/information-schema-sequences.md) | The TiDB implementation of sequences is based on MariaDB. | | [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) | Provides information on slow queries on the current TiDB server. | | [`STATEMENTS_SUMMARY`](/statement-summary-tables.md) | Similar to performance_schema statement summary in MySQL. | diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index cf689ab4f93cd..f891b0c8f6067 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -25,7 +25,7 @@ The detailed user scenarios are as follows: ## Specify placement rules -To specify placement rules, first create a placement policy by [`CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-create-placement-policy.md): +To specify placement rules, first create a placement policy using [`CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-create-placement-policy.md): ```sql CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; @@ -41,13 +41,13 @@ ALTER TABLE t2 PLACEMENT POLICY=myplacementpolicy; A placement policy is not associated with any database schema and has the global scope. Therefore, assigning a placement policy does not require any additional privileges over the `CREATE TABLE` privilege. -It is also possible to alter placement policies by [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md), and the changes will propagate to all objects with the corresponding policy. +To modify a placement policy, you can use [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md), and the changes will propagate to all objects assigned with the corresponding policy. ```sql ALTER PLACEMENT POLICY myplacementpolicy FOLLOWERS=5; ``` -Finally, you can use [`DROP PLACEMENT POLICY`](/sql-statements/sql-statement-drop-placement-policy.md) to drop policies that are not attached to any table or partition: +To drop policies that are not attached to any table or partition, you can use [`DROP PLACEMENT POLICY`](/sql-statements/sql-statement-drop-placement-policy.md): ```sql DROP PLACEMENT POLICY myplacementpolicy; @@ -73,7 +73,7 @@ Create Policy: CREATE PLACEMENT POLICY myplacementpolicy PRIMARY_REGION="us-east 1 row in set (0.00 sec) ``` -It is also possible to view definition of placement policies through [`INFORMATION_SCHEMA.PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) table. +You can also view definitions of placement policies using the [`INFORMATION_SCHEMA.PLACEMENT_POLICIES`](/information-schema/information-schema-placement-policies.md) table. ```sql tidb> select * from information_schema.placement_policies\G @@ -185,7 +185,7 @@ CREATE TABLE t1 ( ); ``` -If partitions have no attached policies, it will try to apply possibly existed policy on the table. For example, `pEurope` will apply `europe` policy, but `pAsia` will apply policy `p1` from table `t1`. If `t1` has no assigned policies, `pAsia` will not apply any policy, too. +If a partition has no attached policies, it tries to apply possibly existing policies on the table. For example, the `pEurope` partition will apply the `europe` policy, but the `pAsia` partition will apply the `p1` policy from table `t1`. If `t1` has no assigned policies, `pAsia` will not apply any policy, too. ### Set the default placement for a schema @@ -213,7 +213,7 @@ CREATE TABLE t4 (a INT); -- Creates a table t4 with the default policy p3. ALTER PLACEMENT POLICY p3 FOLLOWERS=3; -- The table with policy p3 (t4) will have FOLLOWERS=3. ``` -Note that this is different from inheritance between partitions and tables, where changing policy of tables will affect their partitions. Tables only inherit the policy of schema when it is created without attaching policies, and modify policies of schemas does not affect created tables. +Note that this is different from the inheritance between partitions and tables, where changing the policy of tables will affect their partitions. Tables inherit the policy of schema only when they are created without policies attached, and modifying the policies of schemas does not affect created tables. ### Advanced placement options @@ -247,14 +247,14 @@ In dictionary format, constraints also indicate a number of instances that apply > > Dictionary and list formats are based on the YAML parser, but the YAML syntax might be incorrectly parsed. For example, `"{+disk=ssd:1,+disk=hdd:2}"` is incorrectly parsed as `'{"+disk=ssd:1": null, "+disk=hdd:1": null}'`. But `"{+disk=ssd: 1,+disk=hdd: 1}"` is correctly parsed as `'{"+disk=ssd": 1, "+disk=hdd": 1}'`. -## Tools Compatibility +## Compatibility with tools | Tool Name | Minimum supported version | Description | | --- | --- | --- | -| Backup & Restore (BR) | 6.0 | Supports imports and exports of placement rules，refer [BR Compatibility](/br/backup-and-restore-tool.md#Compatibility] | +| Backup & Restore (BR) | 6.0 | Supports importing and exporting placement rules. Refer to [BR Compatibility](/br/backup-and-restore-tool.md#compatibility] for details. | | TiDB Lightning | Not compatible yet | | -| TiCDC | 6.0 | Ignore placement rules, and will not synchronize to the downstream | -| TiDB Binlog | 6.0 | Ignore placement rules, and will not synchronize to the downstream | +| TiCDC | 6.0 | Ignores placement rules, and does not replicate the rules to the downstream | +| TiDB Binlog | 6.0 | Ignores placement rules, and does not replicate the rules to the downstream | ## Known limitations diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index bf611feb1b3bb..83aebed2ab009 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -11,7 +11,7 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. > > If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. -`ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions attached this placement policy will automatically be updated. +`ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions attached with this placement policy will automatically be updated. `ALTER PLACEMENT POLICY` _replaces_ the previous policy with the new definition. It does not _merge_ the old policy with the new one. In the following example, `FOLLOWERS=4` is lost when the `ALTER PLACEMENT POLICY` is executed: From 327dbbc9d2cf0b08f15f0e1adf783b3128edb969 Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 15:28:48 +0800 Subject: [PATCH 16/19] Update system-variables.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index 576f916c0a8e6..6ab16da5d1ced 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1405,7 +1405,7 @@ explain select * from t where age=5; - Scope: SESSION | GLOBAL - Default value: `STRICT` - Possible values: `STRICT`, `IGNORE` -- This variable controls whether DDL statements ignores [Placement Rules in SQL](/placement-rules-in-sql.md). When the variable is `IGNORE`, all placement rule options will be ignored. +- This variable controls whether DDL statements ignore the [placement rules specified in SQL](/placement-rules-in-sql.md). When the variable value is `IGNORE`, all placement rule options are ignored. - It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if invalid placement rules are assigned. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. ### tidb_pprof_sql_cpu New in v4.0 From 1958aae12f178de2da8acd88ca09efc34faaad3b Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 15:28:54 +0800 Subject: [PATCH 17/19] Update sql-statements/sql-statement-alter-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-alter-placement-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index 83aebed2ab009..81e558ea39cd5 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -63,7 +63,7 @@ AdvancedPlacementOption ::= ```sql CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; CREATE TABLE t1 (i INT) PLACEMENT POLICY=p1; -- attach policy p1 to table t1 -ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; -- rules of t1 will be updated automatically +ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; -- The rules of t1 will be updated automatically. SHOW CREATE PLACEMENT POLICY p1\G; ``` From 9980a6b03350314f493fe742a135faeaa4649e59 Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 15:29:11 +0800 Subject: [PATCH 18/19] Update system-variables.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index 6ab16da5d1ced..f55fda04cb28c 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1400,7 +1400,7 @@ explain select * from t where age=5; - Default value: `ON` - This variable controls whether to enable the [ANALYZE configuration persistence](/statistics.md#persist-analyze-configurations) feature. -### tidb_placement_mode New in v6.0.0 +### tidb_placement_mode (New in v6.0.0) - Scope: SESSION | GLOBAL - Default value: `STRICT` From 8b7d130c706fb2f21e68723b5b56bb005c69372e Mon Sep 17 00:00:00 2001 From: xhe Date: Wed, 23 Mar 2022 15:31:08 +0800 Subject: [PATCH 19/19] reword Signed-off-by: xhe --- sql-statements/sql-statement-alter-placement-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index 81e558ea39cd5..e27bc872a6f20 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -62,7 +62,7 @@ AdvancedPlacementOption ::= ```sql CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; -CREATE TABLE t1 (i INT) PLACEMENT POLICY=p1; -- attach policy p1 to table t1 +CREATE TABLE t1 (i INT) PLACEMENT POLICY=p1; -- Assign policy p1 to table t1 ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; -- The rules of t1 will be updated automatically. SHOW CREATE PLACEMENT POLICY p1\G; ```