`RESTORE` and `IMPORT INTO` multi-region functionality docs #11966

kathancox · 2021-10-14T13:43:16Z

Closes #10937 Draft of features for RESTORE & IMPORT INTO 21.2 multi-region features:

RESTORE doc changes:

Added section to the RESTORE page on restoring with the different table localities and their nuances. The RBT & RBR ordering/matching limitations are outlined here and on the known limitations page for 21.2. (Closes backupccl: allow restoring into multi-region databases #11283)
Added a bullet on the default primary region setting (Closes backupccl: respect sql.defaults.primary_region during RESTORE #11373 )
Note on SET LOCALITY page
Added the skip-localities-check option to the option table and this section (Closes backupccl: check mismatched cluster regions on backup and restore #11278)

IMPORT INTO doc changes:

Removed the section on this from the known limitation page (Closes IMPORT INTO regional by row tables is supported. #11919 )
Removed the consideration bullet and added that RBR in import into is now available
Added a bullet that import into now supports in computed columns default expressions with user-defined types (Closes release-21.2: importccl: add IMPORT INTO UDT support for default+computed columns #11720 )

cockroach-teamcity · 2021-10-14T13:43:23Z

This change is

github-actions · 2021-10-14T13:43:49Z

Files changed:

_includes/v21.2/backups/no-multiregion-table-backups.md
_includes/v21.2/known-limitations/rbr-restore-no-support.md
_includes/v21.2/known-limitations/restore-multiregion-match.md
_includes/v21.2/known-limitations/restore-tables-non-multi-reg.md
v21.2/import-into.md
v21.2/import.md
v21.2/known-limitations.md
v21.2/restore.md
v21.2/set-locality.md

netlify · 2021-10-14T14:41:49Z

✔️ Netlify Preview

🔨 Explore the source changes: 8a6fc16

🔍 Inspect the deploy log: https://app.netlify.com/sites/cockroachdb-docs/deploys/6182e8b6ecc70d0007562e16

😎 Browse the preview: https://deploy-preview-11966--cockroachdb-docs.netlify.app

kathancox · 2021-10-14T15:07:39Z

@livlobo I think Paul and Ellie worked on quite a few of the PRs

kathancox · 2021-10-28T16:03:48Z

@livlobo @dt @adityamaru Liv has just opened #12108 the changes in which will be live in a patch release. Can we prioritize reviewing this PR for the GA release in a couple of weeks and then the enhancements to MR backup and table restores I can start work on — once this is merged — ready for the patch release.

livlobo · 2021-10-29T21:22:16Z

v21.2/known-limitations.md

-    ~~~ sql
-    EXPORT INTO CSV 'nodelocal://0/src_rbr' FROM SELECT crdb_region, i from src_rbr;
-    ~~~
+### Using `RESTORE` with multi-region table localities


We should also add that restoring global/RBT tables into a non-MR DB is not supported + tracking issue: backupccl: allow restore of GLOBAL/RBT tables in a non-MR database #71502

And in the Backup limitations page of MR -- we should add that: Table level backups are NOT supported + tracking issue: backupccl: support for multi-region table backups #71180

livlobo

LGTM, Kathryn - subject to update to the 2 additional known limitations.

adityamaru · 2021-11-03T13:02:24Z

_includes/v21.2/known-limitations/rbr-restore-no-support.md

@@ -0,0 +1 @@
+Restoring [`REGIONAL BY ROW`](multiregion-overview.html#regional-by-row-tables) tables is currently not supported. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/pull/71178)


Should the issue be this cockroachdb/cockroach#67269?

rmloveland

LGTM, with some opinions about a few things

rmloveland · 2021-11-03T18:04:53Z

_includes/v21.2/known-limitations/restore-multiregion-match.md

+
+    The following must also be true for `RESTORE` to be successful:
+
+    * The regions of the source database and the regions of the destination database have the same set of regions.


First mention of 'regions' could link to our docs on database regions: https://www.cockroachlabs.com/docs/dev/multiregion-overview.html#database-regions

rmloveland · 2021-11-03T18:05:38Z

_includes/v21.2/known-limitations/restore-multiregion-match.md

+    The following must also be true for `RESTORE` to be successful:
+
+    * The regions of the source database and the regions of the destination database have the same set of regions.
+    * The regions were added to each of the databases in the same order.


interesting .. .this is gonna be a tough one for many users to know, I think

rmloveland · 2021-11-03T18:06:25Z

_includes/v21.2/known-limitations/restore-multiregion-match.md

+
+    * The regions of the source database and the regions of the destination database have the same set of regions.
+    * The regions were added to each of the databases in the same order.
+    * The databases have the same primary region.


'primary region' could link to https://www.cockroachlabs.com/docs/v21.2/set-primary-region

rmloveland · 2021-11-03T18:08:42Z

_includes/v21.2/known-limitations/restore-multiregion-match.md

+    > ALTER DATABASE destination_database ADD region "us-east1";  
+    ~~~
+
+    Furthermore, this scenario has mismatched regions between the databases since the regions were not added to the database in the same order.


I would find it clearer if this said something like "In addition, the following scenario ..." for the following reasons:

"furthermore" makes it sound like this is a continuation of the previous example

"this" is somewhat ambiguous because it makes me think it could be referring to the previous example

rmloveland · 2021-11-03T18:36:13Z

v21.2/import.md

@@ -22,7 +22,7 @@ The `IMPORT` [statement](sql-statements.html) imports the following types of dat
 - `IMPORT` cannot be used with [user-defined types](create-type.html). Use [`IMPORT INTO`](import-into.html) instead.
 - `IMPORT` is a blocking statement. To run an import job asynchronously, use the [`DETACHED`](#options-detached) option.
 - `IMPORT` cannot be used within a [rolling upgrade](upgrade-cockroach-version.html).
- {% include {{page.version.version}}/sql/import-into-regional-by-row-table.md %}
+- `IMPORT` cannot directly import data to `REGIONAL BY ROW` tables that are part of [multi-region databases](multiregion-overview.html). Instead, use [`IMPORT INTO`](import-into.html) which now supports importing into `REGIONAL BY ROW` tables.


suggest removing the already implied "now" (it's always now on the internet)

if you want to call out the timeliness aspect of this being a new thing in v21.2, maybe an edit like

"IMPORT cannot directly .... new in v21.2: Instead, use IMPORT INTO which supports importing into REGIONAL BY ROW tables"

also, FWIW as an outsider I find the IMPORT vs IMPORT INTO distinction quite confusing - esp if feature sets are different. nothing to do from the POV of this PR however ....

rmloveland · 2021-11-03T18:45:30Z

v21.2/restore.md

+
+* A cluster's regions will be checked before a restore. Mismatched regions between [backup](known-limitations.html#backup-of-multi-region-tables) and restore clusters will be flagged before the restore begins, which allows for a decision between updating the [cluster localities](cockroach-start.html#locality) or restoring with the [`skip_localities_check`](#skip-localities-check) option to continue with the restore regardless.
+
+* A database that is restored with the `sql.defaults.primary_region` [cluster setting](cluster-settings.html) will have the `PRIMARY REGION` from this cluster setting assigned to the target database.


'PRIMARY REGION' could link to https://www.cockroachlabs.com/docs/v21.2/set-primary-region

rmloveland · 2021-11-03T18:47:39Z

v21.2/restore.md

+
+* `RESTORE` supports restoring **non**-multi-region tables into a multi-region database and sets the table locality as [`REGIONAL BY TABLE`](multiregion-overview.html#regional-tables) to the primary region of the target database.
+
+* Restoring tables from multi-region databases with table localities set to `REGIONAL BY TABLE`, [`REGIONAL BY TABLE IN PRIMARY REGION`](set-locality.html#regional-by-table), and [`GLOBAL`](set-locality.html#global) to another multi-region database is supported. Note that when restoring a `REGIONAL BY TABLE IN PRIMARY REGION` table, if the primary region is different in the source database to the target database this will be implicitly changed on restore.


The "Note that ..." caveat seems important and easy to miss, may be worth breaking into a nested bullet under this one for the Buzzfeed listicle scanners like me

rmloveland · 2021-11-03T18:48:52Z

v21.2/restore.md

+The ordering of regions and how region matching is determined is a known limitation. See the [Known Limitations](#known-limitations) section for the tracking issues on limitations around `RESTORE` and multi-region support.
+
+For more on multi-region databases, see the [Multi-Region Capabilities Overview](multiregion-overview.html).
+


PS these docs are awesome (in case I forgot to say earlier)

rmloveland · 2021-11-03T18:50:24Z

v21.2/restore.md

+## Known limitations
+
+* {% include {{ page.version.version }}/known-limitations/restore-aost.md %} [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/53044)
+* To successfully [restore a table into a multi-region database](#restoring-to-multi-region-databases), it is necessary for the order and regions to match between the source and destination database. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/71071)


I think this one could benefit from linking to your more full explanation on the Known Limitations page. If I hadn't read that first I would not know what "the order ... to match" means

rmloveland · 2021-11-03T18:52:34Z

v21.2/set-locality.md

@@ -36,6 +36,10 @@ The user must be a member of the [`admin`](authorization.html#roles) or [owner](

 ## Examples

+{{site.data.alerts.callout_info}}
+[`RESTORE`](restore.html) on [`REGIONAL BY TABLE`](#regional-by-table) and [`GLOBAL`](#global) tables is supported with some limitations — see [Restoring to multi-region databases](restore.html#restoring-to-multi-region-databases) for more detail. Tables set to a [`REGIONAL BY ROW`](#regional-by-row) table locality cannot be restored.


I think it's worth breaking these sentences with a newline between them since one thing is saying "you can" and one is saying "you cannot"

Also think it's worth adding a mention and link to the full Known Limitations page section from the "Tables set to RBR cannot be restored" part

kathancox · 2021-11-03T20:32:33Z

TFTR @rmloveland !

kathancox requested review from livlobo and adityamaru October 14, 2021 15:04

kathancox requested a review from dt October 19, 2021 20:17

livlobo reviewed Oct 29, 2021

View reviewed changes

livlobo approved these changes Oct 29, 2021

View reviewed changes

kathancox force-pushed the multi-region-bulk-features branch from 88e1538 to 3b3f018 Compare November 1, 2021 16:04

kathancox removed the request for review from dt November 3, 2021 12:59

adityamaru approved these changes Nov 3, 2021

View reviewed changes

Draft of RESTORE and IMPORT multi-region docs

8a307b0

kathancox force-pushed the multi-region-bulk-features branch from 3b3f018 to 8a307b0 Compare November 3, 2021 14:30

kathancox requested a review from rmloveland November 3, 2021 14:39

rmloveland approved these changes Nov 3, 2021

View reviewed changes

Rich's feedback incorporated

8a6fc16

kathancox merged commit f2780e4 into master Nov 3, 2021

kathancox deleted the multi-region-bulk-features branch November 3, 2021 20:19

exalate-issue-sync bot mentioned this pull request Jan 5, 2022

Add docs on restoring multi-region databases #9932

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`RESTORE` and `IMPORT INTO` multi-region functionality docs #11966

`RESTORE` and `IMPORT INTO` multi-region functionality docs #11966

kathancox commented Oct 14, 2021

cockroach-teamcity commented Oct 14, 2021

github-actions bot commented Oct 14, 2021 •

edited

Loading

netlify bot commented Oct 14, 2021 •

edited

Loading

kathancox commented Oct 14, 2021

kathancox commented Oct 28, 2021

livlobo Oct 29, 2021

livlobo left a comment

adityamaru Nov 3, 2021

rmloveland left a comment

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

rmloveland Nov 3, 2021

kathancox commented Nov 3, 2021

		@@ -0,0 +1 @@
		Restoring [`REGIONAL BY ROW`](multiregion-overview.html#regional-by-row-tables) tables is currently not supported. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/pull/71178)


		The following must also be true for `RESTORE` to be successful:

		* The regions of the source database and the regions of the destination database have the same set of regions.


		* A cluster's regions will be checked before a restore. Mismatched regions between [backup](known-limitations.html#backup-of-multi-region-tables) and restore clusters will be flagged before the restore begins, which allows for a decision between updating the [cluster localities](cockroach-start.html#locality) or restoring with the [`skip_localities_check`](#skip-localities-check) option to continue with the restore regardless.

		* A database that is restored with the `sql.defaults.primary_region` [cluster setting](cluster-settings.html) will have the `PRIMARY REGION` from this cluster setting assigned to the target database.


		* `RESTORE` supports restoring non-multi-region tables into a multi-region database and sets the table locality as [`REGIONAL BY TABLE`](multiregion-overview.html#regional-tables) to the primary region of the target database.

		* Restoring tables from multi-region databases with table localities set to `REGIONAL BY TABLE`, [`REGIONAL BY TABLE IN PRIMARY REGION`](set-locality.html#regional-by-table), and [`GLOBAL`](set-locality.html#global) to another multi-region database is supported. Note that when restoring a `REGIONAL BY TABLE IN PRIMARY REGION` table, if the primary region is different in the source database to the target database this will be implicitly changed on restore.

		The ordering of regions and how region matching is determined is a known limitation. See the [Known Limitations](#known-limitations) section for the tracking issues on limitations around `RESTORE` and multi-region support.

		For more on multi-region databases, see the [Multi-Region Capabilities Overview](multiregion-overview.html).

RESTORE and IMPORT INTO multi-region functionality docs #11966

RESTORE and IMPORT INTO multi-region functionality docs #11966

Conversation

kathancox commented Oct 14, 2021

cockroach-teamcity commented Oct 14, 2021

github-actions bot commented Oct 14, 2021 • edited Loading

netlify bot commented Oct 14, 2021 • edited Loading

kathancox commented Oct 14, 2021

kathancox commented Oct 28, 2021

Choose a reason for hiding this comment

livlobo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rmloveland left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kathancox commented Nov 3, 2021

`RESTORE` and `IMPORT INTO` multi-region functionality docs #11966

`RESTORE` and `IMPORT INTO` multi-region functionality docs #11966

github-actions bot commented Oct 14, 2021 •

edited

Loading

netlify bot commented Oct 14, 2021 •

edited

Loading