Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docdb] [YCQL] Restore should preserve the exact partitioning that the source tablets had #6628

Closed
bmatican opened this issue Dec 11, 2020 · 1 comment
Assignees
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature

Comments

@bmatican
Copy link
Contributor

Currently on backup, we preserve both the table metadata, as well as the per-tablet metadata, which would include the individual partition ranges for each tablet.

However, on restore, we recreate the table, with the same number of tablets, but we don't reuse the partition info from the backup! Instead, we just create a new table, with N tablets, evenly split in the partition space...

Historically, this has been fine because

  • the partition computation code has always been the same -- so the N partitions in the backup would have the same (start, end) as the new N created on restore
  • there was no mechanism to create skewed partitions -- the code to generate partitions would create even (at worse off by 1) splits

In the future though, there's at least two mechanisms that will change these assumptions

  • tablet splitting -- a tablet splitting into two will lead to two smaller partitions
  • explicit YSQL syntax to specify split points -- this could lead to adhoc distribution of split points

Implementation notes from @OlegLoginov

In a few words: currently CatalogManager::RecreateTable() fills CreateTableRequestPB to recreate the table. It fills partition schema too:
*req.mutable_partition_schema() = meta.partition_schema(); 
but in CtalogManager::CreateTable() it is overwritten by CreatePartitions():
  } else {
    s = PartitionSchema::FromPB(req.partition_schema(), schema, &partition_schema);
    RETURN_NOT_OK(partition_schema.CreatePartitions(num_tablets, &partitions));
  }
(it's even loaded from the Req, but the result status is ignored and "partition_schema" is overwritten- based on 'num_tablets'.)
These lines must be updated to cover cases when the partition schema can be taken from the Req.

The fix should not be complex. But we need carefully test all cases...
cql/sql, colocated, cql/sql backup restoring, hash tables, range tables, presplit tables... may be something else

Oleg, can you also share a snippet of how to run yb_backup locally against a yb-ctl cluster, so this can be tested easily manually?

cc @ttyusupov

@bmatican bmatican added the area/docdb YugabyteDB core features label Dec 11, 2020
@OlegLoginov
Copy link
Contributor

Command lines to start yb_backup manually (examples for an YCQL table):

python3 ./managed/devops/bin/yb_backup.py --keyspace a --table t --backup_location backup --masters 127.0.0.1 --no_ssh --remote_ysql_dump_binary=build/latest/postgres/bin/ysql_dump --remote_yb_admin_binary=build/latest/bin/yb-admin --storage_type nfs  --verbose create

python3 ./managed/devops/bin/yb_backup.py  --keyspace a2 --backup_location ./backup/table-a.t --masters 127.0.0.1 --no_ssh --remote_ysql_shell_binary=bin/ysqlsh --remote_ysql_dump_binary=build/latest/postgres/bin/ysql_dump --remote_yb_admin_binary=build/latest/bin/yb-admin --storage_type nfs --verbose restore

mikhpolitov added a commit that referenced this issue Feb 3, 2021
Summary: Read Table Partitions from Snapshot

Test Plan: ybd --gtest_filter CreateTableITest.TestCreateTableWithDefinedPartition -n 100

Reviewers: timur, bogdan, oleg

Reviewed By: oleg

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D10402
mikhpolitov added a commit that referenced this issue Feb 4, 2021
Summary:
Read Table Partitions from Snapshot

Original diff: D10402 / 3ce9d33

Test Plan:
Jenkins: rebase: 2.4, hot

ybd --gtest_filter CreateTableITest.TestCreateTableWithDefinedPartition -n 100

Reviewers: timur, bogdan, oleg

Reviewed By: oleg

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D10537
polarweasel pushed a commit to lizayugabyte/yugabyte-db that referenced this issue Mar 9, 2021
Summary: Read Table Partitions from Snapshot

Test Plan: ybd --gtest_filter CreateTableITest.TestCreateTableWithDefinedPartition -n 100

Reviewers: timur, bogdan, oleg

Reviewed By: oleg

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D10402
@ttyusupov ttyusupov added the kind/enhancement This is an enhancement of an existing feature label Jul 29, 2021
@mikhpolitov mikhpolitov changed the title [docdb] Restore should preserve the exact partitioning that the source tablets had [docdb] [YCQL] Restore should preserve the exact partitioning that the source tablets had Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature
Projects
None yet
Development

No branches or pull requests

4 participants