diff --git a/TOC.md b/TOC.md index 69486b98c1dd1..4e68f31bd986a 100644 --- a/TOC.md +++ b/TOC.md @@ -158,6 +158,7 @@ - [Migrate Large Datasets from MySQL](/migrate-large-mysql-to-tidb.md) - [Migrate and Merge MySQL Shards of Small Datasets](/migrate-small-mysql-shards-to-tidb.md) - [Migrate and Merge MySQL Shards of Large Datasets](/migrate-large-mysql-shards-to-tidb.md) + - [Migrate from Vitess](/migrate-from-vitess.md) - [Migrate from MariaDB](/migrate-from-mariadb.md) - [Migrate from CSV Files](/migrate-from-csv-files-to-tidb.md) - [Migrate from SQL Files](/migrate-from-sql-files-to-tidb.md) diff --git a/media/vitess_to_tidb.png b/media/vitess_to_tidb.png new file mode 100644 index 0000000000000..ab1348c1918fb Binary files /dev/null and b/media/vitess_to_tidb.png differ diff --git a/media/vitess_to_tidb_dm.png b/media/vitess_to_tidb_dm.png new file mode 100644 index 0000000000000..3fea6f7679c6c Binary files /dev/null and b/media/vitess_to_tidb_dm.png differ diff --git a/media/vitess_to_tidb_dumpling_local.png b/media/vitess_to_tidb_dumpling_local.png new file mode 100644 index 0000000000000..5542df73f2241 Binary files /dev/null and b/media/vitess_to_tidb_dumpling_local.png differ diff --git a/migrate-from-vitess.md b/migrate-from-vitess.md new file mode 100644 index 0000000000000..74f4bfd190a7b --- /dev/null +++ b/migrate-from-vitess.md @@ -0,0 +1,41 @@ +--- +title: Migrate Data from Vitess to TiDB +summary: Learn about the tools to migrate data from Vitess to TiDB. +--- + +# Migrate Data from Vitess to TiDB + +This document describes the tools that you can use to migrate data from [Vitess](https://vitess.io/) to TiDB. + +Because the backend of Vitess is based on MySQL, when migrating data from Vitess to TiDB, you can use the same migration tools that apply to MySQL, such as [Dumpling](/dumpling-overview.md), [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md), and [TiDB Data Migration (DM)](/dm/dm-overview.md). Note that these tools should be set up for each shard in Vitess for data migration. + +Generally, before data migration, it is recommended to configure the DM task to set `task-mode` to `all` and `import-mode` to `physical`. For more information, see [Task configuration file template (advanced)](/dm/task-configuration-file-full.md#task-configuration-file-template-advanced). + +If your data size exceeds 10 TiB, it is recommended to do the import in two steps: + +1. Use Dumpling and TiDB Lightning to import existing data. +2. Use DM to import incremental data. + +In addition to these tools, you can also use [Debezium connector for Vitess](https://debezium.io/documentation/reference/connectors/vitess.html). This connector enables you to use [Kafka Connect](https://kafka.apache.org/documentation/#connect) or [Apache Flink](https://nightlies.apache.org/flink/flink-docs-stable/) to stream changes from Vitess to TiDB. + +Because both Vitess and TiDB support the MySQL protocol and SQL dialect, changes at the application level are expected to be small. For tasks directly managing sharding or other implementation-specific aspects, however, the changes might be larger. To facilitate the data migration from Vitess to TiDB, TiDB introduces the [`VITESS_HASH()`](/functions-and-operators/tidb-functions.md) function, which returns the hash of a string that is compatible with Vitess' HASH function. + +## Examples + +### Dumpling and TiDB Lightning + +The following two examples show how Dumpling and TiDB Lightning work together to migrate data from Vitess to TiDB. + +- In this example, TiDB Lightning uses the [logical import mode](/tidb-lightning/tidb-lightning-logical-import-mode.md), which first encodes data into SQL statements and then runs the SQL statements to import data. + + ![Vitess to TiDB Migration with TiDB backend](/media/vitess_to_tidb.png) + +- In this example, TiDB Lightning uses the [physical import mode](/tidb-lightning/tidb-lightning-physical-import-mode.md) to directly ingest data into TiKV. + + ![Vitess to TiDB Migration with local backend](/media/vitess_to_tidb_dumpling_local.png) + +### DM + +The following example shows how [DM](/dm/dm-overview.md) migrates data from Vitess to TiDB. + +![Vitess to TiDB with DM](/media/vitess_to_tidb_dm.png) diff --git a/migration-overview.md b/migration-overview.md index 326eced1fe5c1..47f2ab15736fe 100644 --- a/migration-overview.md +++ b/migration-overview.md @@ -44,6 +44,12 @@ If the data size of the sharded tables is large (for example, larger than 1 TiB) - [Migrate and Merge MySQL Shards of Large Datasets to TiDB](/migrate-large-mysql-shards-to-tidb.md) +## Migrate data from Vitess to TiDB + +To migrate data from Vitess to TiDB, the following guide is available: + +- [Migrate Data from Vitess to TiDB](/migrate-from-vitess.md) + ## Migrate data from files to TiDB - [Migrate data from CSV files to TiDB](/migrate-from-csv-files-to-tidb.md) @@ -61,4 +67,4 @@ The following features can improve the migration process and might meet more nee - [Continuous Replication from Databases that Use gh-ost or pt-osc](/migrate-with-pt-ghost.md) - [Migrate Data to a Downstream TiDB Table with More Columns](/migrate-with-more-columns-downstream.md) - [Filter Binlog Events](/filter-binlog-event.md) -- [Filter DML Events Using SQL Expressions](/filter-dml-event.md) \ No newline at end of file +- [Filter DML Events Using SQL Expressions](/filter-dml-event.md)