migration: Add information about Vitess to TiDB migration

[dveeden]

What is changed, added or deleted? (Required)

Related to:

• migration: Add information about non-MySQL to TiDB migration #5782
• VITESS_HASH function documentation #5718

Which TiDB version(s) do your changes apply to? (Required)

• master (the latest development version)
• v8.5 (TiDB 8.5 versions)
• v5.1 (TiDB 5.1 versions)
• v5.0 (TiDB 5.0 versions)
• v4.0 (TiDB 4.0 versions)
• v3.1 (TiDB 3.1 versions)
• v3.0 (TiDB 3.0 versions)
• v2.1 (TiDB 2.1 versions)

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[dveeden]

Comments from @morgo from when this was part of another PR: #5782 (comment)

[dveeden]

Things to note:

• The MySQL instances hosting the data for Vitess have a _vt schema. This is metadata and probably should be excluded.
• The schema in the backend database has a vt_ prefix that should probably be removed. Not sure where in the process this should be done.

If we try to run dumpling directly on vtgate this happens:

[dvaneeden@dve-carbon local]$ tiup dumpling -P 15306
Starting component `dumpling`: /home/dvaneeden/.tiup/components/dumpling/v5.0.2/dumpling -P 15306
Release version: v5.0.2
Git commit hash: 95099237b7beb80f66efaac9f3e02bd01e68bcc2
Git branch:      heads/refs/tags/v5.0.2
Build timestamp: 2021-06-07 08:39:29Z
Go version:      go version go1.13 linux/amd64

[2021/06/17 14:27:16.486 +02:00] [INFO] [versions.go:55] ["Welcome to dumpling"] ["Release Version"=v5.0.2] ["Git Commit Hash"=95099237b7beb80f66efaac9f3e02bd01e68bcc2] ["Git Branch"=heads/refs/tags/v5.0.2] ["Build timestamp"="2021-06-07 08:39:29"] ["Go Version"="go version go1.13 linux/amd64"]
[2021/06/17 14:27:16.488 +02:00] [INFO] [config.go:600] ["detect server type"] [type=MySQL]
[2021/06/17 14:27:16.488 +02:00] [INFO] [config.go:619] ["detect server version"] [version=8.0.25]
[2021/06/17 14:27:16.488 +02:00] [INFO] [dump.go:83] ["begin to run Dump"] [conf="{\"s3\":{\"endpoint\":\"\",\"region\":\"\",\"storage-class\":\"\",\"sse\":\"\",\"sse-kms-key-id\":\"\",\"acl\":\"\",\"access-key\":\"\",\"secret-access-key\":\"\",\"provider\":\"\",\"force-path-style\":true,\"use-accelerate-endpoint\":false},\"gcs\":{\"endpoint\":\"\",\"storage-class\":\"\",\"predefined-acl\":\"\",\"credentials-file\":\"\"},\"AllowCleartextPasswords\":false,\"SortByPk\":true,\"NoViews\":true,\"NoHeader\":false,\"NoSchemas\":false,\"NoData\":false,\"CompleteInsert\":false,\"TransactionalConsistency\":true,\"EscapeBackslash\":true,\"DumpEmptyDatabase\":true,\"PosAfterConnect\":false,\"CompressType\":0,\"Host\":\"127.0.0.1\",\"Port\":15306,\"Threads\":4,\"User\":\"root\",\"Security\":{\"CAPath\":\"\",\"CertPath\":\"\",\"KeyPath\":\"\"},\"LogLevel\":\"info\",\"LogFile\":\"\",\"LogFormat\":\"text\",\"OutputDirPath\":\"./export-2021-06-17T14:27:16+02:00\",\"StatusAddr\":\":8281\",\"Snapshot\":\"\",\"Consistency\":\"flush\",\"CsvNullValue\":\"\\\\N\",\"SQL\":\"\",\"CsvSeparator\":\",\",\"CsvDelimiter\":\"\\\"\",\"Databases\":[],\"Where\":\"\",\"FileType\":\"sql\",\"ServerInfo\":{\"HasTiKV\":false,\"ServerType\":1,\"ServerVersion\":\"8.0.25\"},\"Rows\":0,\"ReadTimeout\":900000000000,\"TiDBMemQuotaQuery\":0,\"FileSize\":0,\"StatementSize\":1000000,\"SessionParams\":{},\"Tables\":null}"]
[2021/06/17 14:27:16.490 +02:00] [ERROR] [main.go:77] ["dump failed error stack info"] [error="sql: START TRANSACTION /*!40108 WITH CONSISTENT SNAPSHOT */: Error 1105: syntax error at position 55 near 'WITH'"] [errorVerbose="Error 1105: syntax error at position 55 near 'WITH'\nsql: START TRANSACTION /*!40108 WITH CONSISTENT SNAPSHOT */\ngithub.com/pingcap/dumpling/v4/export.createConnWithConsistency\n\tgithub.com/pingcap/dumpling@/v4/export/sql.go:587\ngithub.com/pingcap/dumpling/v4/export.(*Dumper).Dump\n\tgithub.com/pingcap/dumpling@/v4/export/dump.go:119\nmain.main\n\tgithub.com/pingcap/dumpling@/cmd/dumpling/main.go:74\nruntime.main\n\truntime/proc.go:203\nruntime.goexit\n\truntime/asm_amd64.s:1357"]

dump failed: sql: START TRANSACTION /*!40108 WITH CONSISTENT SNAPSHOT */: Error 1105: syntax error at position 55 near 'WITH'
Error: run `/home/dvaneeden/.tiup/components/dumpling/v5.0.2/dumpling` (wd:/home/dvaneeden/.tiup/data/SaaugB3) failed: exit status 1

Even with --consistency none this happens

[qiancai]

@Frank945946 would you please take a look at this PR? Thanks.

[ti-chi-bot]

@Frank945946: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mjonss]

LGTM, with some suggestions

[ti-chi-bot]

@mjonss: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

LGTM, with some suggestions

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dveeden]

@mjonss @qiancai This PR has been sitting here for years now. What things are required to get this merged?

[qiancai]

@mjonss @qiancai This PR has been sitting here for years now. What things are required to get this merged?

It looks good to me. Let me start working on the corresponding Chinese PR and merge both PRs afterwards.

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-11-17 09:28:35.491006828 +0000 UTC m=+4414113.078116958: ☑️ agreed by lance6716.
• 2024-11-15 01:56:56.753867322 +0000 UTC m=+580578.944736320: ☑️ agreed by hfxsd.

[qiancai]

/approve

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: qiancai

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [qiancai]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment