Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tidb-lightning-faq.md from OnCALL (#6840) #6883

Merged
merged 5 commits into from
Nov 22, 2021
Merged

Update tidb-lightning-faq.md from OnCALL (#6840) #6883

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c9a279c
Update tidb-lightning-faq.md
hfxsd Nov 18, 2021
82ae484
Apply suggestions from code review
hfxsd Nov 18, 2021
c9dbcb3
Update tidb-lightning/tidb-lightning-faq.md
hfxsd Nov 19, 2021
f808329
Update tidb-lightning/tidb-lightning-faq.md
hfxsd Nov 19, 2021
c6db9b4
Apply suggestions from code review
hfxsd Nov 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions tidb-lightning/tidb-lightning-faq.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,22 +222,21 @@ Try the latest version! Maybe there is new speed improvement.

## `checksum failed: checksum mismatched remote vs local`

**Cause**: The checksum of a table in the local data source and the remote imported database differ. This error has several deeper reasons:
**Cause**: The checksum of a table in the local data source and the remote imported database differ. This error has several deeper reasons. You can further locate the reason by checking the log that contains `checksum mismatched`.

1. The table might already have data before. These old data can affect the final checksum.
The lines that contain `checksum mismatched` provide the information `total_kvs: x vs y`, where `x` indicates the number of key-value pairs (KV pairs) calculated by the target cluster after the import is completed, and `y` indicates the number of key-value pairs generated by the local data source.

2. If the remote checksum is 0, which means nothing is imported, it is possible that the cluster is too hot and fails to take in any data.

3. If the data is mechanically generated, ensure it respects the constrains of the table:

* `AUTO_INCREMENT` columns need to be positive, and do not contain the value "0".
* The UNIQUE and PRIMARY KEYs must have no duplicated entries.

4. If TiDB Lightning has failed before and was not properly restarted, a checksum mismatch may happen due to data being out-of-sync.
- If `x` is greater, it means that there are more KV pairs in the target cluster.
- It is possible that this table is not empty before the import and therefore affects the data checksum. It is also possible that TiDB Lightning has previously failed and shut down, but did not restart correctly.
- If `y` is greater, it means that there are more KV pairs in the local data source.
- If the checksum of the target database is all 0, it means that no import has occurred. It is possible that the cluster is too busy to receive any data.
- It is possible that the exported data contains duplicate data, such as the UNIQUE and PRIMARY KEYs with duplicate values, or that the downstream table structure is case-insensitive while the data is case-sensitive.
- Other possible reasons
- If the data source is machine-generated and not backed up by Dumpling, make sure the data conforms to the table limits. For example, the AUTO_INCREMENT column needs to be positive and not 0.

**Solutions**:

1. Delete the corrupted data using `tidb-lightning-ctl`, and restart TiDB Lightning to import the affected tables again.
1. Delete the corrupted data using `tidb-lightning-ctl`, check the table structure and the data, and restart TiDB Lightning to import the affected tables again.

{{< copyable "shell-regular" >}}

Expand Down