Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CDC Synchronize big transaction is too slow #1683

Closed
Tracked by #5231
Tammyxia opened this issue Apr 21, 2021 · 2 comments · Fixed by #5712
Closed
Tracked by #5231

CDC Synchronize big transaction is too slow #1683

Tammyxia opened this issue Apr 21, 2021 · 2 comments · Fixed by #5712
Labels
area/ticdc Issues or PRs related to TiCDC. bug-from-internal-test Bugs found by internal testing. subject/new-feature Denotes an issue or pull request adding a new feature. type/enhancement The issue or PR belongs to an enhancement.

Comments

@Tammyxia
Copy link

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error.
  • Pause changfeed.
  • Start transaction operation which has 5GB data in TiDB, this process duration time is about half an hour.
  • Resume changfeed, observe the big transaction sync status.
  1. What did you expect to see?

  2. What did you see instead?
    sync duration time is more than 5 hours,
    image

  3. Versions of the cluster

    • Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

      Release Version: v5.0.1
      
      
      

Edition: Community
Git Commit Hash: 4141f837ad4655a567119db743b7b752a95f5aa2
Git Branch: heads/refs/tags/v5.0.1
UTC Build Time: 2021-04-17 04:28:49

    ```

- TiCDC version (execute `cdc version`):

    ```

Release Version: v5.0.0-master
Git Commit Hash: 6363c48
Git Branch: heads/refs/tags/v5.0.1
UTC Build Time: 2021-04-17 04:40:20

    ```
@Tammyxia Tammyxia added the type/bug The issue is confirmed as a bug. label Apr 21, 2021
@Tammyxia Tammyxia changed the title Syncing big transaction is too slow CDC Synchronize big transaction is too slow Apr 21, 2021
@amyangfei amyangfei added the bug-from-internal-test Bugs found by internal testing. label Apr 27, 2021
@Tammyxia
Copy link
Author

Tammyxia commented May 7, 2021

The workload in upstream is "start transaction; update more then 10 million rows; commit".

@AkiraXie AkiraXie added the area/ticdc Issues or PRs related to TiCDC. label Dec 6, 2021
@overvenus
Copy link
Member

It's more like a feature request, we will add it to TiCDC roadmap, thanks!

@overvenus overvenus added subject/new-feature Denotes an issue or pull request adding a new feature. and removed type/bug The issue is confirmed as a bug. severity/major labels Dec 8, 2021
@CharlesCheung96 CharlesCheung96 added the type/enhancement The issue or PR belongs to an enhancement. label May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ticdc Issues or PRs related to TiCDC. bug-from-internal-test Bugs found by internal testing. subject/new-feature Denotes an issue or pull request adding a new feature. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
6 participants