-
Notifications
You must be signed in to change notification settings - Fork 289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
puller(ticdc): always split update kv entries in sink safe mode #11224
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. @@ Coverage Diff @@
## master #11224 +/- ##
================================================
+ Coverage 55.2784% 55.6355% +0.3571%
================================================
Files 1001 1001
Lines 136158 137306 +1148
================================================
+ Hits 75266 76391 +1125
+ Misses 55407 55383 -24
- Partials 5485 5532 +47 |
/test dm-integration-test |
5 similar comments
/test dm-integration-test |
/test dm-integration-test |
/test dm-integration-test |
/test dm-integration-test |
/test dm-integration-test |
/retest |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: asddongmen, CharlesCheung96 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
Signed-off-by: ti-chi-bot <[email protected]>
In response to a cherrypick label: new pull request created to branch |
Signed-off-by: ti-chi-bot <[email protected]>
In response to a cherrypick label: new pull request created to branch |
Signed-off-by: ti-chi-bot <[email protected]>
In response to a cherrypick label: new pull request created to branch |
In response to a cherrypick label: new pull request created to branch |
…de (pingcap#11224) (pingcap#11656)" This reverts commit 1722e6f.
What problem does this PR solve?
Issue Number: close #11231
What is changed and how it works?
After introduce #10919, we choose to just split some update kv entries in puller module when changefeed starts and avoid split any update events in sink module.
This makes it possible to meet duplicate entry error during normal run and cause changefeed to restart.
Although after restart, puller can split the conflict update events and changefeed can continue to run normally. Some customer may be unhappy with this behavior if their workload has many conflict data which cause changefeed restart occasionally. So we need a workaround to avoid restart.
This pr introduce a type
PullerSplitUpdateMode
to describe how puller handle update kv entries. We keep all split logic unchanged as in #10919, and introduce a new behaviour:When mysql sink is in safe mode, we set
PullerSplitUpdateMode
toPullerSplitUpdateModeAlways
. This means to split all update kv entries in puller. So if the customer don't want changefeed to restart, they can configsafe-mode
to true to avoid the restart.Check List
Tests
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note