Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

simple (ticdc): support send all tables bootstrap message at changefeed start #11239

Merged
merged 10 commits into from
Jun 20, 2024
Merged

simple (ticdc): support send all tables bootstrap message at changefeed start #11239

merged 10 commits into from
Jun 20, 2024

Conversation

asddongmen
Copy link
Contributor

@asddongmen asddongmen commented Jun 4, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #11315

What is changed and how it works?

A boolean parameter send-all-bootstrap-at-start is added, which is set to false by default.
If this parameter is set to true, the changefeed, upon starting, will send the schema of all synchronized tables downstream as Bootstrap events. This will be done synchronously in the ddlSink. After the Bootstrap events are synchronized, row changes will begin to be synchronized.

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

None

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 4, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. affect-ticdc-config-docs Pull requests that affect TiCDC configuration docs. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 4, 2024
@asddongmen
Copy link
Contributor Author

/test all

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jun 14, 2024
@asddongmen asddongmen requested a review from 3AceShowHand June 14, 2024 09:31
@asddongmen asddongmen marked this pull request as ready for review June 14, 2024 10:04
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 14, 2024
@3AceShowHand
Copy link
Contributor

/retest

@asddongmen
fix make check
43c7a55 
Signed-off-by: dongmen <[email protected]>
@asddongmen
fix ut
732a5fa 
Signed-off-by: dongmen <[email protected]>
@asddongmen asddongmen requested a review from sdojjy June 17, 2024 02:15
@asddongmen asddongmen self-assigned this Jun 17, 2024
@codecov Codecov
Copy link

codecov bot commented Jun 17, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 68.00000% with 24 lines in your changes missing coverage. Please review.

Project coverage is 57.7412%. Comparing base (999d393) to head (8ea0f6d).
Report is 6 commits behind head on master.

Additional details and impacted files
Components Coverage Δ
cdc 61.4548% <68.0000%> (+0.0225%) ⬆️
dm 51.4356% <ø> (+0.2325%) ⬆️
engine 63.5518% <ø> (+0.1638%) ⬆️
Flag Coverage Δ
unit 57.7412% <68.0000%> (+0.1098%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #11239        +/-   ##
================================================
+ Coverage   57.6313%   57.7412%   +0.1098%     
================================================
  Files           850        850                
  Lines        126256     126634       +378     
================================================
+ Hits          72763      73120       +357     
- Misses        48088      48092         +4     
- Partials       5405       5422        +17

@asddongmen
Copy link
Contributor Author

/retest

@asddongmen asddongmen added the require-LGT2 This PR needs to get two LGTMs before merging. label Jun 17, 2024
hongyunyan
hongyunyan reviewed Jun 17, 2024
View reviewed changes
cdc/owner/ddl_sink.go Outdated Show resolved Hide resolved
cdc/owner/ddl_manager.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 17, 2024
@asddongmen
resolve comments
4fb9bcd 
Signed-off-by: dongmen <[email protected]>
hongyunyan
hongyunyan approved these changes Jun 18, 2024
View reviewed changes
Copy link
Collaborator

@hongyunyan hongyunyan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest is LGTM

cdc/owner/ddl_manager.go Outdated Show resolved Hide resolved
cdc/owner/ddl_manager.go Outdated Show resolved Hide resolved
cdc/owner/ddl_manager_test.go Outdated Show resolved Hide resolved
cdc/owner/ddl_manager_test.go Outdated Show resolved Hide resolved
cdc/owner/ddl_manager_test.go Outdated Show resolved Hide resolved
cdc/owner/ddl_sink.go Show resolved Hide resolved
@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 18, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 18, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-06-17 08:05:47.002720788 +0000 UTC m=+15673.488209632: ☑️ agreed by 3AceShowHand.
  • 2024-06-18 02:01:41.785244765 +0000 UTC m=+80228.270733597: ☑️ agreed by hongyunyan.

@asddongmen
resolve comments 2
8ea0f6d 
Signed-off-by: dongmen <[email protected]>
@asddongmen
Copy link
Contributor Author

/retest

1 similar comment
@asddongmen
Copy link
Contributor Author

/retest

@asddongmen asddongmen removed the require-LGT2 This PR needs to get two LGTMs before merging. label Jun 18, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 20, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3AceShowHand, benmeadowcroft, hongyunyan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Jun 20, 2024
@ti-chi-bot ti-chi-bot bot merged commit 58636ed into pingcap:master Jun 20, 2024
27 of 28 checks passed
@ti-chi-bot ti-chi-bot added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Jul 10, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #11400.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jul 10, 2024
@asddongmen @ti-chi-bot
This is an automated cherry-pick of pingcap#11239
a9d0544 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Jul 10, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this pull request Jul 24, 2024
@ti-chi-bot
simple (ticdc): support send all tables bootstrap message at changefe…
b00c3ee 
…ed start (#11239) (#11400)

close #11315
@ti-chi-bot ti-chi-bot added the needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. label Jul 30, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #11439.

@ti-chi-bot ti-chi-bot mentioned this pull request Jul 30, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this pull request Aug 2, 2024
@ti-chi-bot
simple (ticdc): support send all tables bootstrap message at changefe…
8a8bed9 
…ed start (#11239) (#11439)

close #11315
@3AceShowHand 3AceShowHand mentioned this pull request Sep 11, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hongyunyan hongyunyan hongyunyan approved these changes

@3AceShowHand 3AceShowHand 3AceShowHand approved these changes

@benmeadowcroft benmeadowcroft benmeadowcroft approved these changes

@sdojjy sdojjy Awaiting requested review from sdojjy

Assignees

@asddongmen asddongmen

Labels
affect-ticdc-config-docs Pull requests that affect TiCDC configuration docs. approved lgtm needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Simple protocol: Send all Bootstrap when changefeed starts
5 participants
@asddongmen @3AceShowHand @ti-chi-bot @benmeadowcroft @hongyunyan