Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

split the update event when using avro or csv protocol #9086

Closed
3AceShowHand opened this issue May 29, 2023 · 2 comments · Fixed by #9079
Closed

split the update event when using avro or csv protocol #9086

3AceShowHand opened this issue May 29, 2023 · 2 comments · Fixed by #9079
Assignees
@3AceShowHand
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. report/community The community has encountered this bug. type/enhancement The issue or PR belongs to an enhancement.

Comments

@3AceShowHand
Copy link
Contributor

Avro and CSV Protocol, does not emit the old value for the update event to downstream, so if the primary key is updated, the old data cannot be deleted from the downstream data system.

By split the update event into delete event and insert event, the old data can be delete first, and then insert the new data.
@3AceShowHand 3AceShowHand self-assigned this May 29, 2023
@3AceShowHand 3AceShowHand added component/sink Sink component. type/enhancement The issue or PR belongs to an enhancement. area/ticdc Issues or PRs related to TiCDC. labels May 29, 2023
@3AceShowHand 3AceShowHand mentioned this issue May 29, 2023
Merged
@ti-chi-bot ti-chi-bot bot closed this as completed in #9079 Jun 3, 2023
ti-chi-bot bot pushed a commit that referenced this issue Jun 3, 2023
@3AceShowHand
config(ticdc): enable-old-value always false if using avro or csv as …
6537ab8 
…the encoding protocol (#9079)

close #9086
@ti-chi-bot ti-chi-bot mentioned this issue Jun 3, 2023
Merged
@nongfushanquan
Copy link
Contributor

/label affects-7.1
/label affects-6.5

@ti-chi-bot ti-chi-bot bot added affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. labels Jun 4, 2023
ti-chi-bot bot pushed a commit that referenced this issue Jun 5, 2023
@ti-chi-bot
config(ticdc): enable-old-value always false if using avro or csv as …
1e2f277 
…the encoding protocol (#9079) (#9127)

close #9086
@nongfushanquan nongfushanquan mentioned this issue Jun 9, 2023
Merged
17 tasks
@nongfushanquan nongfushanquan mentioned this issue Jun 21, 2023
Merged
16 tasks
@ti-chi-bot ti-chi-bot mentioned this issue Jun 30, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jul 3, 2023
@ti-chi-bot
config(ticdc): enable-old-value always false if using avro or csv as …
d8374fb 
…the encoding protocol (#9079) (#9318)

close #9086
@nongfushanquan nongfushanquan mentioned this issue Jul 14, 2023
Merged
17 tasks
@seiya-annie
Copy link

/found community

@ti-chi-bot ti-chi-bot bot added the report/community The community has encountered this bug. label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@3AceShowHand 3AceShowHand

Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. report/community The community has encountered this bug. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
3 participants
@3AceShowHand @seiya-annie @nongfushanquan