Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

streamer: fix duplicate event when reparse relay using GTID #1525

Merged
merged 6 commits into from
Mar 24, 2021

Conversation

lance6716
Copy link
Collaborator

@lance6716 lance6716 commented Mar 23, 2021

What problem does this PR solve?

part of #1524

What is changed and how it works?

when DM uses GTID and needs to return a reparse location, return last XID location so that replaceWithHeartbeat will be correctly calculated

Check List

Tests

  • Unit test

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

@lance6716
Copy link
Collaborator Author

/cc @GMHDBJD @lichunzhu

@lance6716
Copy link
Collaborator Author

/run-all-tests

@lance6716
Copy link
Collaborator Author

/run-compatibility-test

@lance6716
Copy link
Collaborator Author

/auto-cc

@ti-chi-bot ti-chi-bot requested review from 5kbpers and GMHDBJD March 23, 2021 09:11
@lance6716 lance6716 added needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 status/PTAL This PR is ready for review. Add this label back after committing new changes labels Mar 23, 2021
@ti-chi-bot ti-chi-bot added the status/LGT1 One reviewer already commented LGTM label Mar 23, 2021
@GMHDBJD
Copy link
Collaborator

GMHDBJD commented Mar 24, 2021

Will sync unit get duplicate events? (like gtid_event, table map event, rows event)

@lance6716
Copy link
Collaborator Author

lance6716 commented Mar 24, 2021

Will sync unit get duplicate events? (like gtid_event, table map event, rows event)

Yes, currently we rely on safe mode. I added a todo of changing the behaviour of sending binlog to "sending DML after see XID". Do you think we should do it now?

@GMHDBJD
Copy link
Collaborator

GMHDBJD commented Mar 24, 2021

Yes, currently we rely on safe mode. I added a todo of changing the behaviour of sending binlog to "sending DML after see XID". Do you think we should do it now?

I think we may simply return both currentGTID and latestPos for parseFile or save latest gtid now. Otherwise, if safemode has been closed, we may have duplicate data. Or if sending DML after see XID is easy, that's ok.

@lance6716 lance6716 added this to the v2.0.2 milestone Mar 24, 2021
Copy link
Collaborator

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • GMHDBJD
  • lichunzhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added status/LGT2 Two reviewers already commented LGTM, ready for merge and removed status/LGT1 One reviewer already commented LGTM labels Mar 24, 2021
@lance6716
Copy link
Collaborator Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 3ed9b0a

@ti-chi-bot ti-chi-bot merged commit 47b68f9 into pingcap:master Mar 24, 2021
ti-srebot pushed a commit to ti-srebot/dm that referenced this pull request Mar 24, 2021
@ti-srebot
Copy link

cherry pick to release-2.0 in PR #1532

@ti-srebot ti-srebot added already-cherry-pick-2.0 The related PR is already cherry-picked to release-2.0. Add this label once the PR is cherry-picked and removed needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 labels Mar 24, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
already-cherry-pick-2.0 The related PR is already cherry-picked to release-2.0. Add this label once the PR is cherry-picked size/L status/can-merge status/LGT2 Two reviewers already commented LGTM, ready for merge status/PTAL This PR is ready for review. Add this label back after committing new changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants