mounter(ticdc): fix mounter add default value type unsupported #3846

maxshuang · 2021-12-13T07:13:15Z

What problem does this PR solve?

Fix default value unsupported bug cause by amend feature of tidb, related to issue #3793

Amend feature of TiDB:
As for add column DDL, if a DDL occur between a trx, the trx will not redo but commit the old schema data directly, which will cause TiCDC use new schema to parse old schema data.

What is changed and how it works?

After call "getDefaultOrZeroValue", we need to call "formatColVal" to format data from the tidb type to go supported sql type

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
Start 2 transactions:

Transaction 1:

Create table test_column(id int primary key);
Insert Into test_column values(1);
begin;
Insert Into test_column values(2);
// hold the trx untill Transaction 2 commit DDL
commit;

Transaction 2:

alter table test_column add column c0 date not null;

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

Need to cherry-pick to the release branch
Need to update the documentation
Need to update key monitor metrics in both TiCDC document and official document

Release note

Fix mounter default date value not support

close #3793

ti-chi-bot · 2021-12-13T07:13:16Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

amyangfei
overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

overvenus

Rest LGTM

cdc/entry/mounter.go

cdc/entry/mounter_test.go

amyangfei · 2021-12-13T07:33:08Z

cdc/entry/mounter_test.go

+
+func testGetDefaultZeroValue(t *testing.T, colAndRess []columnInfoAndResult) {
+	for _, colAndRes := range colAndRess {
+		val, _, _, _ := getDefaultOrZeroValue(&colAndRes.ColInfo)


Better to check the other return values

May need more unit tests also. I will add more in other pr.

cdc/entry/mounter_test.go

overvenus

LGTM

cdc/entry/mounter.go

amyangfei · 2021-12-13T08:07:49Z

/merge

ti-chi-bot · 2021-12-13T08:07:52Z

This pull request has been accepted and is ready to merge.

Commit hash: 80f59cb

ti-chi-bot · 2021-12-13T08:19:26Z

Merge canceled because a new commit is pushed.

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2021-12-13T12:17:12Z

In response to a cherrypick label: new pull request created: #3855.

overvenus · 2021-12-13T12:17:15Z

/run-cherry-picker

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2021-12-13T12:17:49Z

In response to a cherrypick label: new pull request created: #3856.

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2021-12-13T12:18:25Z

In response to a cherrypick label: new pull request created: #3857.

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2021-12-13T12:19:04Z

In response to a cherrypick label: new pull request created: #3858.

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot · 2021-12-13T12:19:41Z

In response to a cherrypick label: new pull request created: #3859.

#3855)

#3856)

#3846) (#3855)" This reverts commit 238d38d.

#3846) (#3856)" This reverts commit 1cc8776.

#3846) (#3855)" (#3921)

#3846) (#3856)" (#3922) This reverts commit 1cc8776.

…ap#3846) * mounter(ticdc): fix mounter add default value type unsupported close pingcap#3793 * test(mounter(ticdc)): fix unit test panic for enum * comment(mounter(ticdc)): add some return value comment for mounter * test(mounter(ticdc)): fix default value size unmatch error in uts * mounter(ticdc): change comment Co-authored-by: Neil Shen <[email protected]> * test(mounter(ticdc)): close size check temporary * chore(mounter(ticdc)): fix unused lint Co-authored-by: Neil Shen <[email protected]>

close #3793

mounter(ticdc): fix mounter add default value type unsupported

b55bb25

close #3793

ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 13, 2021

maxshuang requested review from liuzix, amyangfei, Rustin170506 and overvenus December 13, 2021 07:13

overvenus reviewed Dec 13, 2021

View reviewed changes

cdc/entry/mounter.go Show resolved Hide resolved

cdc/entry/mounter_test.go Outdated Show resolved Hide resolved

test(mounter(ticdc)): fix unit test panic for enum

aa1e847

amyangfei reviewed Dec 13, 2021

View reviewed changes

comment(mounter(ticdc)): add some return value comment for mounter

80f59cb

maxshuang requested a review from 3AceShowHand December 13, 2021 07:47

overvenus approved these changes Dec 13, 2021

View reviewed changes

cdc/entry/mounter.go Outdated Show resolved Hide resolved

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 13, 2021

amyangfei approved these changes Dec 13, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Dec 13, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 13, 2021

test(mounter(ticdc)): fix default value size unmatch error in uts

ae6e73d

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Dec 13, 2021

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Dec 13, 2021

This is an automated cherry-pick of pingcap#3846

77c78d3

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) #3855

Merged

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Dec 13, 2021

This is an automated cherry-pick of pingcap#3846

67b3b35

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) #3856

Merged

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Dec 13, 2021

This is an automated cherry-pick of pingcap#3846

c7a6785

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) #3857

Closed

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Dec 13, 2021

This is an automated cherry-pick of pingcap#3846

105437c

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) #3858

Closed

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Dec 13, 2021

This is an automated cherry-pick of pingcap#3846

c8a38c2

Signed-off-by: ti-chi-bot <[email protected]>

ti-chi-bot mentioned this pull request Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) #3859

Closed

ti-chi-bot added a commit that referenced this pull request Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) (

238d38d

#3855)

ti-chi-bot added a commit that referenced this pull request Dec 16, 2021

mounter(ticdc): fix mounter add default value type unsupported (#3846) (

1cc8776

#3856)

maxshuang pushed a commit that referenced this pull request Dec 16, 2021

Revert "mounter(ticdc): fix mounter add default value type unsupported (

972b299

#3846) (#3855)" This reverts commit 238d38d.

maxshuang pushed a commit that referenced this pull request Dec 16, 2021

Revert "mounter(ticdc): fix mounter add default value type unsupported (

f143b05

#3846) (#3856)" This reverts commit 1cc8776.

Rustin170506 pushed a commit that referenced this pull request Dec 16, 2021

Revert "mounter(ticdc): fix mounter add default value type unsupported (

af41784

#3846) (#3855)" (#3921)

maxshuang mentioned this pull request Dec 16, 2021

Data Inconsistency when meet TiDB "amend + add column default 'xxx'" #3929

Closed

overvenus pushed a commit that referenced this pull request Dec 21, 2021

Revert "mounter(ticdc): fix mounter add default value type unsupported (

809a330

#3846) (#3856)" (#3922) This reverts commit 1cc8776.

maxshuang mentioned this pull request Jan 24, 2022

Fix data inconsistency caused by default value #4446

Closed

maxshuang mentioned this pull request Feb 8, 2022

Fix unsupported data type issue caused by default value filling logic #4518

Merged

overvenus pushed a commit that referenced this pull request Feb 8, 2022

mounter(ticdc): fix mounter add default value type unsupported (#3846)

0828409

close #3793

maxshuang mentioned this pull request Feb 22, 2022

Fix data inconsistency caused by default value #4652

Merged

overvenus pushed a commit that referenced this pull request Feb 22, 2022

mounter(ticdc): fix mounter add default value type unsupported (#3846)

7519d6c

close #3793

maxshuang mentioned this pull request Apr 12, 2022

Batch cherrypick default-value prs to release-5.2 #5156

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mounter(ticdc): fix mounter add default value type unsupported #3846

mounter(ticdc): fix mounter add default value type unsupported #3846

maxshuang commented Dec 13, 2021 •

edited

Loading

ti-chi-bot commented Dec 13, 2021 •

edited

Loading

overvenus left a comment

amyangfei Dec 13, 2021

maxshuang Dec 13, 2021

overvenus left a comment

amyangfei commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

overvenus commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

mounter(ticdc): fix mounter add default value type unsupported #3846

mounter(ticdc): fix mounter add default value type unsupported #3846

Conversation

maxshuang commented Dec 13, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Dec 13, 2021 • edited Loading

overvenus left a comment

Choose a reason for hiding this comment

amyangfei Dec 13, 2021

Choose a reason for hiding this comment

maxshuang Dec 13, 2021

Choose a reason for hiding this comment

overvenus left a comment

Choose a reason for hiding this comment

amyangfei commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

overvenus commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

ti-chi-bot commented Dec 13, 2021

maxshuang commented Dec 13, 2021 •

edited

Loading

ti-chi-bot commented Dec 13, 2021 •

edited

Loading