Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

syncer: fix generated column in where condition #60

Merged
merged 9 commits into from
Mar 2, 2019

Conversation

amyangfei
Copy link
Contributor

What problem does this PR solve?

Currently we remove all generated column for DML, but generated column in where condition is valid.

What is changed and how it works?

  • keep generated column in index searching
  • keep generated column in where condition when generating update/delete DML

Check List

Tests

  • Integration test

Side effects

  • Increased code complexity

@amyangfei amyangfei added priority/normal Minor change, requires approval from ≥1 primary reviewer status/WIP This PR is still work in progress type/bug-fix Bug fix labels Feb 26, 2019
@amyangfei amyangfei added status/PTAL This PR is ready for review. Add this label back after committing new changes and removed status/WIP This PR is still work in progress labels Feb 27, 2019
@amyangfei
Copy link
Contributor Author

PTAL @GregoryIan @csuzhangxc

@amyangfei
Copy link
Contributor Author

/run-all-tests

3 similar comments
@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-all-tests

@amyangfei
Copy link
Contributor Author

/run-all-tests

syncer/dml.go Outdated

if len(defaultIndexColumns) == 0 {
defaultIndexColumns = getAvailableIndexColumn(indexColumns, oldValues)
defaultIndexColumns = getAvailableIndexColumn(originalIndexColumns, oriOldValues)
}

ks := genMultipleKeys(columns, oldValues, indexColumns)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use oriOldValues?

@IANTHEREAL
Copy link
Collaborator

Rest LGTM

@amyangfei
Copy link
Contributor Author

PTAL @GregoryIan

sqls := make([]string, 0, len(dataSeq))
keys := make([][]string, 0, len(dataSeq))
values := make([][]interface{}, 0, len(dataSeq))
columnList := genColumnList(columns)
columnPlaceholders := genColumnPlaceholders(len(columns))
for _, data := range dataSeq {
for dataIdx, data := range dataSeq {
if len(data) != len(columns) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also check originalColumns and originalDataSeq ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed, data/columns and origianlData/originalColumns have the same length distance ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add column and data length check in generated column prune

syncer/dml.go Outdated
@@ -140,16 +177,30 @@ func genUpdateSQLs(schema string, table string, data [][]interface{}, columns []
changedValues = append(changedValues, castUnsigned(changedData[i], columns[i].unsigned, columns[i].tp))
}

oriOldValues := make([]interface{}, 0, len(oriOldData))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

genUpdateSQLs, `genInsertSQLs`` has these codes in same pattern:

...values
for i := range data {
    values = append(values,....)
}

if len(columns) == len(originalColumns) {
  originalVals = values
} else {
    for {....}
}

can we extract it a function?

@IANTHEREAL
Copy link
Collaborator

LGTM

@IANTHEREAL IANTHEREAL added status/LGT1 One reviewer already commented LGTM and removed status/PTAL This PR is ready for review. Add this label back after committing new changes labels Mar 1, 2019
Copy link
Member

@csuzhangxc csuzhangxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@csuzhangxc csuzhangxc added status/LGT2 Two reviewers already commented LGTM, ready for merge and removed status/LGT1 One reviewer already commented LGTM labels Mar 2, 2019
@amyangfei amyangfei merged commit e0caf42 into pingcap:master Mar 2, 2019
@amyangfei amyangfei deleted the fix-gen-col-in-where branch March 2, 2019 13:58
lichunzhu pushed a commit to lichunzhu/dm that referenced this pull request Apr 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
priority/normal Minor change, requires approval from ≥1 primary reviewer status/LGT2 Two reviewers already commented LGTM, ready for merge type/bug-fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants