Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

check block schema in exchange operator #1506

Merged
merged 9 commits into from
Mar 8, 2021
Merged

check block schema in exchange operator #1506

merged 9 commits into from
Mar 8, 2021

Conversation

windtalker
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #1500

Problem Summary:

The DAG request from TiDB sometimes has wrong field type, it will cause some unexpected problems (i.e. #1492, #1464)

What is changed and how it works?

Proposal: xxx

What's Changed:

How it Works:
Check block struct in exchange operator so all the schema mismatch error will throw the same error.

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note

  • check block struct in exchange operator

@windtalker windtalker added the type/enhancement The issue or PR belongs to an enhancement. label Mar 3, 2021
@windtalker
Copy link
Contributor Author

/run-all-tests

@windtalker
Copy link
Contributor Author

/run-all-tests

@windtalker windtalker changed the title check block struct in exchange operator check block schema in exchange operator Mar 4, 2021
fzhedu
fzhedu reviewed Mar 5, 2021
View reviewed changes
dbms/src/Flash/Coprocessor/CHBlockChunkCodec.cpp Outdated
@@ -43,10 +67,13 @@ void writeData(const IDataType & type, const ColumnPtr & column, WriteBuffer & o

void CHBlockChunkCodecStream::encode(const Block & block, size_t start, size_t end)
{
/// only check block schema in CHBlock codec because for both
/// Default codec and Arrow codec, it add implicit convert to
/// convert the input to the target output types.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clear comments pls.

fzhedu
fzhedu reviewed Mar 5, 2021
View reviewed changes
Copy link
Contributor

@fzhedu fzhedu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after fix the comments, rest LGTM

@windtalker
Copy link
Contributor Author

/run-all-tests

1 similar comment
@windtalker
Copy link
Contributor Author

/run-all-tests

@fzhedu fzhedu self-requested a review March 8, 2021 09:43
fzhedu
fzhedu approved these changes Mar 8, 2021
View reviewed changes
Copy link
Contributor

@fzhedu fzhedu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@ti-srebot
Copy link
Collaborator

@fzhedu, Thanks for your review. The bot only counts LGTMs from Reviewers and higher roles, but you're still welcome to leave your comments. See the corresponding SIG page for more information. Related SIG: tiflash(slack).

@windtalker windtalker added the status/can-merge Indicates a PR has been approved by a committer. label Mar 8, 2021
@ti-srebot
Copy link
Collaborator

/run-all-tests

@ti-srebot ti-srebot merged commit 32abb89 into pingcap:master Mar 8, 2021
@windtalker windtalker deleted the check_exchange_schema branch May 11, 2021 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tisonkun tisonkun tisonkun left review comments

@fzhedu fzhedu fzhedu approved these changes

Assignees
No one assigned
Labels
status/can-merge Indicates a PR has been approved by a committer. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Exchange operator should check the schema before sending/receiver data
4 participants
@windtalker @ti-srebot @fzhedu @tisonkun