-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
lightning: pick the first file to check schema #27607
Conversation
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
/component lightning |
/sig migrate |
"reflect" | ||
"sort" | ||
"strconv" | ||
"strings" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
? move them back
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
694ec9b
to
70a20dc
Compare
Co-authored-by: glorv <[email protected]>
/label needs-cherry-pick-5.2 |
/merge |
This pull request has been accepted and is ready to merge. Commit hash: 772016b
|
Signed-off-by: ti-srebot <[email protected]>
cherry pick to release-5.2 in PR #27623 |
Is it safe to check schema for one random file? cc @lonng |
In the common case that the source files are exported by mydumper/dumpling or other similar tools, source files' schema should be the same, so only check one scheme should be enough. In other case, e.g. user manually created source files with different columns, this check may not find potential error and lightning will still raise an error after parse the incompatible file. Since this kind of situation is rare, so I think this compromise between performance and correctness is ok. cc @IANTHEREAL |
What problem does this PR solve?
Issue Number: close #27605
Problem Summary:
check all files costs too much time.
What is changed and how it works?
What's Changed:
pick one random file to check schema is valid.
Check List
Tests
Release note