Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize load data assignment expressions #46081

Closed
gengliqi opened this issue Aug 14, 2023 · 0 comments
Closed

Optimize load data assignment expressions #46081

gengliqi opened this issue Aug 14, 2023 · 0 comments
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. sig/execution SIG execution type/enhancement The issue or PR belongs to an enhancement.

Comments

@gengliqi
Copy link
Contributor

gengliqi commented Aug 14, 2023 •
edited
Loading

Enhancement

image

According to the above CPU flame graph, most of the time spends on rewriteAstExpr.
The root cause is that the ASTs of assignment expressions in load data are rewritten for each row, which may make the load data super slow.

#43492 has already fixed this issue, but only the TiDB clusters whose versions are greater than or equal to v7.2 contain this pr. In addition, there are some PR for load data code refactoring in v7.1.

So a tiny fix for this issue is needed for the previous TiDB version.
@gengliqi gengliqi added the type/enhancement The issue or PR belongs to an enhancement. label Aug 14, 2023
@gengliqi gengliqi mentioned this issue Aug 14, 2023
Merged
12 tasks
@ti-chi-bot ti-chi-bot added affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. labels Aug 15, 2023
ti-chi-bot bot pushed a commit that referenced this issue Aug 15, 2023
@gengliqi
executor: optimize load data assignment expressions (#46082)
3144a46 
close #46081
@ti-chi-bot ti-chi-bot mentioned this issue Aug 15, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 15, 2023
@ti-chi-bot
executor: optimize load data assignment expressions (#46082) (#46110)
ab110b5 
close #46081
@windtalker windtalker mentioned this issue Aug 23, 2023
Merged
17 tasks
@lance6716 lance6716 mentioned this issue Aug 31, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 4, 2023
@lance6716
executor: optimize load data assignment expression (#46563)
6f1bdf7 
close #46081
@jebter jebter added the sig/execution SIG execution label Sep 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. sig/execution SIG execution type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jebter @gengliqi @ti-chi-bot