Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: reuse chunk for GetMatchedRowsAndPtrs calls (#48090) #48103

Merged

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #48090

What problem does this PR solve?

Issue Number: close #48082
Problem Summary:

What is changed and how it works?

Reuse chunk for GetMatchedRowsAndPtrs.
Add a new benchmark case.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    BenchmarkHashJoinExec
Old:
 go test -benchmem -run=^$ -bench ^BenchmarkHashJoinExec$ github.com/pingcap/tidb/pkg/executor
goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/pkg/executor
cpu: 13th Gen Intel(R) Core(TM) i9-13900KF
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                   2         625585277 ns/op        2428636920 B/op   512892 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                    19          64306968 ns/op        278193261 B/op    112877 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                      2         510499472 ns/op        3491117756 B/op  2213412 allocs/op
BenchmarkHashJoinExec/(rows:5,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                             26020             48058 ns/op           87853 B/op        934 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                         79          16758415 ns/op        15902526 B/op     512736 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                           92          15112195 ns/op        12524965 B/op     112666 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                                    9         123780841 ns/op        894195201 B/op   1714292 allocs/op


New:
go test -benchmem -run=^$ -bench ^BenchmarkHashJoinExec$ github.com/pingcap/tidb/pkg/executor                      
goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/pkg/executor
cpu: 13th Gen Intel(R) Core(TM) i9-13900KF
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                   2         627044000 ns/op        2428678160 B/op   512971 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                    18          63483060 ns/op        278192079 B/op    112888 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                      2         502814122 ns/op        3639970448 B/op  1216829 allocs/op
BenchmarkHashJoinExec/(rows:5,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                             28047             48197 ns/op           87847 B/op        934 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                         70          16853512 ns/op        15902144 B/op     512739 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                           82          14872368 ns/op        12524746 B/op     112660 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                                   22          53258487 ns/op        40282174 B/op    1113179 allocs/op

Profile for the new case: left is the master, right is this pr.
image

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-6.5 This PR is cherry-picked to release-6.5 from a source PR. labels Oct 30, 2023
Signed-off-by: Yang Keao <[email protected]>
@YangKeao YangKeao force-pushed the cherry-pick-48090-to-release-6.5 branch from a698876 to 98f0baf Compare October 31, 2023 07:05
Copy link
Member

@YangKeao YangKeao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Oct 31, 2023
@YangKeao YangKeao requested a review from wshwsh12 October 31, 2023 07:07
@ti-chi-bot ti-chi-bot added the cherry-pick-approved Cherry pick PR approved by release team. label Nov 2, 2023
Copy link

ti-chi-bot bot commented Nov 2, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lcwangchao, YangKeao

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [YangKeao,lcwangchao]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 2, 2023
Copy link

ti-chi-bot bot commented Nov 2, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-31 07:05:44.720313392 +0000 UTC m=+2936742.307423537: ☑️ agreed by YangKeao.
  • 2023-11-02 09:13:17.625950945 +0000 UTC m=+3117195.213061353: ☑️ agreed by lcwangchao.

@lcwangchao
Copy link
Collaborator

/retest

Copy link

codecov bot commented Nov 3, 2023

Codecov Report

❗ No coverage uploaded for pull request base (release-6.5@62766de). Click here to learn what that means.
The diff coverage is n/a.

Additional details and impacted files
@@               Coverage Diff                @@
##             release-6.5     #48103   +/-   ##
================================================
  Coverage               ?   73.6524%           
================================================
  Files                  ?       1085           
  Lines                  ?     348799           
  Branches               ?          0           
================================================
  Hits                   ?     256899           
  Misses                 ?      75421           
  Partials               ?      16479           

@ti-chi-bot ti-chi-bot bot merged commit 9e55944 into pingcap:release-6.5 Nov 3, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved cherry-pick-approved Cherry pick PR approved by release team. lgtm release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-6.5 This PR is cherry-picked to release-6.5 from a source PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants