Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only (1.7% qps on oltp_read_write) #135898

Closed
tbg opened this issue Nov 21, 2024 · 2 comments · Fixed by #137072
Closed
Assignees
Labels
branch-master Failures and bugs on the master branch. C-performance Perf of queries or internals. Solution not expected to change functional behavior. o-perf-efficiency Related to performance efficiency P-2 Issues/test failures with a fix SLA of 3 months T-sql-queries SQL Queries Team

Comments

@tbg
Copy link
Member

tbg commented Nov 21, 2024

In an ad-hoc experiment, changing sql.defaults.distsql to off resulted in a 7.5% improvement in qps, from around 40k to around 43k.1

DistSQL is likely used for the sysbench "range" queries, which scan 100 consecutive keys. It likely makes no sense to use DistSQL here, as the overhead does not get amortized. There are no unbounded scans here either.

Epic: CRDB-42584

Jira issue: CRDB-44780

Footnotes

  1. https://cockroachlabs.slack.com/archives/C07P0CNLP51/p1732192681032459?thread_ts=1732092976.359539&cid=C07P0CNLP51

@tbg tbg added C-performance Perf of queries or internals. Solution not expected to change functional behavior. branch-master Failures and bugs on the master branch. P-2 Issues/test failures with a fix SLA of 3 months o-perf-efficiency Related to performance efficiency P-1 Issues/test failures with a fix SLA of 1 month and removed P-2 Issues/test failures with a fix SLA of 3 months labels Nov 21, 2024
@exalate-issue-sync exalate-issue-sync bot added P-2 Issues/test failures with a fix SLA of 3 months and removed P-1 Issues/test failures with a fix SLA of 1 month labels Nov 21, 2024
@tbg tbg changed the title sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only (conjecture: 3.5% cpu on oltp_read_write) Nov 21, 2024
@mgartner mgartner added the T-sql-queries SQL Queries Team label Nov 21, 2024
@github-project-automation github-project-automation bot moved this to Triage in SQL Queries Nov 21, 2024
@michae2
Copy link
Collaborator

michae2 commented Nov 26, 2024

[sql queries triage] We think the aggregation forces us to choose a distributed plan. We could adjust physical planning heuristics, but this is tricky. We could introduce a row-count threshold to decide whether to distribute, along with a cluster setting.

We've also talked about moving the decision of whether to distribute into the optimizer (#47226), but that isn't a quick fix.

Putting into v25.1 to think about whether there is a quick fix.

@michae2 michae2 moved this from Triage to 25.1 Release in SQL Queries Nov 26, 2024
@ajstorm
Copy link
Collaborator

ajstorm commented Nov 27, 2024

I'm only seeing a 1.7% benefit in my oltp_read_write tests. Revising this estimate downward for now.

@ajstorm ajstorm changed the title sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only (conjecture: 3.5% cpu on oltp_read_write) sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only (1.7% cpu on oltp_read_write) Nov 27, 2024
@ajstorm ajstorm changed the title sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only (1.7% cpu on oltp_read_write) sql: turning off DistSQL gives 7.5% qps boost on oltp_read_only (1.7% qps on oltp_read_write) Nov 27, 2024
@yuzefovich yuzefovich moved this from 25.1 Release to Active in SQL Queries Dec 10, 2024
@craig craig bot closed this as completed in 7575548 Dec 11, 2024
@github-project-automation github-project-automation bot moved this from Active to Done in SQL Queries Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-performance Perf of queries or internals. Solution not expected to change functional behavior. o-perf-efficiency Related to performance efficiency P-2 Issues/test failures with a fix SLA of 3 months T-sql-queries SQL Queries Team
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants