Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

distsql: dynamically adjust the number of distsql runners #84459

Closed
yuzefovich opened this issue Jul 14, 2022 · 0 comments · Fixed by #84946
Closed

distsql: dynamically adjust the number of distsql runners #84459

yuzefovich opened this issue Jul 14, 2022 · 0 comments · Fixed by #84946
Assignees
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team

Comments

@yuzefovich
Copy link
Member

yuzefovich commented Jul 14, 2022

Currently, we spin up 16 DistSQL runner goroutines whose job is to perform the SetupFlow gRPC call when setting up the distributed query plan. If that pool of 16 workers is used up, then the main goroutine of the query on the gateway sequentially performs those gRPC calls which will add to the query execution latency. I think it'd be worthwhile to introduce an ability to dynamically adjust this pool based on a cluster setting. Probably it'd be good to make the initial value also dependent on the number of CPUs on the node.

My current thinking is that we'd probably want to introduce a coordinator goroutine that would be notified whenever there is a change to the cluster setting. This coordinator would then cancel some workers when shrinking and spin up some new ones when growing the workers pool.

Jira issue: CRDB-17670

@yuzefovich yuzefovich added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Jul 14, 2022
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Jul 14, 2022
@yuzefovich yuzefovich self-assigned this Jul 14, 2022
@craig craig bot closed this as completed in 314baa5 Aug 1, 2022
@mgartner mgartner moved this to Done in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant