Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workload: ycsb-e creates a single hot range #87279

Closed
kvoli opened this issue Sep 1, 2022 · 1 comment · Fixed by #89217
Closed

workload: ycsb-e creates a single hot range #87279

kvoli opened this issue Sep 1, 2022 · 1 comment · Fixed by #89217
Assignees
Labels
A-kv-distribution Relating to rebalancing and leasing. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-kv KV Team

Comments

@kvoli
Copy link
Collaborator

kvoli commented Sep 1, 2022

Describe the problem

Running the YCSB-E workload, one range has 99% of the request load and remains above the split threshold, despite not splitting.

image

The range appears to have sufficient rows to split:

image

 /Table/106/1 to /Max

This causes a hotspot to develop on the leaseholder for this range. Which does not dissipate.

To Reproduce

export gce_project=cockroach-ephemeral
export email=austen-ycsbe
roachprod create $email -n 8 --gce-machine-type=n1-standard-16

# recent commit on master
roachprod stage $email cockroach 4f0065ad010cee1ce50bbc65019b26f93a7dc6ad
roachprod start $email:1-7

# stimulate some rebalancing activity
roachprod run $email:8 -- './cockroach workload init ycsb --workload E --drop {pgurl:1-7}'
roachprod run $email:8 -- './cockroach workload run ycsb --max-rate 50000 --workload E {pgurl:1-7}'

Jira issue: CRDB-19250

@kvoli kvoli added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Sep 1, 2022
@blathers-crl blathers-crl bot added the T-kv KV Team label Sep 23, 2022
@kvoli kvoli added the A-kv-distribution Relating to rebalancing and leasing. label Sep 23, 2022
@kvoli
Copy link
Collaborator Author

kvoli commented Sep 23, 2022

observability to diagnose this issue #87276

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-distribution Relating to rebalancing and leasing. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-kv KV Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants