Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allocator: deflake full disk test #100589

Merged

Conversation

kvoli
Copy link
Collaborator

@kvoli kvoli commented Apr 4, 2023

In #97409 we introduced cluster settings to control the disk fullness threshold for rebalancing towards a store and shedding replicas off of the store. The TestAllocatorFullDisks assumes the total number of range bytes is equal or less than the rebalance threshold of the nodes, however the test was updated to use the shed threshold instead. This caused the test to flake occasionally as there was more than the expected amount of total range bytes.

This patch changes the ranges per node calculation to use the rebalance threshold again, instead of the shed threshold

dev test pkg/kv/kvserver/allocator/allocatorimpl -f TestAllocatorFullDisks -v --stress
...
15714 runs so far, 0 failures, over 39m45s

Fixes: #100033

Release note: None

@kvoli kvoli added the backport-23.1.x Flags PRs that need to be backported to 23.1 label Apr 4, 2023
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@kvoli kvoli self-assigned this Apr 4, 2023
In cockroachdb#97409 we introduced cluster settings to control the disk fullness
threshold for rebalancing towards a store and shedding replicas off of
the store. The `TestAllocatorFullDisks` assumes the total number of
range bytes is equal or less than the rebalance threshold of the nodes,
however the test was updated to use the shed threshold instead. This
caused the test to flake occasionally as there was more than the
expected amount of total range bytes.

This patch changes the ranges per node calculation to use the rebalance
threshold again, instead of the shed threshold

Fixes: cockroachdb#100033

Release note: None
@kvoli kvoli force-pushed the 230404.deflake-allocator-full-disks branch from c384560 to 3b94693 Compare April 4, 2023 14:36
@kvoli kvoli marked this pull request as ready for review April 4, 2023 15:19
@kvoli kvoli requested a review from a team as a code owner April 4, 2023 15:19
Copy link
Collaborator

@andrewbaptist andrewbaptist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@kvoli
Copy link
Collaborator Author

kvoli commented Apr 4, 2023

TYFTR

bors r=andrewbaptist

@craig
Copy link
Contributor

craig bot commented Apr 4, 2023

Build succeeded:

@craig craig bot merged commit 76956df into cockroachdb:master Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.1.x Flags PRs that need to be backported to 23.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kv/kvserver/allocator/allocatorimpl: TestAllocatorFullDisks failed
4 participants