Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-22.2: kvserver: deflake test store capacity after split #112317

Merged
merged 1 commit into from
Oct 13, 2023

Conversation

kvoli
Copy link
Collaborator

@kvoli kvoli commented Oct 13, 2023

Backport 1/1 commits from #98515.

/cc @cockroachdb/release


This commit defales TestStoreCapacityAfterSplit. Previously it was possible for the replica load stats which underpins Capacity to be reset. The reset caused the recording duration to fall short of min stats duration, which led to a 0 value being reported for writes in store capacity.

This commit bumps the manual clock twice and removes redundant leaseholder checks within a retry loop. The combination of these two changes makes the test much less likely to flake.

The test is now unskipped.

dev test pkg/kv/kvserver -f TestStoreCapacityAfterSplit -v --stress
...
4410 runs so far, 0 failures, over 6m10s

Resolves: #92677
Release note: None


Fixes: #111738
Release justification: Test only change.

This commit deflakes `TestStoreCapacityAfterSplit`. Previously it was
possible for the replica load stats which underpins Capacity to be
reset. The reset caused the recording duration to fall short of min
stats duration, which led to a 0 value being reported for writes in
store capacity.

This commit bumps the manual clock twice and removes redundant
leaseholder checks within a retry loop. The combination of these two
changes makes the test much less likely to flake.

The test is now unskipped.

```
dev test pkg/kv/kvserver -f TestStoreCapacityAfterSplit -v --stress
...
4410 runs so far, 0 failures, over 6m10s
```

Resolves: cockroachdb#92677

Release note: None
@kvoli kvoli self-assigned this Oct 13, 2023
@blathers-crl
Copy link

blathers-crl bot commented Oct 13, 2023

Thanks for opening a backport.

Please check the backport criteria before merging:

  • Patches should only be created for serious issues or test-only changes.
  • Patches should not break backwards-compatibility.
  • Patches should change as little code as possible.
  • Patches should not change on-disk formats or node communication protocols.
  • Patches should not add new functionality.
  • Patches must not add, edit, or otherwise modify cluster versions; or add version gates.
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
  • There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
  • The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
  • New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters.
  • The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.

Add a brief release justification to the body of your PR to justify this backport.

Some other things to consider:

  • What did we do to ensure that a user that doesn’t know & care about this backport, has no idea that it happened?
  • Will this work in a cluster of mixed patch versions? Did we test that?
  • If a user upgrades a patch version, uses this feature, and then downgrades, what happens?

@blathers-crl blathers-crl bot added the backport Label PR's that are backports to older release branches label Oct 13, 2023
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@kvoli
Copy link
Collaborator Author

kvoli commented Oct 13, 2023

Acceptance failure looks unrelated:

unknown input: {"Time":"2023-10-13T10:50:06.447131085-04:00","Action":"start","Package":"github.com/cockroachdb/cockroach/pkg/acceptance","Test":"PackageLevel","Elapsed":0,"Output":""}

@kvoli kvoli marked this pull request as ready for review October 13, 2023 17:59
@kvoli kvoli requested a review from a team as a code owner October 13, 2023 17:59
@kvoli kvoli requested review from erikgrinaker and andrewbaptist and removed request for erikgrinaker October 13, 2023 18:01
Copy link
Collaborator

@andrewbaptist andrewbaptist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@kvoli
Copy link
Collaborator Author

kvoli commented Oct 13, 2023

TYFTR

@kvoli kvoli merged commit b6a02cb into cockroachdb:release-22.2 Oct 13, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Label PR's that are backports to older release branches
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants