Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv/kvserver: TestStoreRangeGossipOnSplits failed #95566

Closed
cockroach-teamcity opened this issue Jan 20, 2023 · 3 comments · Fixed by #95579
Closed

kv/kvserver: TestStoreRangeGossipOnSplits failed #95566

cockroach-teamcity opened this issue Jan 20, 2023 · 3 comments · Fixed by #95579
Assignees
@kvoli
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-kv KV Team
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jan 20, 2023 •
edited by cockroach-jira-scripts
Loading

kv/kvserver.TestStoreRangeGossipOnSplits failed with artifacts on master @ 0775fcc3b3a71b47dc67a73f2caadbf4b586ec37:

=== RUN   TestStoreRangeGossipOnSplits
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/7334302ae8c22802de8334e77d6901c9/logTestStoreRangeGossipOnSplits4002577389
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2391: gossiped range count 55 more than 1 away from expected 81
    client_split_test.go:2397: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/7334302ae8c22802de8334e77d6901c9/logTestStoreRangeGossipOnSplits4002577389
--- FAIL: TestStoreRangeGossipOnSplits (0.38s)
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-23591
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Jan 20, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Jan 20, 2023
@blathers-crl blathers-crl bot added the T-kv KV Team label Jan 20, 2023
@kvoli
Copy link
Collaborator

kvoli commented Jan 20, 2023 •
edited
Loading

Possibly flaky due to #93945. This changed behavior on what triggers store gossip and how concurrent capacity change triggered store gossip events interact.

@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestStoreRangeGossipOnSplits failed with artifacts on master @ d698e5eaf36bee19901beeff7db3425315402907:

=== RUN   TestStoreRangeGossipOnSplits
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/7438ab096787a15a8ce5f502b5724798/logTestStoreRangeGossipOnSplits2746565145
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2391: gossiped range count 55 more than 1 away from expected 81
    client_split_test.go:2397: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/7438ab096787a15a8ce5f502b5724798/logTestStoreRangeGossipOnSplits2746565145
--- FAIL: TestStoreRangeGossipOnSplits (0.66s)
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestStoreRangeGossipOnSplits failed with artifacts on master @ b38da1d1c5da65cd6855e302296c578de05f0ab2:

=== RUN   TestStoreRangeGossipOnSplits
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeGossipOnSplits351835565
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2391: gossiped range count 55 more than 1 away from expected 81
    client_split_test.go:2397: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeGossipOnSplits351835565
--- FAIL: TestStoreRangeGossipOnSplits (0.26s)

Parameters: TAGS=bazel,gss

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@kvoli kvoli mentioned this issue Jan 20, 2023
Merged
@kvoli kvoli self-assigned this Jan 20, 2023
craig bot pushed a commit that referenced this issue Jan 20, 2023
@kvoli
Merge #95579
47d6f18 
95579: kvserver: fix store disable gossip lease override r=andrewbaptist a=kvoli

In #93945 the store gossip logic was refactored to take into account the relative difference rather than cumulative number of changes. This led to a bug where disabling lease capacity changes from triggering gossip wasn't adhered to in all cases.

This introduced a flake where `TestStoreRangeGossipOnSplits` and the underlying behavior being tested diverged slightly. This patch fixes the issue.

Fixes: #95566

Release note: None

Co-authored-by: Austen McClernon <[email protected]>
@craig craig bot closed this as completed in 171458a Jan 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kvoli kvoli

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-kv KV Team
Projects
None yet
Milestone
23.1
Development

Successfully merging a pull request may close this issue.

kvserver: fix store disable gossip lease override kvoli/cockroach
2 participants
@cockroach-teamcity @kvoli