Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvserver: TestPromoteNonVoterInAddVoter flaked on release-23.1 #105883

Closed
erikgrinaker opened this issue Jun 30, 2023 · 6 comments · Fixed by #106514
Closed

kvserver: TestPromoteNonVoterInAddVoter flaked on release-23.1 #105883

erikgrinaker opened this issue Jun 30, 2023 · 6 comments · Fixed by #106514
Assignees
@kvoli
Labels
A-testing Testing tools and infrastructure branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). T-kv KV Team

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Jun 30, 2023 •
edited by cockroach-jira-scripts
Loading

Seen in a CI failure.

=== RUN   TestPromoteNonVoterInAddVoter
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestPromoteNonVoterInAddVoter2438547430
    test_log_scope.go:79: use -show-logs to present logs inline
    replicate_queue_test.go:2248: condition failed to evaluate within 3m45s: Range counts are not all equal
    panic.go:540: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestPromoteNonVoterInAddVoter2438547430
--- FAIL: TestPromoteNonVoterInAddVoter (462.14s)

Jira issue: CRDB-29255
@erikgrinaker erikgrinaker added C-test-failure Broken test (automatically or manually discovered). T-kv KV Team branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 labels Jun 30, 2023
@kvoli
Copy link
Collaborator

kvoli commented Jun 30, 2023

Seems different from #101519. I should open a backport for that anyway.

@kvoli kvoli mentioned this issue Jun 30, 2023
Merged
@kvoli kvoli mentioned this issue Jul 10, 2023
Merged
@kvoli
Copy link
Collaborator

kvoli commented Jul 10, 2023

This will be inherently flaky under race/stress. The test requires the range count to be balanced initially. This isn't the behavior the test is interested in.

Opened a PR to skip under race.

@mgartner mgartner mentioned this issue Jul 10, 2023
Closed
@kvoli kvoli mentioned this issue Jul 10, 2023
Merged
@tbg
Copy link
Member

tbg commented Jul 11, 2023

@kvoli don't we need this on master as well? I just hit this on master here: https://teamcity.cockroachdb.com/viewLog.html?buildId=10865562&tab=buildResultsDiv&buildTypeId=Cockroach_Ci_Tests_Testrace

@kvoli
Copy link
Collaborator

kvoli commented Jul 11, 2023

@kvoli don't we need this on master as well? I just hit this on master here: teamcity.cockroachdb.com/viewLog.html?buildId=10865562&tab=buildResultsDiv&buildTypeId=Cockroach_Ci_Tests_Testrace

#106514 is the master PR. I'll merge today.

craig bot pushed a commit that referenced this issue Jul 11, 2023
@kvoli @annrpom
Merge #106514 #106615
27e36a0 
106514: kvserver: skip promote non voter under race r=erikgrinaker a=kvoli

TestPromoteNonVoterInAddVoter requires that the initial range count of each store is balanced. Under stress/race builds this takes much longer and may time out before achieved.

TestPromoteNonVoterInAddVoter was already skipped under stress, also skip the test under race.

Resolves: #105883

Release note: None

106615: authors: add Annie Pompa to authors r=<andyyang890> a=annrpom

Release note: None
Epic: None

Co-authored-by: Austen McClernon <[email protected]>
Co-authored-by: Annie Pompa <[email protected]>
@craig craig bot closed this as completed in 077db91 Jul 11, 2023
@blathers-crl blathers-crl bot mentioned this issue Jul 11, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Jul 11, 2023
@kvoli
kvserver: skip promote non voter under race
fa79899 
TestPromoteNonVoterInAddVoter requires that the initial range count of
each store is balanced. Under stress/race builds this takes much longer
and may time out before achieved.

TestPromoteNonVoterInAddVoter was already skipped under stress, also
skip the test under race.

Resolves: #105883

Release note: None
@kvoli kvoli reopened this Jul 11, 2023
@kvoli
Copy link
Collaborator

kvoli commented Jul 11, 2023

Will be closed on #106628

@erikgrinaker
Copy link
Contributor Author

Resolved by #106628.

@tbg tbg added A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kvoli kvoli

Labels
A-testing Testing tools and infrastructure branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). T-kv KV Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvserver: skip promote non voter under race kvoli/cockroach
3 participants
@erikgrinaker @tbg @kvoli