Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

intentresolver: fix test flake for CleanupIntents #35273

Merged
merged 1 commit into from
Mar 6, 2019

Conversation

ajwerner
Copy link
Contributor

The test before was brittle and assumed that batching was always triggered on
size rather than on timing. This mean that the test would fail if something
slowed test execution even though there was no correctness violation. This PR
makes the test less brittle by allowing intents to be resolved in an arbitrary
number of batches. This change has the nice side effect of allowing the wait
time for the requestbatcher to be shortened.

Fixes #35166.

Release note: None

@ajwerner ajwerner requested a review from a team February 28, 2019 18:21
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: once we get rid of the panics by capturing testing.T objects in the send func closures. If we run into issues with t.Fataling on a different goroutine, we can switch the assertions over to using the assert package.

Reviewed 1 of 1 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner)


pkg/storage/intentresolver/intent_resolver_test.go, line 821 at r1 (raw file):

	f = func(ba roachpb.BatchRequest) (*roachpb.BatchResponse, *roachpb.Error) {
		if remaining := atomic.LoadInt64(&toPush); len(ba.Requests) > int(remaining) {
			panic(fmt.Errorf("expected at most %d PushTxnRequests in batch, got %d",

Panicking causes some pretty bad test output (#35166, #35340). Can we plumb a *testing.T in here (and elsewhere) and t.Fatal?

The test before was brittle and assumed that batching was always triggered on
size rather than on timing. This mean that the test would fail if something
slowed test execution even though there was no correctness violation. This PR
makes the test less brittle by allowing intents to be resolved in an arbitrary
number of batches. This change has the nice side effect of allowing the wait
time for the requestbatcher to be shortened.

Fixes cockroachdb#35166.

Release note: None
@ajwerner ajwerner force-pushed the ajwerner/make-test-less-flakey branch from bb3566c to e736216 Compare March 6, 2019 15:22
Copy link
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @nvanbenschoten)


pkg/storage/intentresolver/intent_resolver_test.go, line 821 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Panicking causes some pretty bad test output (#35166, #35340). Can we plumb a *testing.T in here (and elsewhere) and t.Fatal?

I'll admit the panics were lazy. I plumbed a testing.T through to the sendFuncs.

@ajwerner
Copy link
Contributor Author

ajwerner commented Mar 6, 2019

bors r+

craig bot pushed a commit that referenced this pull request Mar 6, 2019
35273: intentresolver: fix test flake for CleanupIntents r=ajwerner a=ajwerner

The test before was brittle and assumed that batching was always triggered on
size rather than on timing. This mean that the test would fail if something
slowed test execution even though there was no correctness violation. This PR
makes the test less brittle by allowing intents to be resolved in an arbitrary
number of batches. This change has the nice side effect of allowing the wait
time for the requestbatcher to be shortened.

Fixes #35166.

Release note: None

Co-authored-by: Andrew Werner <[email protected]>
@craig
Copy link
Contributor

craig bot commented Mar 6, 2019

Build succeeded

@craig craig bot merged commit e736216 into cockroachdb:master Mar 6, 2019
ajwerner added a commit to ajwerner/cockroach that referenced this pull request Mar 12, 2019
Before this change we'd assume that two requests addressed to the same range
would end up in the same batch. This patch adopts the more flexible logic
introduced in cockroachdb#35273 to resolve similar test flakes in TestCleanupIntents.
This new infrastructure behaves correctly when batches are sent due to time
with less batching than the test assumed.

Fixes cockroachdb#35340.

Release note: None
craig bot pushed a commit that referenced this pull request Mar 13, 2019
35667: intentresolver: fix test flake due to timing of batches r=ajwerner a=ajwerner

Before this change we'd assume that two requests addressed to the same range
would end up in the same batch. This patch adopts the more flexible logic
introduced in #35273 to resolve similar test flakes in TestCleanupIntents.
This new infrastructure behaves correctly when batches are sent due to time
with less batching than the test assumed.

Fixes #35340.

Release note: None

Co-authored-by: Andrew Werner <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

teamcity: failed test: TestCleanupIntents
3 participants