Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multitenantccl/tenantcostserver: deflake TestDataDriven #93464

Merged
merged 1 commit into from
Dec 14, 2022

Conversation

ajwerner
Copy link
Contributor

Fixes #93462

The scope for the outer test instead of per file subtest was creating problems for leaktest.

Release note: None

Fixes cockroachdb#93462

The scope for the outer test instead of per file subtest was creating problems
for leaktest.

Release note: None
@ajwerner ajwerner requested a review from a team as a code owner December 12, 2022 20:42
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@renatolabs
Copy link
Contributor

Thanks!

To keep things documented (aka, I don't understand the fix): why is this the fix? I see other datadriven tests where the log.Scope(t).Close(t) call is outside the function passed to Walk, and that doesn't seem to cause problems. Moreover, this test hasn't really changed much recently.

I also started a bisection when I filed the issue, and apparently #92930 made this test flaky.

@ajwerner
Copy link
Contributor Author

From what I could tell, the goroutines were logging goroutines which get launched from here:

func ApplyConfig(config logconfig.Config) (logShutdownFn func(), err error) {

The way that leaktest works is that it makes sure that all goroutines which exist after the test are done existed before it started. The race here was that we launched these goroutines under log.Scope (here) but they had not yet started by the time leaktest picked its snapshot. So, by choosing to not start the logging goroutines until after leaktest picks its snapshot, we remove the hazard of the async goroutines having not yet started.

Copy link
Contributor

@renatolabs renatolabs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ajwerner
Copy link
Contributor Author

TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Dec 13, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Dec 13, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Dec 13, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Dec 14, 2022

Build succeeded:

@craig craig bot merged commit a2631a3 into cockroachdb:master Dec 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

multitenantccl: leaked goroutine in TestDataDriven/cleanup
3 participants