Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test - TestChaosUpload #12043

Merged
merged 7 commits into from
Apr 19, 2022
Merged

Fix flaky test - TestChaosUpload #12043

merged 7 commits into from
Apr 19, 2022

Conversation

Joerger
Copy link
Contributor

@Joerger Joerger commented Apr 18, 2022

Removes call to CheckUploads which was out of place in TestChaosUpload. CheckUploads fails when called concurrently with other calls to CheckUploads and isn't needed in this test.

Before #11551, we didn't see this flaky tests because CheckUploads would always be a noop since no upload would have surpassed the 24 hour grace period within the 2 second test, whereas now CheckUploads is never a noop since the test session uploads aren't tracked by a session tracker.

Fixes the following flaky errors in TestChaosUpload

--- FAIL: TestChaosUpload (0.07s)
    fileasync_chaos_test.go:191: 
        	Error Trace:	fileasync_chaos_test.go:191
        	Error:      	Received unexpected error:
        	           	lock /tmp/teleport-streams1999935953/cc67bbf2-ca87-4176-9547-b9ebe2d5d7d7.tar is acquired by another process
        	Test:       	TestChaosUpload
--- FAIL: TestChaosUpload (0.11s)
    fileasync_chaos_test.go:221: 
        	Error Trace:	fileasync_chaos_test.go:221
        	Error:      	Not equal: 
        	           	expected: 4098
        	           	actual  : 0
        	Test:       	TestChaosUpload
        	Messages:   	event: {aba8002b-345e-413f-a2b5-9fc2d8395685 331c82a6-80b2-40ae-ab8b-9f601f0f93ca <nil> 0001-01-01 00:00:00 +0000 UTC}
--- FAIL: TestChaosUpload (0.15s)
    fileasync_chaos_test.go:202: 
        	Error Trace:	fileasync_chaos_test.go:202
        	Error:      	Received unexpected error:
        	           	lock /tmp/teleport-streams3694352958/fcd138af-26e8-431e-8f0f-4f8bd24c5a06.tar is acquired by another process
        	Test:       	TestChaosUpload
        	Messages:   	
        	           	ERROR REPORT:
        	           	Original Error: *trace.CompareFailedError lock /tmp/teleport-streams3694352958/fcd138af-26e8-431e-8f0f-4f8bd24c5a06.tar is acquired by another process
        	           	Stack Trace:
        	           		/home/bjoerger/gravitational/teleport/lib/utils/fs_unix.go:44 github.com/gravitational/teleport/lib/utils.FSTryWriteLock
        	           		/home/bjoerger/gravitational/teleport/lib/events/filesessions/filestream.go:114 github.com/gravitational/teleport/lib/events/filesessions.(*Handler).CompleteUpload
        	           		/home/bjoerger/gravitational/teleport/lib/events/complete.go:163 github.com/gravitational/teleport/lib/events.(*UploadCompleter).CheckUploads
        	           		/home/bjoerger/gravitational/teleport/lib/events/filesessions/fileasync_chaos_test.go:177 github.com/gravitational/teleport/lib/events/filesessions.TestChaosUpload.func5
        	           		/home/bjoerger/.tools/go/src/runtime/asm_amd64.s:1581 runtime.goexit
        	           	User Message: lock /tmp/teleport-streams3694352958/fcd138af-26e8-431e-8f0f-4f8bd24c5a06.tar is acquired by another process

@github-actions github-actions bot added the audit-log Issues related to Teleports Audit Log label Apr 18, 2022
@Joerger Joerger requested review from zmb3 and r0mant April 18, 2022 20:05
Copy link
Collaborator

@zmb3 zmb3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracking this down.

@Joerger Joerger enabled auto-merge (squash) April 18, 2022 21:14
@Joerger Joerger merged commit 0d702e6 into master Apr 19, 2022
@Joerger Joerger deleted the joerger/flaky-tests-uploader branch April 19, 2022 00:40
Joerger added a commit that referenced this pull request Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audit-log Issues related to Teleports Audit Log backport-required flaky tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants