Fix flaky test - TestAuditOn

[Joerger]

There is a race condition between the UploadCompleter and Uploader. If both attempt to upload the same file, and Uploader loses, then Uploader will return an error.

This race condition became possible with the replacement of the 24 hour grace period in the UploadCompleter with the session tracker - #11551.

I'm not yet sure how to remove the race condition entirely, but this PR will make it much much more unlikely to unblock integration tests in CI.

This PR reduces the likelihood of this race condition by decoupling the UploadCompleter and Uploader, so that the UploacCompleter will always run on a ten minute interval as opposed to the 5 second interval of Uploaders.

Additionally, the proxy forwarding server does not (before this PR) use the session tracker service, so the UploadCompleter could not see whether an upload's session was active and would always try to upload session recordings prematurely.

[zmb3]

Remind me again, what's the difference between Uploader and UploadCompleter? I wonder if we can rename one of them to make things more obvious.

[jakule]

Looks like now TestUploadCompleterCompletesAbandonedUploads is failing. Is this another flaky test or you think it's related to this change?

FAIL: github.com/gravitational/teleport/lib/events.TestUploadCompleterCompletesAbandonedUploads
===================================================
OUTPUT github.com/gravitational/teleport/lib/events.TestUploadCompleterCompletesAbandonedUploads
===================================================
=== RUN   TestUploadCompleterCompletesAbandonedUploads
2022-04-20T01:53:21Z DEBU [AUTH:COMP] Upload e56b79d3-c13d-4c5a-a096-61204e4367aa was abandoned, trying to complete. events/complete.go:163
2022-04-20T01:53:21Z DEBU [AUTH:COMP] Completed upload Upload(session=0be77e76-a95e-4abe-ada5-debba0133269, id=e56b79d3-c13d-4c5a-a096-61204e4367aa). events/complete.go:167
2022-04-20T01:53:21Z DEBU [AUTH:COMP] Found 1 active uploads, completed 1. events/complete.go:144
    complete_test.go:74: 
        	Error Trace:	complete_test.go:74
        	Error:      	Should be false
        	Test:       	TestUploadCompleterCompletesAbandonedUploads
--- FAIL: TestUploadCompleterCompletesAbandonedUploads (0.00s)

[russjones]

@Joerger In my opinion, it's worth spending the extra week or two to eliminate flakiness.

We can pay the cost for 1-2 weeks of dedicated dev time or months of time in aggregate across the team.

@r0mant @zmb3 What do you think?

[Joerger]

@zmb3 @jakule This is passing CI now, can you review again?