Parallelise system tests

[patchwork01]

Description

We could improve the system tests by running multiple tests in parallel, with each test running in its own Sleeper table in the same instance.

This could make the test suite significantly faster to run. It could also be a bit more representative of real use.

Ideally this could run constantly so that whenever we push a commit we would have feedback based on a real deployed instance within an hour. We can investigate the costs incurred by this. This may be impacted by running the tests in parallel, particularly if tasks and instances are constantly coming up and down if the tests are not run in parallel.

This will also be impacted by any tools we have to force scaling, add tasks for compaction/ingest and keep them around to avoid cold starts, including the following:

• Configure waiting for compaction jobs #1796
• Manually start compaction tasks #1798

Here are other issues that are related or required for this:

• Allow using the same file name in simultaneous system tests #1854
• Take system test tables offline after test instead of deleting #3705
• Stop purging queues after a system test #3876
• Enable schedule rules by default in system tests #3722
• Thread safety when interacting with system test deployment in system tests #3947
• Thread safety when connecting to instances in system tests #3775