Add storage for TestSuitesSource

[nulls]

• storage for test suites source
• combined standard and git test suites

[github-advanced-security]

Found 72 potential problems in the proposed changes. Check the Files changed tab for more details.

[github-advanced-security]

Found 71 potential problems in the proposed changes. Check the Files changed tab for more details.

[github-advanced-security]

Found 71 potential problems in the proposed changes. Check the Files changed tab for more details.

[github-advanced-security]

Found 71 potential problems in the proposed changes. Check the Files changed tab for more details.

[github-advanced-security]

Found 83 potential problems in the proposed changes. Check the Files changed tab for more details.

[github-advanced-security]

Found 81 potential problems in the proposed changes. Check the Files changed tab for more details.

[github-advanced-security]

Found 81 potential problems in the proposed changes. Check the Files changed tab for more details.

[orchestr7]

lgtm. No chance that we can properly review this PR, so probably we need to merge it

[orchestr7]

Overview

Introduced a new entity TestSuitesSource. It's a source to fetch and detect test suites and tests. TestSuitesSource describes only where tests are placed A snapshot of TestSuitesSource is a zip archive of files are pulled from source with some specific version.

So, TestSuitesSource is git credentials (Git entity) + branch + relative path in repo. Version is sha1 or hash commit in git repository.

We store snapshots on backend side (it saves them to file system), it will be moved to S3 in future.

The main goal of this task is to add storage for TestSutiesSource: we split fetching test suites with files and run them. Currently, the UI is not changed -- user still can choice standard tests suites or provide git url for personal tests.

For seamless transition, we create TestSuitesSource on the fly. User inputs git-url, git-branch and testRootPath and we search source with such values or create a new one (the same approach for standard test suites, values are hardcoded)

How run looks now:

Standard test suites

They still fetch periodically by preprocessor. We detect test suites and save them in backend + save snapshot of repository

When user runs a new run on standard test suites, we take the latest available version on backend (max by creation time of commit) and run common run using this version

Git test suites

User can input branch or commit\hash\sha1. If user inputs branch, we search or create a new test source for it and then detect a latest commit in git repository. Then we fetch test suites for this version and then trigger a common run using this version If user inputs version, we fetch test suites for this version and then trigger a common run using this version

Common run

We put selected testSuiteIds on execution and execute tests for them (create TestExecution), then initialize agent. Orchestrator downloads additional files (binary for evaluated tools) + test suites sources to a tmp folder. Then orchestrator creates volume using this folder.

Orchestrator download logic

Orchestrator create a tmp directory in format /save-execution-{id}. Then create a sub folder '/save-execution-{id}/test-suites' and download additional files there. Orchestrator download all test suites source snapshot to folder with name in format /save-execution-{id}/test-suites/test-suites-source-{index} (index can be changed to test suites source id in future to simplify debugging). Then orchestrator creates a synthecit toml config in /save-execution-{id}/test-suites/save.toml and run save cli with parameter --include-suites

This should be in some developer's readme 😄
Or inside the code - for example as a Kdoc to some of your classes

[github-advanced-security]

Found 11 potential problems in the proposed changes. Check the Files changed tab for more details.