Use 3rd party task to run integration tests on external service

[tlrx]

Today the Gradle check task is executed on CI by third party tests jobs in order to test the repository plugins using real external storage services.

When the snapshot clean up tool was released in 7.0, the new s3ThirdPartyTests and gcsThidPartyTests tasks were created so that CI can run both the repository plugins integration tests and the snapshot tool integration tests on external services. It seems that these tasks never really made it to CI though.

Today we'd like to also run the searchable snapshot integration tests on external storage services. A previous change on CI jobs that runs the check tasks of repository plugins and searchable snapshot module was introduced but it broke CI builds on branches older than 7.8 (see #56165) and @mark-vieira suggested to use a specific tasks for these tests.

This pull request:

• adds a new azureThirdPartyTests task to the :plugins:repository-azure so that it runs the plugin's integration tests on Azure external service
• changes the searchable snapshot Gradle project so that it executes its appropriate integration tests using the s3ThirdPartyTests, gcsThirdPartyTests and azureThirdPartyTests tasks.

The third party tests CI jobs will be modified to execute the appropriate *ThirdPartyTests.

This pull request will also be backported on the following branches:

• 7.x (Use 3rd party task to run integration tests on external service #56588)
• 7.8 (Use 3rd party task to run integration tests on external service #56589)
• 7.7 (Use 3rd party task to run integration tests on external service #56590)
• 7.6 (Use 3rd party task to run integration tests on external service #56591)
• 6.8 (Use 3rd party task to run integration tests on external service #56592)

Closes #56165

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore)

[mark-vieira]

LGTM 👍

[rjernst]

Thanks for working on fixing this @tlrx. I have a couple questions/suggestions:

• Is the reason we can't just use thirdPartyTest naming because we have separate jobs in CI for each external service? eg we don't want a massive job that has creds for both azure and s3?
• If the above is correct, why do we need to keep thirdPartyTest at all? Could we just rename it in each project appropriately instead of adding another task?
• nit on naming: if we keep a separate task, I think it should be singular, as the rest of our test tasks are named

[mark-vieira]

Is the reason we can't just use thirdPartyTest naming because we have separate jobs in CI for each external service? eg we don't want a massive job that has creds for both azure and s3?

Correct.

If the above is correct, why do we need to keep thirdPartyTest at all? Could we just rename it in each project appropriately instead of adding another task?

I think we can remove it. We don't have any CI jobs which run all third party tests and I'm not sure if folks ever do locally.

[tlrx]

Thanks @rjernst and @mark-vieira. I updated this pull request to follow your suggestions: thirdPartyTest tasks have been removed in favor of s3ThirdPartyTest, azureThirdPartyTest and gcsThirdPartyTest tasks which execute repository plugins YAML REST tests along with *ThirdPartyTests single node integration tests.

I had to move things around and fold some QA projects back into the repository plugins to make the 3rd party tests run on fixtures and external services correctly. Other projects like searchable snapshots (and snapshot tool in 7.x branches) can declare their own s3ThirdPartyTest, azureThirdPartyTest and gcsThirdPartyTest tasks.

I'll wait for this pull request to be validated before updating the CI jobs and other backport PRs.

[mark-vieira]

👍

[original-brownbear]

LGTM from the snapshot side :)

[tlrx]

@rjernst Are you OK with the latest changes?

[rjernst]

LGTM

[tlrx]

Thanks everybody!