Upgrade AWS SDK v1 to v2

[dblock]

Description

Upgrading AWS SDK v1 to v2 in discovery-ec2 and repository-s3.

This PR contains the following potentially breaking changes for end-users.

• The AWS SDK v2 uses aws.ec2MetadataServiceEndpoint instead of com.amazonaws.sdk.ec2MetadataServiceEndpointOverride. If a user set the IMDS endpoint before that will no longer work without a change. Similarly we've renamed com.amazonaws.sdk.stsEndpointOverride to aws.stsEndpoint. What are the use-cases where anyone would actually do this, or was it done for testing only?
• AWS SDK v2 introduced a concept of TokenBucketRetryCondition, which effectively limits retries on a bucket level for APIs that have been failing continuously. It allocates tokens (default is 500), which means that once 500 retries fail for any API on a bucket, new retries will only be allowed once some retries are rejected.

The AWS SDK has other changes, see https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/migration-client-changes.html and https://github.com/aws/aws-sdk-java-v2/blob/master/docs/LaunchChangelog.md for all changes between AWS SDK v1 and v2.

Related Issues

Resolves #7312.

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[Bukhtawar]

Code scan for dependencies, looks like we have just the two plugins consuming AWS SDK

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationIT.testReplicationPostDeleteAndForceMerge

• URL: https://build.ci.opensearch.org/job/gradle-check/16708/
• CommitID: c855e61
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16715/
• CommitID: 5ceface
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16716/
• CommitID: 5988b08
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16717/
• CommitID: cfe9809
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/16721/
• CommitID: a08cb1f

[raghuvanshraj]

@Bukhtawar I've captured all follow up items here: #7885

[reta]

@dblock we are bringing it to 2.x ? :-)

[dblock]

@dblock we are bringing it to 2.x ? :-)

Yes, we should. However we do have some seemingly breaking changes (see description). WDYT? I'll start a backport anyway.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-7372-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b15ffc9caaec0303b91ecaff1ab39bb9e860ba48
# Push it to GitHub
git push --set-upstream origin backport/backport-7372-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7372-to-2.x.

[reta]

We do, but this is a plugin (and those are long due changes), thanks @dblock !

[dblock]

Backport failed above, will need to do a manual one. I can do it next week, @raghuvanshraj lmk if you want to beat me to it.