Use ReplicationFailedException instead of OpensearchException in Repl…

[ayushKataria]

…icationTarget

Signed-off-by: Ayush Kataria [email protected]

Description

Uses a narrower ReplicationFailedException instead of OpensearchException in ReplicationTarget and ReplicationListener

Issues Resolved

[List any issues this PR will resolve]

• #4291

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4252/
• CommitID: f736a78

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4253/
• CommitID: 2d27461

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4262/
• CommitID: a06998e

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4263/
• CommitID: 92d09b6

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4269/
• CommitID: 74830ff

[mch2]

Thanks for the change @ayushKataria. Some minor comments that I think are causing the test failures.

[mch2]

With notifyListener now accepting a ReplicationFailedException, we should wrap the cancellationException in ReplicationFailedException.

        cancellableThreads.setOnCancel((reason, beforeCancelEx) -> {
            // This method only executes when cancellation is triggered by this node and caught by a call to checkForCancel,
            // SegmentReplicationSource does not share CancellableThreads.
            final CancellableThreads.ExecutionCancelledException executionCancelledException =
                new CancellableThreads.ExecutionCancelledException("replication was canceled reason [" + reason + "]");
            notifyListener(new ReplicationFailedException("Segment replication failed to complete", executionCancelledException), false);
            throw executionCancelledException;
        });

There is some special logic above in SegmentReplicationTarget that overrides notifyListener to unwrap the failure and mark it as cancelled. This block will never trigger if the cancellation initiates from the primary however. So thats why the special logic is not in this setOnCancel block.

[ayushKataria]

ahh, thanks for the help. For some reason I was not able to figure the trigger out exactly

[mch2]

A nit - I don't think we need to cast or re-wrap the exception here.

    @Override
    public void notifyListener(ReplicationFailedException e, boolean sendShardFailure) {
        // Cancellations still are passed to our SegmentReplicationListner as failures, if we have failed because of cancellation
        // update the stage.
        if (ExceptionsHelper.unwrap(e, CancellableThreads.ExecutionCancelledException.class) != null) {
            state.setStage(SegmentReplicationState.Stage.CANCELLED);
        }
        listener.onFailure(state(), e, sendShardFailure);
}

[mch2]

With RecoveryFailedException extending ReplicationFailedException I don't think we need to override here in RecoveryTarget?

[ayushKataria]

Yeah, makes sense. This is redundant now, so will remove it.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/4349/
• CommitID: c9bfc98

[codecov-commenter]

Codecov Report

Merging #4725 (f6c25fe) into main (d15795a) will decrease coverage by 0.00%.
The diff coverage is 46.69%.

@@             Coverage Diff              @@
##               main    #4725      +/-   ##
============================================
- Coverage     70.66%   70.65%   -0.01%     
- Complexity    57578    57594      +16     
============================================
  Files          4661     4669       +8     
  Lines        276662   276870     +208     
  Branches      40325    40346      +21     
============================================
+ Hits         195501   195624     +123     
- Misses        64926    64949      +23     
- Partials      16235    16297      +62     

Impacted Files	Coverage Δ
.../delete/DeleteDecommissionStateRequestBuilder.java	0.00% <0.00%> (ø)
...apshots/restore/RestoreSnapshotRequestBuilder.java	15.78% <0.00%> (-0.88%)	⬇️
.../org/opensearch/client/support/AbstractClient.java	32.61% <0.00%> (-0.32%)	⬇️
.../java/org/opensearch/common/util/FeatureFlags.java	50.00% <ø> (ø)
...ateway/TransportNodesListGatewayStartedShards.java	51.00% <0.00%> (-0.68%)	⬇️
...h/index/shard/RemoveCorruptedShardDataCommand.java	81.51% <ø> (ø)
...java/org/opensearch/index/shard/StoreRecovery.java	67.88% <ø> (+0.01%)	⬆️
...h/index/store/InMemoryRemoteSnapshotDirectory.java	0.00% <0.00%> (ø)
...in/java/org/opensearch/indices/IndicesService.java	69.65% <0.00%> (-0.25%)	⬇️
...arch/indices/recovery/RecoveryFailedException.java	100.00% <ø> (ø)
... and 514 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/4350/
• CommitID: c3e49c0

[mch2]

LGTM! Thanks for the change @ayushKataria!

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/4550/
• CommitID: 1b1534b

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4587/
• CommitID: f6c25fe

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/4643/
• CommitID: f6c25fe

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4725-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 6bae823cefad7373a748ce833ce75d80f48b380b
# Push it to GitHub
git push --set-upstream origin backport/backport-4725-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4725-to-2.x.