[Segment Replication] Add Segment Replication backpressure rejection stats to _nodes/stats #10656

Rishikesh1159 · 2023-10-17T00:16:56Z

Description

Add Segment Replication backpressure rejection stats to _nodes/stats

Related Issues

Resolves #10501

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Rishikesh1159 <[email protected]>

github-actions · 2023-10-17T00:26:11Z

Compatibility status:

Checks if related components are compatible with change 3cf7f33

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

Signed-off-by: Rishikesh Pasham <[email protected]>

github-actions · 2023-10-20T00:41:59Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28512/
CommitID: 5765b25
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-20T01:09:34Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28513/
CommitID: 8567968
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/SegmentReplicationRejectionStats.java

server/src/test/java/org/opensearch/index/SegmentReplicationStatsTrackerTests.java

server/src/main/java/org/opensearch/index/SegmentReplicationRejectionStats.java

github-actions · 2023-10-20T03:06:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/28517/
CommitID: 8567968

Signed-off-by: Rishikesh1159 <[email protected]>

…esh1159/OpenSearch into segrep-backpressure-stats

github-actions · 2023-10-20T16:31:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28600/
CommitID: f1bfe18
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-20T17:10:07Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28605/
CommitID: 3cf7f33
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Rishikesh1159 · 2023-10-20T17:13:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/28605/

CommitID: 3cf7f33
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

known flaky test: #10006

github-actions · 2023-10-20T17:59:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28611/
CommitID: 3cf7f33
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Rishikesh1159 · 2023-10-20T20:41:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/28611/

CommitID: 3cf7f33
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

known flaky test: #10709

github-actions · 2023-10-20T21:24:03Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28640/
CommitID: 3cf7f33
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-20T22:54:26Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28642/
CommitID: 3cf7f33
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-21T00:03:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      2 org.opensearch.index.shard.RemoteIndexShardTests.classMethod
      1 org.opensearch.search.SearchWeightedRoutingIT.testSearchAggregationWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.index.shard.RemoteIndexShardTests.testSegRepSucceedsOnPreviousCopiedFiles

URL: https://build.ci.opensearch.org/job/gradle-check/28647/
CommitID: 3cf7f33
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

opensearch-trigger-bot · 2023-10-21T00:14:58Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-10656-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 51626d03f857ca840280d1e57cfb1bdfbba75e2d
# Push it to GitHub
git push --set-upstream origin backport/backport-10656-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-10656-to-2.x.

Signed-off-by: Rishikesh1159 <[email protected]>

…stats to _nodes/stats (opensearch-project#10656) * Initial WIP for adding segrep backpressure to node stats. Signed-off-by: Rishikesh1159 <[email protected]> * Bind SegmentReplicarionStatsTracker in Node.java Signed-off-by: Rishikesh1159 <[email protected]> * remove additional segrep backpressure info from node stats Signed-off-by: Rishikesh1159 <[email protected]> * fix metric name in node stats Signed-off-by: Rishikesh1159 <[email protected]> * Fix compile error. Signed-off-by: Rishikesh1159 <[email protected]> * Fix compile errors. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR. Signed-off-by: Rishikesh1159 <[email protected]> * Update java docs. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR and fix compile errors. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR. Signed-off-by: Rishikesh1159 <[email protected]> * Update unit test. Signed-off-by: Rishikesh1159 <[email protected]> --------- Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh Pasham <[email protected]>

Signed-off-by: Rishikesh1159 <[email protected]>

…stats to _nodes/stats (opensearch-project#10656) * Initial WIP for adding segrep backpressure to node stats. Signed-off-by: Rishikesh1159 <[email protected]> * Bind SegmentReplicarionStatsTracker in Node.java Signed-off-by: Rishikesh1159 <[email protected]> * remove additional segrep backpressure info from node stats Signed-off-by: Rishikesh1159 <[email protected]> * fix metric name in node stats Signed-off-by: Rishikesh1159 <[email protected]> * Fix compile error. Signed-off-by: Rishikesh1159 <[email protected]> * Fix compile errors. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR. Signed-off-by: Rishikesh1159 <[email protected]> * Update java docs. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR and fix compile errors. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR. Signed-off-by: Rishikesh1159 <[email protected]> * Update unit test. Signed-off-by: Rishikesh1159 <[email protected]> --------- Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh Pasham <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

…-project#10835) Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

Rishikesh1159 added 4 commits October 16, 2023 23:47

Initial WIP for adding segrep backpressure to node stats.

6a4e757

Signed-off-by: Rishikesh1159 <[email protected]>

Bind SegmentReplicarionStatsTracker in Node.java

7da3cfb

Signed-off-by: Rishikesh1159 <[email protected]>

remove additional segrep backpressure info from node stats

d9ad4ce

Signed-off-by: Rishikesh1159 <[email protected]>

fix metric name in node stats

00cfe84

Signed-off-by: Rishikesh1159 <[email protected]>

github-actions bot added bug Something isn't working Storage:Remote labels Oct 17, 2023

Rishikesh1159 requested a review from abbashus as a code owner October 20, 2023 00:21

Merge branch 'main' into segrep-backpressure-stats

8567968

Signed-off-by: Rishikesh Pasham <[email protected]>

dreamer-89 reviewed Oct 20, 2023

View reviewed changes

Rishikesh1159 and others added 3 commits October 20, 2023 15:38

Update unit test.

b96a875

Signed-off-by: Rishikesh1159 <[email protected]>

Merge branch 'segrep-backpressure-stats' of https://github.com/Rishik…

f1bfe18

…esh1159/OpenSearch into segrep-backpressure-stats

Merge branch 'opensearch-project:main' into segrep-backpressure-stats

3cf7f33

mch2 approved these changes Oct 20, 2023

View reviewed changes

Rishikesh1159 added the backport 2.x Backport to 2.x branch label Oct 21, 2023

Rishikesh1159 merged commit 51626d0 into opensearch-project:main Oct 21, 2023
18 of 19 checks passed

opensearch-trigger-bot bot added the backport-failed label Oct 21, 2023

Rishikesh1159 mentioned this pull request Oct 22, 2023

[Backport 2.x] [Segment Replication] Backport Adding Segment Replication Backpressure Rejection Stats to _nodes/stats #10834

Merged

8 tasks

Rishikesh1159 added a commit to Rishikesh1159/OpenSearch that referenced this pull request Oct 22, 2023

Update version check made in PR opensearch-project#10656.

d2958f4

Signed-off-by: Rishikesh1159 <[email protected]>

Rishikesh1159 mentioned this pull request Oct 22, 2023

[Segment Replication] Update version check for segment replication backpressure rejection stats #10835

Merged

8 tasks

sachinpkale pushed a commit that referenced this pull request Oct 23, 2023

Update version check made in PR #10656. (#10835)

ef4b327

Signed-off-by: Rishikesh1159 <[email protected]>

ashking94 mentioned this pull request Oct 26, 2023

[DOC] Create documentation corresponding to changes in _nodes/stats api in OpenSearch opensearch-project/documentation-website#5400

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Segment Replication] Add Segment Replication backpressure rejection stats to _nodes/stats #10656

[Segment Replication] Add Segment Replication backpressure rejection stats to _nodes/stats #10656

Rishikesh1159 commented Oct 17, 2023

github-actions bot commented Oct 17, 2023 •

edited

Loading

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

Rishikesh1159 commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Rishikesh1159 commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 21, 2023

opensearch-trigger-bot bot commented Oct 21, 2023

[Segment Replication] Add Segment Replication backpressure rejection stats to _nodes/stats #10656

[Segment Replication] Add Segment Replication backpressure rejection stats to _nodes/stats #10656

Conversation

Rishikesh1159 commented Oct 17, 2023

Description

Related Issues

Check List

github-actions bot commented Oct 17, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

Rishikesh1159 commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

Rishikesh1159 commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 21, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Oct 21, 2023

github-actions bot commented Oct 17, 2023 •

edited

Loading