Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Remote Store] Include SegRep and Remote Store Rejections in node/stats api #10501

Closed
gbbafna opened this issue Oct 9, 2023 · 0 comments · Fixed by #10656 or #10835
Closed

[Remote Store] Include SegRep and Remote Store Rejections in node/stats api #10501

gbbafna opened this issue Oct 9, 2023 · 0 comments · Fixed by #10656 or #10835
Labels
bug Something isn't working Storage:Remote

Comments

@gbbafna
Copy link
Collaborator

gbbafna commented Oct 9, 2023

Describe the bug
We reject write requests when replicas are lagging or remote store segment store is lagging - https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java#L534

We should include this rejections in our node stats in a separate blob of its own on the lines of shard_indexing_pressure/search backpressure

Proposed blobs are

      "remote_store_pressure" : {
       "rejections" : count,
      },
      "segment_replication_pressure" : {
       "rejections" : count,
      },

Alternate

We can have one blob as well to account for both of them

      "remote_store_pressure" : {
       "upload_rejections" : count,
"lag_rejections" : count
      },
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment