Adding concurrent search versions of query count/time metrics #9622

jed326 · 2023-08-29T19:48:36Z

Description

Adding metrics for concurrent segment search

PR has the following new metrics:

Metric	Description
search.concurrent_query_total	The total number of query operations using concurrent segment search.
search.concurrent_query_time_in_millis	The total time for all query operations using concurrent segment search, in milliseconds.
search.concurrent_query_current	The number of query operations using concurrent segment search that are currently running.

New node stats response:

curl "localhost:9200/_nodes/stats/indices/search?pretty" 
{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "runTask",
  "nodes" : {
    "eZDZszatQxKL_sKQZ3hUWg" : {
      "timestamp" : 1693428497044,
      "name" : "runTask-0",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1:9300",
      "roles" : [
        "cluster_manager",
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes" : {
        "testattr" : "test",
        "shard_indexing_pressure_enabled" : "true"
      },
      "indices" : {
        "search" : {
          "open_contexts" : 0,
          "query_total" : 0,
          "query_time_in_millis" : 0,
          "query_current" : 0,
          "concurrent_query_total" : 0,
          "concurrent_query_time_in_millis" : 0,
          "concurrent_query_current" : 0,
          "fetch_total" : 0,
          "fetch_time_in_millis" : 0,
          "fetch_current" : 0,
          "scroll_total" : 0,
          "scroll_time_in_millis" : 0,
          "scroll_current" : 0,
          "point_in_time_total" : 0,
          "point_in_time_time_in_millis" : 0,
          "point_in_time_current" : 0,
          "suggest_total" : 0,
          "suggest_time_in_millis" : 0,
          "suggest_current" : 0
        }
      }
    }
  }
}

index stats:

curl "localhost:9200/my-index-000001/_stats/search?pretty"
{
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "search" : {
        "open_contexts" : 0,
        "query_total" : 0,
        "query_time_in_millis" : 0,
        "query_current" : 0,
        "concurrent_query_total" : 0,
        "concurrent_query_time_in_millis" : 0,
        "concurrent_query_current" : 0,
        "fetch_total" : 0,
        "fetch_time_in_millis" : 0,
        "fetch_current" : 0,
        "scroll_total" : 0,
        "scroll_time_in_millis" : 0,
        "scroll_current" : 0,
        "point_in_time_total" : 0,
        "point_in_time_time_in_millis" : 0,
        "point_in_time_current" : 0,
        "suggest_total" : 0,
        "suggest_time_in_millis" : 0,
        "suggest_current" : 0
      }
    },
    "total" : {
      "search" : {
        "open_contexts" : 0,
        "query_total" : 0,
        "query_time_in_millis" : 0,
        "query_current" : 0,
        "concurrent_query_total" : 0,
        "concurrent_query_time_in_millis" : 0,
        "concurrent_query_current" : 0,
        "fetch_total" : 0,
        "fetch_time_in_millis" : 0,
        "fetch_current" : 0,
        "scroll_total" : 0,
        "scroll_time_in_millis" : 0,
        "scroll_current" : 0,
        "point_in_time_total" : 0,
        "point_in_time_time_in_millis" : 0,
        "point_in_time_current" : 0,
        "suggest_total" : 0,
        "suggest_time_in_millis" : 0,
        "suggest_current" : 0
      }
    }
  },
  "indices" : {
    "my-index-000001" : {
      "uuid" : "_HJlWPGjT1eCaAJvjo7O0Q",
      "primaries" : {
        "search" : {
          "open_contexts" : 0,
          "query_total" : 0,
          "query_time_in_millis" : 0,
          "query_current" : 0,
          "concurrent_query_total" : 0,
          "concurrent_query_time_in_millis" : 0,
          "concurrent_query_current" : 0,
          "fetch_total" : 0,
          "fetch_time_in_millis" : 0,
          "fetch_current" : 0,
          "scroll_total" : 0,
          "scroll_time_in_millis" : 0,
          "scroll_current" : 0,
          "point_in_time_total" : 0,
          "point_in_time_time_in_millis" : 0,
          "point_in_time_current" : 0,
          "suggest_total" : 0,
          "suggest_time_in_millis" : 0,
          "suggest_current" : 0
        }
      },
      "total" : {
        "search" : {
          "open_contexts" : 0,
          "query_total" : 0,
          "query_time_in_millis" : 0,
          "query_current" : 0,
          "concurrent_query_total" : 0,
          "concurrent_query_time_in_millis" : 0,
          "concurrent_query_current" : 0,
          "fetch_total" : 0,
          "fetch_time_in_millis" : 0,
          "fetch_current" : 0,
          "scroll_total" : 0,
          "scroll_time_in_millis" : 0,
          "scroll_current" : 0,
          "point_in_time_total" : 0,
          "point_in_time_time_in_millis" : 0,
          "point_in_time_current" : 0,
          "suggest_total" : 0,
          "suggest_time_in_millis" : 0,
          "suggest_current" : 0
        }
      }
    }
  }
}

curl "localhost:9200/my-index-000001/_stats/search?pretty&level=shards" 
{
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "search" : {
        "open_contexts" : 0,
        "query_total" : 0,
        "query_time_in_millis" : 0,
        "query_current" : 0,
        "concurrent_query_total" : 0,
        "concurrent_query_time_in_millis" : 0,
        "concurrent_query_current" : 0,
        "fetch_total" : 0,
        "fetch_time_in_millis" : 0,
        "fetch_current" : 0,
        "scroll_total" : 0,
        "scroll_time_in_millis" : 0,
        "scroll_current" : 0,
        "point_in_time_total" : 0,
        "point_in_time_time_in_millis" : 0,
        "point_in_time_current" : 0,
        "suggest_total" : 0,
        "suggest_time_in_millis" : 0,
        "suggest_current" : 0
      }
    },
    "total" : {
      "search" : {
        "open_contexts" : 0,
        "query_total" : 0,
        "query_time_in_millis" : 0,
        "query_current" : 0,
        "concurrent_query_total" : 0,
        "concurrent_query_time_in_millis" : 0,
        "concurrent_query_current" : 0,
        "fetch_total" : 0,
        "fetch_time_in_millis" : 0,
        "fetch_current" : 0,
        "scroll_total" : 0,
        "scroll_time_in_millis" : 0,
        "scroll_current" : 0,
        "point_in_time_total" : 0,
        "point_in_time_time_in_millis" : 0,
        "point_in_time_current" : 0,
        "suggest_total" : 0,
        "suggest_time_in_millis" : 0,
        "suggest_current" : 0
      }
    }
  },
  "indices" : {
    "my-index-000001" : {
      "uuid" : "_HJlWPGjT1eCaAJvjo7O0Q",
      "primaries" : {
        "search" : {
          "open_contexts" : 0,
          "query_total" : 0,
          "query_time_in_millis" : 0,
          "query_current" : 0,
          "concurrent_query_total" : 0,
          "concurrent_query_time_in_millis" : 0,
          "concurrent_query_current" : 0,
          "fetch_total" : 0,
          "fetch_time_in_millis" : 0,
          "fetch_current" : 0,
          "scroll_total" : 0,
          "scroll_time_in_millis" : 0,
          "scroll_current" : 0,
          "point_in_time_total" : 0,
          "point_in_time_time_in_millis" : 0,
          "point_in_time_current" : 0,
          "suggest_total" : 0,
          "suggest_time_in_millis" : 0,
          "suggest_current" : 0
        }
      },
      "total" : {
        "search" : {
          "open_contexts" : 0,
          "query_total" : 0,
          "query_time_in_millis" : 0,
          "query_current" : 0,
          "concurrent_query_total" : 0,
          "concurrent_query_time_in_millis" : 0,
          "concurrent_query_current" : 0,
          "fetch_total" : 0,
          "fetch_time_in_millis" : 0,
          "fetch_current" : 0,
          "scroll_total" : 0,
          "scroll_time_in_millis" : 0,
          "scroll_current" : 0,
          "point_in_time_total" : 0,
          "point_in_time_time_in_millis" : 0,
          "point_in_time_current" : 0,
          "suggest_total" : 0,
          "suggest_time_in_millis" : 0,
          "suggest_current" : 0
        }
      },
      "shards" : {
        "0" : [
          {
            "routing" : {
              "state" : "STARTED",
              "primary" : true,
              "node" : "eZDZszatQxKL_sKQZ3hUWg",
              "relocating_node" : null
            },
            "search" : {
              "open_contexts" : 0,
              "query_total" : 0,
              "query_time_in_millis" : 0,
              "query_current" : 0,
              "concurrent_query_total" : 0,
              "concurrent_query_time_in_millis" : 0,
              "concurrent_query_current" : 0,
              "fetch_total" : 0,
              "fetch_time_in_millis" : 0,
              "fetch_current" : 0,
              "scroll_total" : 0,
              "scroll_time_in_millis" : 0,
              "scroll_current" : 0,
              "point_in_time_total" : 0,
              "point_in_time_time_in_millis" : 0,
              "point_in_time_current" : 0,
              "suggest_total" : 0,
              "suggest_time_in_millis" : 0,
              "suggest_current" : 0
            },
            "commit" : {
              "id" : "h1jwwfNuBgfSrZgkzFeeuQ==",
              "generation" : 2,
              "user_data" : {
                "translog_uuid" : "wpo9o-VJSsKO3Er1mqaZGQ",
                "history_uuid" : "ZgEyoIETTpiZH5VTbEQNXA",
                "local_checkpoint" : "-1",
                "max_seq_no" : "-1",
                "max_unsafe_auto_id_timestamp" : "-1"
              },
              "num_docs" : 0
            },
            "seq_no" : {
              "max_seq_no" : -1,
              "local_checkpoint" : -1,
              "global_checkpoint" : -1
            },
            "retention_leases" : {
              "primary_term" : 1,
              "version" : 1,
              "leases" : [
                {
                  "id" : "peer_recovery/eZDZszatQxKL_sKQZ3hUWg",
                  "retaining_seq_no" : 0,
                  "timestamp" : 1693428571649,
                  "source" : "peer recovery"
                }
              ]
            },
            "shard_path" : {
              "state_path" : "/local/home/dengjay/workspace/opensearch-project/OpenSearch/build/testclusters/runTask-0/data/nodes/0",
              "data_path" : "/local/home/dengjay/workspace/opensearch-project/OpenSearch/build/testclusters/runTask-0/data/nodes/0",
              "is_custom_data_path" : false
            }
          }
        ]
      }
    }
  }
}

_cat APIs:

% curl "localhost:9200/_cat/shards?v=true&h=sqc,sqti,sqto,scqc,scqti,scqto"
sqc sqti sqto scqc scqti scqto
  0  8ms    1    0    0s     0
                              
% curl "localhost:9200/_cat/indices?v=true&h=sqc,sqti,sqto,scqc,scqti,scqto"
sqc sqti sqto scqc scqti scqto
  0  8ms    1    0    0s     0

% curl "localhost:9200/_cat/nodes?v=true&h=sqc,sqti,sqto,scqc,scqti,scqto"
sqc sqti sqto scqc scqti scqto
  0  8ms    1    0    0s     0

Related Issues

Resolves #7359

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-08-29T19:52:57Z

Gradle Check (Jenkins) Run Completed with:

RESULT: ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23748/
CommitID: 969ee5f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-29T20:18:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23751/
CommitID: 48e9cf1
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-29T20:50:34Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL: null
CommitID: 272044d
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-29T21:26:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23758/
CommitID: b70ea35
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-30T01:51:25Z

Compatibility status:

Checks if related components are compatible with change 04a13c5

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

github-actions · 2023-08-30T02:08:38Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23777/
CommitID: 04a13c5
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-30T02:18:34Z

Compatibility status:

Checks if related components are compatible with change 2489674

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-08-30T02:35:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23783/
CommitID: 6b67529
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-30T02:45:26Z

Compatibility status:

Checks if related components are compatible with change 6b67529

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-08-30T02:45:27Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23781/
CommitID: 2489674
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

reta · 2023-08-30T20:27:49Z

@reta @sohami could you take another look and let me know if you think any other changes are needed? Thanks!

Super minor comment just to keep tests consistent, but LGTM otherwise, thanks @jed326 !

sohami · 2023-08-30T20:40:01Z

@jed326 Do you have the output of the _stats API with your changes. Just want to make sure that response table is as expected.

jed326 · 2023-08-30T20:40:38Z

@jed326 Do you have the output of the _stats API with your changes. Just want to make sure that response table is as expected.

Sure let me paste it to the top level PR description

jed326 · 2023-08-30T20:53:59Z

@sohami pasted into the PR description, let me know if there are any other API outputs you want me to share. The same SearchStats class is used for all of these responses though so there shouldn't be a difference anywhere.

github-actions · 2023-08-30T21:14:09Z

Compatibility status:

Checks if related components are compatible with change 9f38267

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

Signed-off-by: Jay Deng <[email protected]>

github-actions · 2023-08-30T21:23:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      2 org.opensearch.index.shard.RemoteIndexShardTests.testNRTReplicaWithRemoteStorePromotedAsPrimaryCommitRefresh
      1 org.opensearch.snapshots.CloneSnapshotIT.testCloneAfterRepoShallowSettingDisabled

URL: https://build.ci.opensearch.org/job/gradle-check/23871/
CommitID: 9f38267
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-08-30T21:24:15Z

Codecov Report

Merging #9622 (0ce1c90) into main (60787b8) will decrease coverage by 0.38%.
Report is 6 commits behind head on main.
The diff coverage is 76.81%.

@@             Coverage Diff              @@
##               main    #9622      +/-   ##
============================================
- Coverage     71.53%   71.16%   -0.38%     
+ Complexity    57897    57555     -342     
============================================
  Files          4782     4782              
  Lines        271332   271393      +61     
  Branches      39614    39630      +16     
============================================
- Hits         194107   193134     -973     
- Misses        61255    62044     +789     
- Partials      15970    16215     +245

Files Changed	Coverage Δ
...ain/java/org/opensearch/common/unit/TimeValue.java	`87.50% <ø> (ø)`
...ibs/core/src/main/java/org/opensearch/Version.java	`77.77% <ø> (ø)`
.../src/main/java/org/opensearch/core/ParseField.java	`95.74% <ø> (ø)`
...ava/org/opensearch/core/action/ActionListener.java	`80.37% <0.00%> (-2.81%)`	⬇️
...g/opensearch/core/common/bytes/BytesReference.java	`74.28% <ø> (ø)`
.../core/common/io/stream/NamedWriteableRegistry.java	`100.00% <ø> (ø)`
.../opensearch/core/common/io/stream/StreamInput.java	`88.28% <ø> (-1.18%)`	⬇️
...opensearch/core/common/io/stream/StreamOutput.java	`94.56% <ø> (-1.29%)`	⬇️
.../opensearch/core/common/settings/SecureString.java	`78.78% <ø> (ø)`
...org/opensearch/core/common/unit/ByteSizeValue.java	`99.08% <ø> (ø)`
... and 43 more

... and 430 files with indirect coverage changes

github-actions · 2023-08-30T21:55:54Z

Compatibility status:

Checks if related components are compatible with change 0ce1c90

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

github-actions · 2023-08-30T22:08:45Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.cluster.ClusterHealthIT.testHealthOnClusterManagerFailover

URL: https://build.ci.opensearch.org/job/gradle-check/23876/
CommitID: 0ce1c90
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

opensearch-trigger-bot · 2023-08-30T23:49:41Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9622-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e563a0c307ade2b3ac339374ddbc2638166f7b49
# Push it to GitHub
git push --set-upstream origin backport/backport-9622-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-9622-to-2.x.

…ensearch-project#9622) Signed-off by: Jay Deng <[email protected]>

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]>

…ime metrics (#9622) (#9653) * Adding concurrent search versions of query count and time metrics (#9622) Signed-off-by: Jay Deng <[email protected]> * Add average query concurrency metric for concurrent segment search (#9670) Signed-off-by: Jay Deng <[email protected]> --------- Signed-off-by: Jay Deng <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Co-authored-by: Andriy Redko <[email protected]>

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]> Signed-off-by: Kaushal Kumar <[email protected]>

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]> Signed-off-by: Ivan Brusic <[email protected]>

…ack shard going from idle to non-idle (#9622) (#12969) Signed-off-by: Ruirui Zhang <[email protected]>

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

jed326 mentioned this pull request Aug 29, 2023

Parameterize ITs to run with concurrent search enabled #9441

Merged

6 tasks

jed326 force-pushed the concurrent-search-metrics branch from 969ee5f to 48e9cf1 Compare August 29, 2023 20:06

jed326 force-pushed the concurrent-search-metrics branch from 48e9cf1 to 272044d Compare August 29, 2023 20:48

jed326 force-pushed the concurrent-search-metrics branch from 272044d to b70ea35 Compare August 29, 2023 21:07

jed326 force-pushed the concurrent-search-metrics branch 2 times, most recently from 04a13c5 to 2489674 Compare August 30, 2023 01:48

jed326 force-pushed the concurrent-search-metrics branch from 2489674 to 6b67529 Compare August 30, 2023 02:15

jed326 force-pushed the concurrent-search-metrics branch from 6b67529 to bdcf4cf Compare August 30, 2023 02:50

jed326 marked this pull request as ready for review August 30, 2023 03:01

jed326 requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal and mch2 as code owners August 30, 2023 03:01

jed326 force-pushed the concurrent-search-metrics branch from 93e0ee5 to 9f38267 Compare August 30, 2023 20:42

sohami approved these changes Aug 30, 2023

View reviewed changes

Adding concurrent search versions of query count and time metrics

0ce1c90

Signed-off-by: Jay Deng <[email protected]>

jed326 force-pushed the concurrent-search-metrics branch from 9f38267 to 0ce1c90 Compare August 30, 2023 21:24

reta approved these changes Aug 30, 2023

View reviewed changes

sohami added the backport 2.x Backport to 2.x branch label Aug 30, 2023

sohami merged commit e563a0c into opensearch-project:main Aug 30, 2023
15 checks passed

opensearch-trigger-bot bot added the backport-failed label Aug 30, 2023

jed326 added a commit to jed326/OpenSearch that referenced this pull request Aug 31, 2023

Adding concurrent search versions of query count and time metrics (op…

6372960

…ensearch-project#9622) Signed-off by: Jay Deng <[email protected]>

jed326 mentioned this pull request Aug 31, 2023

[Backport 2.x] Adding concurrent search versions of query count and time metrics (#9622) #9653

Merged

6 tasks

jed326 added a commit to jed326/OpenSearch that referenced this pull request Aug 31, 2023

Adding concurrent search versions of query count and time metrics (op…

141a5f5

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]>

jed326 added a commit to jed326/OpenSearch that referenced this pull request Sep 1, 2023

Adding concurrent search versions of query count and time metrics (op…

4a4b5ee

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]>

jed326 deleted the concurrent-search-metrics branch September 13, 2023 19:10

brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023

Adding concurrent search versions of query count and time metrics (op…

6a0d55f

…ensearch-project#9622) Signed-off-by: Jay Deng <[email protected]> Signed-off-by: Ivan Brusic <[email protected]>

jed326 mentioned this pull request Mar 21, 2024

Add a counter to node stat api to track shard going from idle to non-idle #12768

Merged

8 tasks

msfroh pushed a commit that referenced this pull request Mar 28, 2024

[Backport 2.x] Add a counter to node stat (and _cat/shards) api to tr…

16d4d3f

…ack shard going from idle to non-idle (#9622) (#12969) Signed-off-by: Ruirui Zhang <[email protected]>

sohami mentioned this pull request May 21, 2024

Add Jay Deng as a maintainer #13773

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding concurrent search versions of query count/time metrics #9622

Adding concurrent search versions of query count/time metrics #9622

jed326 commented Aug 29, 2023 •

edited

Loading

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

reta commented Aug 30, 2023

sohami commented Aug 30, 2023

jed326 commented Aug 30, 2023

jed326 commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

codecov bot commented Aug 30, 2023 •

edited

Loading

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

opensearch-trigger-bot bot commented Aug 30, 2023

Adding concurrent search versions of query count/time metrics #9622

Adding concurrent search versions of query count/time metrics #9622

Conversation

jed326 commented Aug 29, 2023 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Aug 29, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 29, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 29, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 29, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

reta commented Aug 30, 2023

sohami commented Aug 30, 2023

jed326 commented Aug 30, 2023

jed326 commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Aug 30, 2023 • edited Loading

Codecov Report

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 30, 2023

jed326 commented Aug 29, 2023 •

edited

Loading

codecov bot commented Aug 30, 2023 •

edited

Loading