Backport/backport 1447 to 1.3 (#1497)

* Adds nodes stats API (#1369) Reconciled master terminology * Adds nodes stats API Signed-off-by: Fanit Kolchina <[email protected]> * Refactored hot threads and info APIs Signed-off-by: Fanit Kolchina <[email protected]> * Incorporated doc review comments Signed-off-by: Fanit Kolchina <[email protected]> * Incorporated review feedback Signed-off-by: Fanit Kolchina <[email protected]> * Formatting changes Signed-off-by: Fanit Kolchina <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/index.md Co-authored-by: Nate Bower <[email protected]> * Update nodes-hot-threads.md * Update nodes-hot-threads.md * Update nodes-hot-threads.md * Update nodes-info.md * Update nodes-info.md * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update nodes-info.md * Update nodes-info.md * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update nodes-info.md * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update _opensearch/rest-api/nodes-apis/nodes-info.md Co-authored-by: Nate Bower <[email protected]> * Update nodes-stats.md * Incorporated nodes stats editorial comments Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Co-authored-by: Nate Bower <[email protected]> (cherry picked from commit 85b18f7) * Fixed KNN search formulas (#1447) Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> (cherry picked from commit 2c74943) Reconciled kNN formulas for 1.3. * Update index.md * Update nodes-stats.md
opensearch-project · Oct 10, 2022 · cab2600 · cab2600
1 parent 457413d
commit cab2600
Show file tree

Hide file tree

Showing 6 changed files with 1,158 additions and 130 deletions.
diff --git a/_opensearch/rest-api/nodes-apis/index.md b/_opensearch/rest-api/nodes-apis/index.md
@@ -8,64 +8,68 @@ nav_order: 5
 
 # Nodes API
 
-The nodes API makes it possible to retrieve information about individual nodes within your cluster. It supports standard parameters such `{timeout}` and `{node-filters}`.
-
-## Timeout
-
-The `{timeout}` parameter can be used to change the default time limit for node response.
-
-Parameter | Type      | Description
-:--- |:----------| :---
-`{timeout}` | TimeValue | Default value is `30s`.
+The nodes API makes it possible to retrieve information about individual nodes within your cluster. 
 
 ## Node filters
 
-Use the `{node-filters}` parameter to filter the target set of nodes in the API response.
+Use the `<node-filters>` parameter to filter the target set of nodes in the API response.
+
+<style>
+table th:first-of-type {
+    width: 25%;
+}
+table th:nth-of-type(2) {
+    width: 10%;
+}
+table th:nth-of-type(3) {
+    width: 65%;
+}
+</style>
 
 Parameter | Type   | Description
 :--- |:-------| :---
-{node-filters} | String | A comma-separated list of resolution mechanisms that OpenSearch uses to identify cluster nodes.
+`<node-filters>` | String | A comma-separated list of resolution mechanisms that OpenSearch uses to identify cluster nodes.
 
-Node filters support several node resolution mechanisms.
+Node filters support several node resolution mechanisms:
 
-- Pre defined constants: `_local`, `_master`, or `_all`
+- Predefined constants: `_local`, `_master`, or `_all`.
 - An exact match for `nodeID`
-- A simple case-sensitive wildcard pattern matching for `node-name`, `host-name`, or `host-IP-address`
-- Node roles where the `<bool>` value is set either to `true` or `false`):
-  - `master:<bool>` 
-  - `data:<bool>`
-  - `ingest:<bool>`
-  - `voting_only:<bool>`
-  - `ml:<bool>`
-  - `coordinating_only:<bool>`
+- A simple case-sensitive wildcard pattern matching for `node-name`, `host-name`, or `host-IP-address`.
+- Node roles where the `<bool>` value is set either to `true` or `false`:
+  - `master:<bool>` refers to all master-eligible nodes.
+  - `data:<bool>` refers to all data nodes.
+  - `ingest:<bool>` refers to all ingest nodes.
+  - `voting_only:<bool>` refers to all voting-only nodes.
+  - `ml:<bool>` refers to all machine learning (ML) nodes.
+  - `coordinating_only:<bool>` refers to all coordinating-only nodes.
 - A simple case-sensitive wildcard pattern matching for node attributes: `<node attribute*>:<attribute value*>`. The wildcard matching pattern can be used in both the key and value at the same time.
 
 Resolution mechanisms are applied sequentially in the order specified by the client. Each mechanism specification can either add or remove nodes.
 
-If you want to get statistics from the elected cluster manager node only, use the following:
+To get statistics from the elected master node only, use the following query :
 
-```bash
+```json
 GET /_nodes/_master/stats
 ```
 
-If you want to get statistics from nodes that are data-only nodes, use the following:
+To get statistics from nodes that are data-only nodes, use the following query:
 
-```bash
+```json
 GET /_nodes/data:true/stats
 ```
 
 ### Order of resolution mechanisms
 
-The order of resolution mechanisms is applied sequentially, and each can add or remove nodes. The following examples yield different results:
+The order of resolution mechanisms is applied sequentially, and each can add or remove nodes. The following examples yield different results.
 
-If you want to get statistics from all the nodes but the cluster manager node, use the following:
+To get statistics from all the nodes except the master node, use the following query:
 
-```bash
+```json
 GET /_nodes/_all,master:false/stats
 ```
 
-However, if we switch the resolution mechanisms, then the result will include all the cluster nodes including the cluster manager node. 
+However, if you switch the resolution mechanisms, the result will include all the cluster nodes, including the master node: 
 
-```bash
-GET /_nodes/cluster_namager:false,_all/stats
-```
+```json
+GET /_nodes/master:false,_all/stats
+```
diff --git a/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md b/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md
@@ -3,7 +3,7 @@ layout: default
 title: Nodes hot threads
 parent: Nodes APIs
 grand_parent: REST API reference
-nav_order: 10
+nav_order: 30
 ---
 
 # Nodes hot threads
@@ -18,24 +18,49 @@ GET /_nodes/hot_threads
 
 ## Path and HTTP methods
 
-```bash
+```json
 GET /_nodes/hot_threads
-GET /_nodes/{nodeId}/hot_threads
+GET /_nodes/<nodeId>/hot_threads
 ```
 
-## URL parameters
+## Path parameters
+
+You can include the following optional path parameter in your request. 
+
+Parameter | Type | Description
+:--- | :--- | :---
+nodeId | String  | A comma-separated list of node IDs used to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`.
 
-You can include the following URL parameters in your request. All parameters are optional.
+## Query parameters
 
-Parameter | Type     | Description
-:--- |:----------| :---
-nodeId | String  | A comma-separated list of node IDs to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`.
-snapshots | Integer  | Number of samples of thread stacktraces. Defaults to `10`.
-interval | TimeValue | Interval between consecutive samples. Defaults to `500ms`.
-threads | Integer   | A number of top busiest threads to return information about. Defaults to `3`.
-ignore_idle_threads | Boolean   | Don’t show threads that are in known-idle states, such as waiting on a socket select or pulling from an empty task queue. Defaults to `true`.
-type | String    | Supported thread types are `cpu`, `wait`, or `block`. Defaults to `cpu`.
-timeout | TimeValue | A request [timeout]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#timeout). Defaults to `30s`.
+You can include the following query parameters in your request. All query parameters are optional.
+
+Parameter | Type | Description
+:--- | :---| :---
+snapshots | Integer | The number of samples of thread stacktraces. Defaults to `10`.
+interval | Time | The interval between consecutive samples. Defaults to `500ms`.
+threads | Integer | The number of the busiest threads to return information about. Defaults to `3`.
+ignore_idle_threads | Boolean   | Don’t show threads that are in known idle states, such as waiting on a socket select or pulling from an empty task queue. Defaults to `true`.
+type | String | Supported thread types are `cpu`, `wait`, or `block`. Defaults to `cpu`.
+timeout | Time | Sets the time limit for node response. Default value is `30s`.
+
+#### Sample request 
+
+```json
+GET /_nodes/hot_threads
+```
+
+#### Sample response
+
+```bash
+::: {opensearch}{F-ByTQzVQ3GQeYzQJArJGQ}{GxbcLdCATPWggOuQHJAoCw}{127.0.0.1}{127.0.0.1:9300}{dimr}{shard_indexing_pressure_enabled=true}
+   Hot threads at 2022-09-29T19:46:44.533Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
+
+    0.1% (455.5micros out of 500ms) cpu usage by thread 'ScheduledMetricCollectorsExecutor'
+     10/10 snapshots sharing following 2 elements
+       [email protected]/java.lang.Thread.sleep(Native Method)
+       org.opensearch.performanceanalyzer.collectors.ScheduledMetricCollectorsExecutor.run(ScheduledMetricCollectorsExecutor.java:100)
+```
 
 ## Response
 
@@ -56,7 +81,7 @@ Line segment | Description
 `{dimr}` | Node roles (d=data, i=ingest, m=cluster&nbsp;manager, r=remote&nbsp;cluster&nbsp;client).
 `{zone=west-a2, shard_indexing_pressure_enabled=true}` | Node attributes.
 
-Then follows information about threads of selected type.
+Then information about threads of the selected type is provided.
 
 ```bash
 ::: {global-eu-35}{uFPbKLDOTlOmdnwUlKW8sw}{OAM8OT5CQAyasWuIDeVyUA}{global-eu-35.local}{[gdv2:a284:2acv:5fa6:0:3a2:7260:74cf]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}

diff --git a/_opensearch/rest-api/nodes-apis/nodes-info.md b/_opensearch/rest-api/nodes-apis/nodes-info.md
@@ -3,12 +3,12 @@ layout: default
 title: Nodes info
 parent: Nodes APIs
 grand_parent: REST API reference
-nav_order: 20
+nav_order: 10
 ---
 
 # Nodes info
 
-Represents mostly static information about your cluster's nodes, including but not limited to:
+The nodes info API represents mostly static information about your cluster's nodes, including but not limited to:
 
 - Host system information 
 - JVM 
@@ -19,77 +19,76 @@ Represents mostly static information about your cluster's nodes, including but n
 
 ## Example
 
-To get information on all nodes in a cluster:
+To get information about all nodes in a cluster, use the following query:
 
-```bash
+```json
 GET /_nodes
 ```
 
-To get thread pool information from the cluster manager node only:
+To get thread pool information about the cluster manager node only, use the following query:
 
-```bash
-GET /_nodes/master:true/thread_pool?pretty
+```json
+GET /_nodes/master:true/thread_pool
 ```
 
 ## Path and HTTP methods
 
 ```bash
 GET /_nodes
-GET /_nodes/{nodeId}
-GET /_nodes/{metrics}
-GET /_nodes/{nodeId}/{metrics}
+GET /_nodes/<nodeId>
+GET /_nodes/<metrics>
+GET /_nodes/<nodeId>/<metrics>
 # or full path equivalent
-GET /_nodes/{nodeId}/info/{metrics}
+GET /_nodes/<nodeId>/info/<metrics>
 ```
 
-## URL parameters
+## Path parameters
 
-You can include the following URL parameters in your request. All parameters are optional.
+The following table lists the available path parameters. All path parameters are optional.
 
-Parameter | Type   | Description
+Parameter | Type | Description
 :--- |:-------| :---
-nodeId | String | A comma-separated list of nodeIds to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`.
-metrics | String | A comma-separated list of metric groups that will be included in the response. For example `jvm,thread_pools`. Defaults to all metrics.
-timeout | TimeValue | A request [timeout]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#timeout). Defaults to `30s`.
+nodeId | String | A comma-separated list of nodeIds used to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`.
+metrics | String | A comma-separated list of metric groups that will be included in the response. For example, `jvm,thread_pool`. Defaults to all metrics.
 
-The following are listed for all available metric groups:
+The following table lists all available metric groups.
 
 Metric | Description
 :--- |:----
-`settings` | A node's settings. This is combination of the default settings, custom settings from [configuration file]({{site.url}}{{site.baseurl}}/opensearch/configuration/#configuration-file) and dynamically [updated settings]({{site.url}}{{site.baseurl}}/opensearch/configuration/#update-cluster-settings-using-the-api).
-`os` | Static information about the host OS, including version, processor architecture and available/allocated processors.
-`process` | Contains process OD.
-`jvm` | Detailed static information about running JVM, including arguments.
-`thread_pool` | Configured options for all individual thread pools.
-`transport` | Mostly static information about the transport layer.
-`http` | Mostly static information about the HTTP layer.
-`plugins` | Information about installed plugins and modules.
-`ingest` | Information about ingest pipelines and available ingest processors.
-`aggregations` | Information about available [aggregations]({{site.url}}{{site.baseurl}}/opensearch/aggregations).
-`indices` | Static index settings configured at the node level.
-
-## Response
-
-The response contains basic node identification and build info for every node
-matching the `{nodeId}` request parameter.
+settings | A node's settings. This is a combination of the default settings, custom settings from the [configuration file]({{site.url}}{{site.baseurl}}/opensearch/configuration/#configuration-file), and dynamically [updated settings]({{site.url}}{{site.baseurl}}/opensearch/configuration/#update-cluster-settings-using-the-api).
+os | Static information about the host OS, including version, processor architecture, and available/allocated processors.
+process | Contains the process ID.
+jvm | Detailed static information about the running JVM, including arguments.
+thread_pool | Configured options for all individual thread pools.
+transport | Mostly static information about the transport layer.
+http | Mostly static information about the HTTP layer.
+plugins | Information about installed plugins and modules.
+ingest | Information about ingest pipelines and available ingest processors.
+aggregations | Information about available [aggregations]({{site.url}}{{site.baseurl}}/opensearch/aggregations).
+indices | Static index settings configured at the node level.
+
+## Query parameters
+
+You can include the following query parameters in your request. All query parameters are optional.
+
+Parameter | Type | Description
+:--- |:-------| :---
+flat_settings| Boolean | Specifies whether to return the `settings` object of the response in flat format. Default is `false`.
+timeout | Time | Sets the time limit for node response. Default value is `30s`.
 
-Metric | Description
-:--- |:----
-name | A node name.
-transport_address | A node transport address.
-host | A node host address.
-ip | A node host ip address.
-version | A node OpenSearch version.
-build_type | A build type, like `rpm`,`docker`, `zip`, etc.
-build_hash | A git commit hash of the build.
-roles | A node's role.
-attributes | A node attributes.
-
-The response also contains one or more metric groups, depending on the `{metrics}` request parameter.
+#### Sample request
+
+The following query requests the `process` and `transport` metrics from the cluster manager node: 
 
 ```json
-GET /_nodes/master:true/process,transport?pretty
+GET /_nodes/cluster_manager:true/process,transport
+```
 
+#### Sample response
+
+The response contains the metric groups specified in the `<metrics>` request parameter (in this case, `process` and `transport`):
+
+```json
 {
   "_nodes": {
     "total": 1,
@@ -133,6 +132,34 @@ GET /_nodes/master:true/process,transport?pretty
 }
 ```
 
+## Response fields
+
+The response contains the basic node identification and build info for every node matching the `<nodeId>` request parameter. The following table lists the response fields.
+
+Field | Description
+:--- |:----
+name | The node's name.
+transport_address | The node's transport address.
+host | The node's host address.
+ip | The node's host IP address.
+version | The node's OpenSearch version.
+build_type | The node's build type, like `rpm`, `docker`, `tar`, etc.
+build_hash | The git commit hash of the build.
+total_indexing_buffer | The maximum heap size in bytes used to hold newly indexed documents. Once this heap size is exceeded, the documents are written to disk.
+roles | The list of the node's roles.
+attributes | The node's attributes.
+os | Information about the OS, including name, version, architecture, refresh interval, and the number of available and allocated processors.
+process | Information about the currently running process, including PID, refresh interval, and `mlockall`, which specifies whether the process address space has been successfully locked in memory. 
+jvm | Information about the JVM, including PID, version, memory information, garbage collector information, and arguments.
+thread_pool | Information about the thread pool.
+transport | Information about the transport address, including bound address, publish address, and profiles.
+http | Information about the HTTP address, including bound address, publish address, and maximum content length, in bytes.
+plugins | Information about the installed plugins, including name, version, OpenSearch version, Java version, description, class name, custom folder name, a list of extended plugins, and `has_native_controller`, which specifies whether the plugin has a native controller process. 
+modules | Information about the modules, including name, version, OpenSearch version, Java version, description, class name, custom folder name, a list of extended plugins, and `has_native_controller`, which specifies whether the plugin has a native controller process. Modules are different from plugins because modules are loaded into OpenSearch automatically, while plugins have to be installed manually.
+ingest | Information about ingest pipelines and processors.
+aggregations | Information about the available aggregation types.
+
+
 ## Required permissions
 
 If you use the security plugin, make sure you have the appropriate permissions: `cluster:monitor/nodes/info`.