diff --git a/_opensearch/rest-api/nodes-apis/index.md b/_opensearch/rest-api/nodes-apis/index.md index b210b9b458..4f45d75666 100644 --- a/_opensearch/rest-api/nodes-apis/index.md +++ b/_opensearch/rest-api/nodes-apis/index.md @@ -8,64 +8,68 @@ nav_order: 5 # Nodes API -The nodes API makes it possible to retrieve information about individual nodes within your cluster. It supports standard parameters such `{timeout}` and `{node-filters}`. - -## Timeout - -The `{timeout}` parameter can be used to change the default time limit for node response. - -Parameter | Type | Description -:--- |:----------| :--- -`{timeout}` | TimeValue | Default value is `30s`. +The nodes API makes it possible to retrieve information about individual nodes within your cluster. ## Node filters -Use the `{node-filters}` parameter to filter the target set of nodes in the API response. +Use the `` parameter to filter the target set of nodes in the API response. + + Parameter | Type | Description :--- |:-------| :--- -{node-filters} | String | A comma-separated list of resolution mechanisms that OpenSearch uses to identify cluster nodes. +`` | String | A comma-separated list of resolution mechanisms that OpenSearch uses to identify cluster nodes. -Node filters support several node resolution mechanisms. +Node filters support several node resolution mechanisms: -- Pre defined constants: `_local`, `_master`, or `_all` +- Predefined constants: `_local`, `_master`, or `_all`. - An exact match for `nodeID` -- A simple case-sensitive wildcard pattern matching for `node-name`, `host-name`, or `host-IP-address` -- Node roles where the `` value is set either to `true` or `false`): - - `master:` - - `data:` - - `ingest:` - - `voting_only:` - - `ml:` - - `coordinating_only:` +- A simple case-sensitive wildcard pattern matching for `node-name`, `host-name`, or `host-IP-address`. +- Node roles where the `` value is set either to `true` or `false`: + - `master:` refers to all master-eligible nodes. + - `data:` refers to all data nodes. + - `ingest:` refers to all ingest nodes. + - `voting_only:` refers to all voting-only nodes. + - `ml:` refers to all machine learning (ML) nodes. + - `coordinating_only:` refers to all coordinating-only nodes. - A simple case-sensitive wildcard pattern matching for node attributes: `:`. The wildcard matching pattern can be used in both the key and value at the same time. Resolution mechanisms are applied sequentially in the order specified by the client. Each mechanism specification can either add or remove nodes. -If you want to get statistics from the elected cluster manager node only, use the following: +To get statistics from the elected master node only, use the following query : -```bash +```json GET /_nodes/_master/stats ``` -If you want to get statistics from nodes that are data-only nodes, use the following: +To get statistics from nodes that are data-only nodes, use the following query: -```bash +```json GET /_nodes/data:true/stats ``` ### Order of resolution mechanisms -The order of resolution mechanisms is applied sequentially, and each can add or remove nodes. The following examples yield different results: +The order of resolution mechanisms is applied sequentially, and each can add or remove nodes. The following examples yield different results. -If you want to get statistics from all the nodes but the cluster manager node, use the following: +To get statistics from all the nodes except the master node, use the following query: -```bash +```json GET /_nodes/_all,master:false/stats ``` -However, if we switch the resolution mechanisms, then the result will include all the cluster nodes including the cluster manager node. +However, if you switch the resolution mechanisms, the result will include all the cluster nodes, including the master node: -```bash -GET /_nodes/cluster_namager:false,_all/stats -``` \ No newline at end of file +```json +GET /_nodes/master:false,_all/stats +``` diff --git a/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md b/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md index 66777a839c..db45f74aeb 100644 --- a/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md +++ b/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md @@ -3,7 +3,7 @@ layout: default title: Nodes hot threads parent: Nodes APIs grand_parent: REST API reference -nav_order: 10 +nav_order: 30 --- # Nodes hot threads @@ -18,24 +18,49 @@ GET /_nodes/hot_threads ## Path and HTTP methods -```bash +```json GET /_nodes/hot_threads -GET /_nodes/{nodeId}/hot_threads +GET /_nodes//hot_threads ``` -## URL parameters +## Path parameters + +You can include the following optional path parameter in your request. + +Parameter | Type | Description +:--- | :--- | :--- +nodeId | String | A comma-separated list of node IDs used to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`. -You can include the following URL parameters in your request. All parameters are optional. +## Query parameters -Parameter | Type | Description -:--- |:----------| :--- -nodeId | String | A comma-separated list of node IDs to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`. -snapshots | Integer | Number of samples of thread stacktraces. Defaults to `10`. -interval | TimeValue | Interval between consecutive samples. Defaults to `500ms`. -threads | Integer | A number of top busiest threads to return information about. Defaults to `3`. -ignore_idle_threads | Boolean | Don’t show threads that are in known-idle states, such as waiting on a socket select or pulling from an empty task queue. Defaults to `true`. -type | String | Supported thread types are `cpu`, `wait`, or `block`. Defaults to `cpu`. -timeout | TimeValue | A request [timeout]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#timeout). Defaults to `30s`. +You can include the following query parameters in your request. All query parameters are optional. + +Parameter | Type | Description +:--- | :---| :--- +snapshots | Integer | The number of samples of thread stacktraces. Defaults to `10`. +interval | Time | The interval between consecutive samples. Defaults to `500ms`. +threads | Integer | The number of the busiest threads to return information about. Defaults to `3`. +ignore_idle_threads | Boolean | Don’t show threads that are in known idle states, such as waiting on a socket select or pulling from an empty task queue. Defaults to `true`. +type | String | Supported thread types are `cpu`, `wait`, or `block`. Defaults to `cpu`. +timeout | Time | Sets the time limit for node response. Default value is `30s`. + +#### Sample request + +```json +GET /_nodes/hot_threads +``` + +#### Sample response + +```bash +::: {opensearch}{F-ByTQzVQ3GQeYzQJArJGQ}{GxbcLdCATPWggOuQHJAoCw}{127.0.0.1}{127.0.0.1:9300}{dimr}{shard_indexing_pressure_enabled=true} + Hot threads at 2022-09-29T19:46:44.533Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true: + + 0.1% (455.5micros out of 500ms) cpu usage by thread 'ScheduledMetricCollectorsExecutor' + 10/10 snapshots sharing following 2 elements + java.base@17.0.4/java.lang.Thread.sleep(Native Method) + org.opensearch.performanceanalyzer.collectors.ScheduledMetricCollectorsExecutor.run(ScheduledMetricCollectorsExecutor.java:100) +``` ## Response @@ -56,7 +81,7 @@ Line segment | Description `{dimr}` | Node roles (d=data, i=ingest, m=cluster manager, r=remote cluster client). `{zone=west-a2, shard_indexing_pressure_enabled=true}` | Node attributes. -Then follows information about threads of selected type. +Then information about threads of the selected type is provided. ```bash ::: {global-eu-35}{uFPbKLDOTlOmdnwUlKW8sw}{OAM8OT5CQAyasWuIDeVyUA}{global-eu-35.local}{[gdv2:a284:2acv:5fa6:0:3a2:7260:74cf]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true} diff --git a/_opensearch/rest-api/nodes-apis/nodes-info.md b/_opensearch/rest-api/nodes-apis/nodes-info.md index 78e79bd44e..c62fc4b222 100644 --- a/_opensearch/rest-api/nodes-apis/nodes-info.md +++ b/_opensearch/rest-api/nodes-apis/nodes-info.md @@ -3,12 +3,12 @@ layout: default title: Nodes info parent: Nodes APIs grand_parent: REST API reference -nav_order: 20 +nav_order: 10 --- # Nodes info -Represents mostly static information about your cluster's nodes, including but not limited to: +The nodes info API represents mostly static information about your cluster's nodes, including but not limited to: - Host system information - JVM @@ -19,77 +19,76 @@ Represents mostly static information about your cluster's nodes, including but n ## Example -To get information on all nodes in a cluster: +To get information about all nodes in a cluster, use the following query: -```bash +```json GET /_nodes ``` -To get thread pool information from the cluster manager node only: +To get thread pool information about the cluster manager node only, use the following query: -```bash -GET /_nodes/master:true/thread_pool?pretty +```json +GET /_nodes/master:true/thread_pool ``` ## Path and HTTP methods ```bash GET /_nodes -GET /_nodes/{nodeId} -GET /_nodes/{metrics} -GET /_nodes/{nodeId}/{metrics} +GET /_nodes/ +GET /_nodes/ +GET /_nodes// # or full path equivalent -GET /_nodes/{nodeId}/info/{metrics} +GET /_nodes//info/ ``` -## URL parameters +## Path parameters -You can include the following URL parameters in your request. All parameters are optional. +The following table lists the available path parameters. All path parameters are optional. -Parameter | Type | Description +Parameter | Type | Description :--- |:-------| :--- -nodeId | String | A comma-separated list of nodeIds to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`. -metrics | String | A comma-separated list of metric groups that will be included in the response. For example `jvm,thread_pools`. Defaults to all metrics. -timeout | TimeValue | A request [timeout]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#timeout). Defaults to `30s`. +nodeId | String | A comma-separated list of nodeIds used to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`. +metrics | String | A comma-separated list of metric groups that will be included in the response. For example, `jvm,thread_pool`. Defaults to all metrics. -The following are listed for all available metric groups: +The following table lists all available metric groups. Metric | Description :--- |:---- -`settings` | A node's settings. This is combination of the default settings, custom settings from [configuration file]({{site.url}}{{site.baseurl}}/opensearch/configuration/#configuration-file) and dynamically [updated settings]({{site.url}}{{site.baseurl}}/opensearch/configuration/#update-cluster-settings-using-the-api). -`os` | Static information about the host OS, including version, processor architecture and available/allocated processors. -`process` | Contains process OD. -`jvm` | Detailed static information about running JVM, including arguments. -`thread_pool` | Configured options for all individual thread pools. -`transport` | Mostly static information about the transport layer. -`http` | Mostly static information about the HTTP layer. -`plugins` | Information about installed plugins and modules. -`ingest` | Information about ingest pipelines and available ingest processors. -`aggregations` | Information about available [aggregations]({{site.url}}{{site.baseurl}}/opensearch/aggregations). -`indices` | Static index settings configured at the node level. - -## Response - -The response contains basic node identification and build info for every node -matching the `{nodeId}` request parameter. +settings | A node's settings. This is a combination of the default settings, custom settings from the [configuration file]({{site.url}}{{site.baseurl}}/opensearch/configuration/#configuration-file), and dynamically [updated settings]({{site.url}}{{site.baseurl}}/opensearch/configuration/#update-cluster-settings-using-the-api). +os | Static information about the host OS, including version, processor architecture, and available/allocated processors. +process | Contains the process ID. +jvm | Detailed static information about the running JVM, including arguments. +thread_pool | Configured options for all individual thread pools. +transport | Mostly static information about the transport layer. +http | Mostly static information about the HTTP layer. +plugins | Information about installed plugins and modules. +ingest | Information about ingest pipelines and available ingest processors. +aggregations | Information about available [aggregations]({{site.url}}{{site.baseurl}}/opensearch/aggregations). +indices | Static index settings configured at the node level. + +## Query parameters + +You can include the following query parameters in your request. All query parameters are optional. + +Parameter | Type | Description +:--- |:-------| :--- +flat_settings| Boolean | Specifies whether to return the `settings` object of the response in flat format. Default is `false`. +timeout | Time | Sets the time limit for node response. Default value is `30s`. -Metric | Description -:--- |:---- -name | A node name. -transport_address | A node transport address. -host | A node host address. -ip | A node host ip address. -version | A node OpenSearch version. -build_type | A build type, like `rpm`,`docker`, `zip`, etc. -build_hash | A git commit hash of the build. -roles | A node's role. -attributes | A node attributes. - -The response also contains one or more metric groups, depending on the `{metrics}` request parameter. +#### Sample request + +The following query requests the `process` and `transport` metrics from the cluster manager node: ```json -GET /_nodes/master:true/process,transport?pretty +GET /_nodes/cluster_manager:true/process,transport +``` +#### Sample response + +The response contains the metric groups specified in the `` request parameter (in this case, `process` and `transport`): + +```json { "_nodes": { "total": 1, @@ -133,6 +132,34 @@ GET /_nodes/master:true/process,transport?pretty } ``` +## Response fields + +The response contains the basic node identification and build info for every node matching the `` request parameter. The following table lists the response fields. + +Field | Description +:--- |:---- +name | The node's name. +transport_address | The node's transport address. +host | The node's host address. +ip | The node's host IP address. +version | The node's OpenSearch version. +build_type | The node's build type, like `rpm`, `docker`, `tar`, etc. +build_hash | The git commit hash of the build. +total_indexing_buffer | The maximum heap size in bytes used to hold newly indexed documents. Once this heap size is exceeded, the documents are written to disk. +roles | The list of the node's roles. +attributes | The node's attributes. +os | Information about the OS, including name, version, architecture, refresh interval, and the number of available and allocated processors. +process | Information about the currently running process, including PID, refresh interval, and `mlockall`, which specifies whether the process address space has been successfully locked in memory. +jvm | Information about the JVM, including PID, version, memory information, garbage collector information, and arguments. +thread_pool | Information about the thread pool. +transport | Information about the transport address, including bound address, publish address, and profiles. +http | Information about the HTTP address, including bound address, publish address, and maximum content length, in bytes. +plugins | Information about the installed plugins, including name, version, OpenSearch version, Java version, description, class name, custom folder name, a list of extended plugins, and `has_native_controller`, which specifies whether the plugin has a native controller process. +modules | Information about the modules, including name, version, OpenSearch version, Java version, description, class name, custom folder name, a list of extended plugins, and `has_native_controller`, which specifies whether the plugin has a native controller process. Modules are different from plugins because modules are loaded into OpenSearch automatically, while plugins have to be installed manually. +ingest | Information about ingest pipelines and processors. +aggregations | Information about the available aggregation types. + + ## Required permissions If you use the security plugin, make sure you have the appropriate permissions: `cluster:monitor/nodes/info`. diff --git a/_opensearch/rest-api/nodes-apis/nodes-stats.md b/_opensearch/rest-api/nodes-apis/nodes-stats.md new file mode 100644 index 0000000000..02eacca91a --- /dev/null +++ b/_opensearch/rest-api/nodes-apis/nodes-stats.md @@ -0,0 +1,972 @@ +--- +layout: default +title: Nodes stats +parent: Nodes APIs +grand_parent: REST API reference +nav_order: 20 +--- + +# Nodes stats + +The nodes stats API returns statistics about your cluster. + +## Path and HTTP methods + +```json +GET /_nodes/stats +GET /_nodes//stats +GET /_nodes/stats/ +GET /_nodes//stats/ +GET /_nodes/stats// +GET /_nodes//stats// +``` + +## Path parameters + +The following table lists the available path parameters. All path parameters are optional. + +Parameter | Type | Description +:--- | :--- | :--- +nodeId | String | A comma-separated list of nodeIds used to filter results. Supports [node filters]({{site.url}}{{site.baseurl}}/opensearch/rest-api/nodes-apis/index/#node-filters). Defaults to `_all`. +metric | String | A comma-separated list of metric groups that will be included in the response. For example, `jvm,fs`. See the list of all metrics below. Defaults to all metrics. +index_metric | String | A comma-separated list of index metric groups that will be included in the response. For example, `docs,store`. See the list of all index metrics below. Defaults to all index metrics. + +The following table lists all available metric groups. + +Metric | Description +:--- |:---- +indices | Index statistics, such as size, document count, and search, index, and delete times for documents. +os | Statistics about the host OS, including load, memory, and swapping. +process | Statistics about processes, including their memory consumption, open file descriptors, and CPU usage. +jvm | Statistics about the JVM, including memory pool, buffer pool, and garbage collection, and the number of loaded classes. +fs | File system statistics, such as read/write statistics, data path, and free disk space. +transport | Transport layer statistics about send/receive in cluster communication. +http | Statistics about the HTTP layer. +breaker | Statistics about the field data circuit breakers. +script | Statistics about scripts, such as compilations and cache evictions. +discovery | Statistics about cluster states. +ingest | Statistics about ingest pipelines. +adaptive_selection | Statistics about adaptive replica selection, which selects an eligible node using shard allocation awareness. +script_cache | Statistics about script cache. +indexing_pressure | Statistics about the node's indexing pressure. +shard_indexing_pressure | Statistics about shard indexing pressure. + +To filter the information returned for the `indices` metric, you can use specific `index_metric` values. You can use these only when you use the following query types: + +```json +GET _nodes/stats/ +GET _nodes/stats/_all +GET _nodes/stats/indices +``` +The following index metrics are supported: + +- docs +- store +- indexing +- get +- search +- merge +- refresh +- flush +- warmer +- query_cache +- fielddata +- completion +- segments +- translog +- request_cache + +For example, the following query requests statistics for `docs` and `search`: + +```json +GET _nodes/stats/indices/docs,search +``` + +## Query parameters + +The following table lists the available query parameters. All query parameters are optional. + +Parameter | Type | Description +:--- | :--- | :--- +completion_fields | String | The fields to include in completion statistics. Supports comma-separated lists and wildcard expressions. +fielddata_fields | String | The fields to include in fielddata statistics. Supports comma-separated lists and wildcard expressions. +fields | String | The fields to include. Supports comma-separated lists and wildcard expressions. +groups | String | A comma-separated list of search groups to include in the search statistics. +level | String | Specifies whether statistics are aggregated at the cluster, index, or shard level. Valid values are `indices`, `node`, and `shard`. +timeout | Time | Sets the time limit for node response. Default is `30s`. +include_segment_file_sizes | Boolean | If segment statistics are requested, this field specifies to return the aggregated disk usage of every Lucene index file. Default is `false`. + +#### Sample request + +```json +GET _nodes/stats/ +``` + +#### Sample response + +```json +{ + "_nodes" : { + "total" : 1, + "successful" : 1, + "failed" : 0 + }, + "cluster_name" : "docker-cluster", + "nodes" : { + "F-ByTQzVQ3GQeYzQJArJGQ" : { + "timestamp" : 1664484195257, + "name" : "opensearch", + "transport_address" : "127.0.0.1:9300", + "host" : "127.0.0.1", + "ip" : "127.0.0.1:9300", + "roles" : [ + "master", + "data", + "ingest", + "remote_cluster_client" + ], + "attributes" : { + "shard_indexing_pressure_enabled" : "true" + }, + "indices" : { + "docs" : { + "count" : 13160, + "deleted" : 12 + }, + "store" : { + "size_in_bytes" : 6263461, + "reserved_in_bytes" : 0 + }, + "indexing" : { + "index_total" : 0, + "index_time_in_millis" : 0, + "index_current" : 0, + "index_failed" : 0, + "delete_total" : 204, + "delete_time_in_millis" : 427, + "delete_current" : 0, + "noop_update_total" : 0, + "is_throttled" : false, + "throttle_time_in_millis" : 0 + }, + "get" : { + "total" : 4, + "time_in_millis" : 18, + "exists_total" : 4, + "exists_time_in_millis" : 18, + "missing_total" : 0, + "missing_time_in_millis" : 0, + "current" : 0 + }, + "search" : { + "open_contexts" : 0, + "query_total" : 194, + "query_time_in_millis" : 467, + "query_current" : 0, + "fetch_total" : 194, + "fetch_time_in_millis" : 143, + "fetch_current" : 0, + "scroll_total" : 0, + "scroll_time_in_millis" : 0, + "scroll_current" : 0, + "suggest_total" : 0, + "suggest_time_in_millis" : 0, + "suggest_current" : 0 + }, + "merges" : { + "current" : 0, + "current_docs" : 0, + "current_size_in_bytes" : 0, + "total" : 1, + "total_time_in_millis" : 5, + "total_docs" : 12, + "total_size_in_bytes" : 3967, + "total_stopped_time_in_millis" : 0, + "total_throttled_time_in_millis" : 0, + "total_auto_throttle_in_bytes" : 251658240 + }, + "refresh" : { + "total" : 74, + "total_time_in_millis" : 201, + "external_total" : 57, + "external_total_time_in_millis" : 314, + "listeners" : 0 + }, + "flush" : { + "total" : 28, + "periodic" : 28, + "total_time_in_millis" : 1261 + }, + "warmer" : { + "current" : 0, + "total" : 45, + "total_time_in_millis" : 99 + }, + "query_cache" : { + "memory_size_in_bytes" : 0, + "total_count" : 0, + "hit_count" : 0, + "miss_count" : 0, + "cache_size" : 0, + "cache_count" : 0, + "evictions" : 0 + }, + "fielddata" : { + "memory_size_in_bytes" : 356, + "evictions" : 0 + }, + "completion" : { + "size_in_bytes" : 0, + "fields" : { } + }, + "segments" : { + "count" : 17, + "memory_in_bytes" : 0, + "terms_memory_in_bytes" : 0, + "stored_fields_memory_in_bytes" : 0, + "term_vectors_memory_in_bytes" : 0, + "norms_memory_in_bytes" : 0, + "points_memory_in_bytes" : 0, + "doc_values_memory_in_bytes" : 0, + "index_writer_memory_in_bytes" : 0, + "version_map_memory_in_bytes" : 0, + "fixed_bit_set_memory_in_bytes" : 288, + "max_unsafe_auto_id_timestamp" : -1, + "file_sizes" : { } + }, + "translog" : { + "operations" : 12, + "size_in_bytes" : 1452, + "uncommitted_operations" : 12, + "uncommitted_size_in_bytes" : 1452, + "earliest_last_modified_age" : 164160 + }, + "request_cache" : { + "memory_size_in_bytes" : 1649, + "evictions" : 0, + "hit_count" : 0, + "miss_count" : 18 + }, + "recovery" : { + "current_as_source" : 0, + "current_as_target" : 0, + "throttle_time_in_millis" : 0 + } + }, + "os" : { + "timestamp" : 1664484195263, + "cpu" : { + "percent" : 0, + "load_average" : { + "1m" : 0.0, + "5m" : 0.0, + "15m" : 0.0 + } + }, + "mem" : { + "total_in_bytes" : 13137076224, + "free_in_bytes" : 9265442816, + "used_in_bytes" : 3871633408, + "free_percent" : 71, + "used_percent" : 29 + }, + "swap" : { + "total_in_bytes" : 4294967296, + "free_in_bytes" : 4294967296, + "used_in_bytes" : 0 + }, + "cgroup" : { + "cpuacct" : { + "control_group" : "/", + "usage_nanos" : 338710071600 + }, + "cpu" : { + "control_group" : "/", + "cfs_period_micros" : 100000, + "cfs_quota_micros" : -1, + "stat" : { + "number_of_elapsed_periods" : 0, + "number_of_times_throttled" : 0, + "time_throttled_nanos" : 0 + } + }, + "memory" : { + "control_group" : "/", + "limit_in_bytes" : "9223372036854771712", + "usage_in_bytes" : "1432346624" + } + } + }, + "process" : { + "timestamp" : 1664484195263, + "open_file_descriptors" : 556, + "max_file_descriptors" : 65536, + "cpu" : { + "percent" : 0, + "total_in_millis" : 170870 + }, + "mem" : { + "total_virtual_in_bytes" : 6563344384 + } + }, + "jvm" : { + "timestamp" : 1664484195264, + "uptime_in_millis" : 21232111, + "mem" : { + "heap_used_in_bytes" : 308650480, + "heap_used_percent" : 57, + "heap_committed_in_bytes" : 536870912, + "heap_max_in_bytes" : 536870912, + "non_heap_used_in_bytes" : 147657128, + "non_heap_committed_in_bytes" : 152502272, + "pools" : { + "young" : { + "used_in_bytes" : 223346688, + "max_in_bytes" : 0, + "peak_used_in_bytes" : 318767104, + "peak_max_in_bytes" : 0, + "last_gc_stats" : { + "used_in_bytes" : 0, + "max_in_bytes" : 0, + "usage_percent" : -1 + } + }, + "old" : { + "used_in_bytes" : 67068928, + "max_in_bytes" : 536870912, + "peak_used_in_bytes" : 67068928, + "peak_max_in_bytes" : 536870912, + "last_gc_stats" : { + "used_in_bytes" : 34655744, + "max_in_bytes" : 536870912, + "usage_percent" : 6 + } + }, + "survivor" : { + "used_in_bytes" : 18234864, + "max_in_bytes" : 0, + "peak_used_in_bytes" : 32721280, + "peak_max_in_bytes" : 0, + "last_gc_stats" : { + "used_in_bytes" : 18234864, + "max_in_bytes" : 0, + "usage_percent" : -1 + } + } + } + }, + "threads" : { + "count" : 80, + "peak_count" : 80 + }, + "gc" : { + "collectors" : { + "young" : { + "collection_count" : 18, + "collection_time_in_millis" : 199 + }, + "old" : { + "collection_count" : 0, + "collection_time_in_millis" : 0 + } + } + }, + "buffer_pools" : { + "mapped" : { + "count" : 23, + "used_in_bytes" : 6232113, + "total_capacity_in_bytes" : 6232113 + }, + "direct" : { + "count" : 63, + "used_in_bytes" : 9050069, + "total_capacity_in_bytes" : 9050068 + }, + "mapped - 'non-volatile memory'" : { + "count" : 0, + "used_in_bytes" : 0, + "total_capacity_in_bytes" : 0 + } + }, + "classes" : { + "current_loaded_count" : 20693, + "total_loaded_count" : 20693, + "total_unloaded_count" : 0 + } + }, + "thread_pool" : { + "OPENSEARCH_ML_TASK_THREAD_POOL" : { + "threads" : 0, + "queue" : 0, + "active" : 0, + "rejected" : 0, + "largest" : 0, + "completed" : 0 + }, + "ad-batch-task-threadpool" : { + "threads" : 0, + "queue" : 0, + "active" : 0, + "rejected" : 0, + "largest" : 0, + "completed" : 0 + }, + ... + }, + "fs" : { + "timestamp" : 1664484195264, + "total" : { + "total_in_bytes" : 269490393088, + "free_in_bytes" : 261251477504, + "available_in_bytes" : 247490805760 + }, + "data" : [ + { + "path" : "/usr/share/opensearch/data/nodes/0", + "mount" : "/ (overlay)", + "type" : "overlay", + "total_in_bytes" : 269490393088, + "free_in_bytes" : 261251477504, + "available_in_bytes" : 247490805760 + } + ], + "io_stats" : { } + }, + "transport" : { + "server_open" : 0, + "total_outbound_connections" : 0, + "rx_count" : 0, + "rx_size_in_bytes" : 0, + "tx_count" : 0, + "tx_size_in_bytes" : 0 + }, + "http" : { + "current_open" : 5, + "total_opened" : 1108 + }, + "breakers" : { + "request" : { + "limit_size_in_bytes" : 322122547, + "limit_size" : "307.1mb", + "estimated_size_in_bytes" : 0, + "estimated_size" : "0b", + "overhead" : 1.0, + "tripped" : 0 + }, + "fielddata" : { + "limit_size_in_bytes" : 214748364, + "limit_size" : "204.7mb", + "estimated_size_in_bytes" : 356, + "estimated_size" : "356b", + "overhead" : 1.03, + "tripped" : 0 + }, + "in_flight_requests" : { + "limit_size_in_bytes" : 536870912, + "limit_size" : "512mb", + "estimated_size_in_bytes" : 0, + "estimated_size" : "0b", + "overhead" : 2.0, + "tripped" : 0 + }, + "parent" : { + "limit_size_in_bytes" : 510027366, + "limit_size" : "486.3mb", + "estimated_size_in_bytes" : 308650480, + "estimated_size" : "294.3mb", + "overhead" : 1.0, + "tripped" : 0 + } + }, + "script" : { + "compilations" : 0, + "cache_evictions" : 0, + "compilation_limit_triggered" : 0 + }, + "discovery" : { + "cluster_state_queue" : { + "total" : 0, + "pending" : 0, + "committed" : 0 + }, + "published_cluster_states" : { + "full_states" : 2, + "incompatible_diffs" : 0, + "compatible_diffs" : 10 + } + }, + "ingest" : { + "total" : { + "count" : 0, + "time_in_millis" : 0, + "current" : 0, + "failed" : 0 + }, + "pipelines" : { } + }, + "adaptive_selection" : { + "F-ByTQzVQ3GQeYzQJArJGQ" : { + "outgoing_searches" : 0, + "avg_queue_size" : 0, + "avg_service_time_ns" : 501024, + "avg_response_time_ns" : 794105, + "rank" : "0.8" + } + }, + "script_cache" : { + "sum" : { + "compilations" : 0, + "cache_evictions" : 0, + "compilation_limit_triggered" : 0 + }, + "contexts" : [ + { + "context" : "aggregation_selector", + "compilations" : 0, + "cache_evictions" : 0, + "compilation_limit_triggered" : 0 + }, + { + "context" : "aggs", + "compilations" : 0, + "cache_evictions" : 0, + "compilation_limit_triggered" : 0 + }, + ... + ] + }, + "indexing_pressure" : { + "memory" : { + "current" : { + "combined_coordinating_and_primary_in_bytes" : 0, + "coordinating_in_bytes" : 0, + "primary_in_bytes" : 0, + "replica_in_bytes" : 0, + "all_in_bytes" : 0 + }, + "total" : { + "combined_coordinating_and_primary_in_bytes" : 40256, + "coordinating_in_bytes" : 40256, + "primary_in_bytes" : 45016, + "replica_in_bytes" : 0, + "all_in_bytes" : 40256, + "coordinating_rejections" : 0, + "primary_rejections" : 0, + "replica_rejections" : 0 + }, + "limit_in_bytes" : 53687091 + } + }, + "shard_indexing_pressure" : { + "stats" : { }, + "total_rejections_breakup_shadow_mode" : { + "node_limits" : 0, + "no_successful_request_limits" : 0, + "throughput_degradation_limits" : 0 + }, + "enabled" : false, + "enforced" : false + } + } + } +} +``` + +## Response fields + +The following table lists all response fields. + +| Field | Data Type | Description | +| :--- | :--- | :--- | +| _nodes | Object | Statistics about the nodes that are returned. | +| _nodes.total | Integer | The total number of nodes for this request. | +| _nodes.successful | Integer | The number of nodes for which the request was successful. | +| _nodes.failed | Integer | The number of nodes for which the request failed. If there are nodes for which the request failed, the failure message is included. | +| cluster_name | String | The name of the cluster. | +| [nodes](#nodes) | Object | Statistics for the nodes included in this request. | + +### `nodes` + +The `nodes` object contains all nodes that are returned by the request, along with their IDs. Each node has the following properties. + +Field | Data Type | Description +:--- | :--- | :--- +timestamp | Integer | The time the nodes statistics were collected, in milliseconds since the epoch. +name | String | The name of the node. +transport_address | IP address | The host and port of the transport layer that is used by nodes in a cluster to communicate internally. +host | IP address | The network host of the node. +ip | IP address | The IP address and port of the node. +roles | Array | The roles of the node (for example, `master`, `data`, or `ingest`). +attributes | Object | The attributes of the node (for example, `shard_indexing_pressure_enabled`). +[indices](#indices) | Object | Index statistics for each index that has shards on the node. +[os](#os) | Object | Statistics about the OS for the node. +[process](#process) | Object | Process statistics for the node. +[jvm](#jvm) | Object | Statistics about the JVM for the node. +[thread_pool](#thread_pool)| Object | Statistics about each thread pool for the node. +[fs](#fs) | Object | Statistics about the file stores for the node. +[transport](#transport) | Object | Transport statistics for the node. +http | Object | HTTP statistics for the node. +http.current_open | Integer | The number of currently open HTTP connections for the node. +http.total_opened | Integer | The total number of HTTP connections the node has opened since it started. +[breakers](#breakers) | Object | Statistics about the circuit breakers for the node. +[script](#script-and-script_cache)| Object | Script statistics for the node. +[script_cache](#script-and-script_cache)| Object | Script cache statistics for the node. +[discovery](#discovery) | Object | Node discovery statistics for the node. +[ingest](#ingest) | Object | Ingest statistics for the node. +[adaptive_selection](#adaptive_selection) | Object | Statistics about adaptive selections for the node. +[indexing_pressure](#indexing_pressure) | Object | Statistics related to the node's indexing pressure. +[shard_indexing_pressure](#shard_indexing_pressure) | Object | Statistics related to indexing pressure at the shard level. + +### `indices` + +The `indices` object contains the index statistics for each index with shards on this node. Each index has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +docs | Object | Document statistics for all primary shards that exist on the node. +docs.
    count | Integer | The number of documents reported by Lucene. Excludes deleted documents and recently indexed documents that are not yet assigned to a segment. Nested documents are counted separately. +docs.
    deleted | Integer | The number of deleted documents reported by Lucene. Excludes recent deletion operations that have not yet affect the segment. +store | Object | Statistics about the shard sizes of the shards on the node. +store.
    size_in_bytes | Integer | Total size of all shards on the node. +store.
    reserved_in_bytes | Integer | The predicted number of bytes the shard store will grow to be because of activities such as restoring snapshots and peer recoveries. +indexing | Object | Statistics about indexing operations for the node. +indexing.
    index_total | Integer | The total number of indexing operations on the node. +indexing.
    index_time_in_millis | Integer | The total time for all indexing operations, in milliseconds. +indexing.
    index_current | Integer | The number of indexing operations that are currently running. +indexing.
    index_failed | Integer | The number of indexing operations that have failed. +indexing.
    delete_total | Integer | The total number of deletions. +indexing.
    delete_time_in_millis | Integer | The total time for all deletion operations, in milliseconds. +indexing.
    delete_current | Integer | The number of deletion operations that are currently running. +indexing.
    noop_update_total | Integer | The total number of noop operations. +indexing.
    is_throttled | Boolean | Specifies whether any operations were throttled. +indexing.
    throttle_time_in_millis | Integer | The total time for throttling operations, in milliseconds. +get | Object | Statistics about the get operations for the node. +get.
    total | Integer | The total number of get operations. +get.
    time_in_millis | Integer | The total time for all get operations, in milliseconds. +get.
    exists_total | Integer | The total number of successful get operations. +get.
    exists_time_in_millis | Integer | The total time for all successful get operations, in milliseconds. +get.
    missing_total | Integer | The number of failed get operations. +get.
    missing_time_in_millis | Integer | The total time for all failed get operations, in milliseconds. +get.
    current | Integer | The number of get operations that are currently running. +search | Object | Statistics about the search operations for the node. +search.
    open_contexts | Integer | The number of open search contexts. +search.
    query_total | Integer | The total number of query operations. +search.
    query_time_in_millis | Integer | The total time for all query operations, in milliseconds. +search.
    query_current | Integer | The number of query operations that are currently running. +search.
    fetch_total | Integer | The total number of fetch operations. +search.
    fetch_time_in_millis | Integer | The total time for all fetch operations, in milliseconds. +search.
    fetch_current | Integer | The number of fetch operations that are currently running. +search.
    scroll_total | Integer | The total number of scroll operations. +search.
    scroll_time_in_millis | Integer | The total time for all scroll operations, in milliseconds. +search.
    scroll_current | Integer | The number of scroll operations that are currently running. +search.
    suggest_total | Integer | The total number of suggest operations. +search.
    suggest_time_in_millis | Integer | The total time for all suggest operations, in milliseconds. +search.
    suggest_current | Integer | The number of suggest operations that are currently running. +merges | Object | Statistics about merge operations for the node. +merges.
    current | Integer | The number of merge operations that are currently running. +merges.
    current_docs | Integer | The number of document merges that are currently running. +merges.
    current_size_in_bytes | Integer | The memory size, in bytes, that is used to perform current merge operations. +merges.
    total | Integer | The total number of merge operations. +merges.
    total_time_in_millis | Integer | The total time for merges, in milliseconds. +merges.
    total_docs | Integer | The total number of documents that have been merged. +merges.
    total_size_in_bytes | Integer | The total size of all merged documents, in bytes. +merges.
    total_stopped_time_in_millis | Integer | The total time spent on stopping merge operations, in milliseconds. +merges.
    total_throttled_time_in_millis | Integer | The total time spent on throttling merge operations, in milliseconds. +merges.
    total_auto_throttle_in_bytes | Integer | The total size of automatically throttled merge operations, in bytes. +refresh | Object | Statistics about refresh operations for the node. +refresh.
    total | Integer | The total number of refresh operations. +refresh.
    total_time_in_millis | Integer | The total time for all refresh operations, in milliseconds. +refresh.
    external_total | Integer | The total number of external refresh operations. +refresh.
    external_total_time_in_millis | Integer | The total time for all external refresh operations, in milliseconds. +refresh.
    listeners | Integer | The number of refresh listeners. +flush | Object | Statistics about flush operations for the node. +flush.
    total | Integer | The total number of flush operations. +flush.
    periodic | Integer | The total number of periodic flush operations. +flush.
    total_time_in_millis | Integer | The total time for all flush operations, in milliseconds. +warmer | Object | Statistics about the index warming operations for the node. +warmer.
    current | Integer | The number of current index warming operations. +warmer.
    total | Integer | The total number of index warming operations. +warmer.
    total_time_in_millis | Integer | The total time for all index warming operations, in milliseconds. +query_cache | Statistics about query cache operations for the node. +query_cache.
    memory_size_in_bytes | Integer | The amount of memory used for the query cache for all shards in the node. +query_cache.
    total_count | Integer | The total number of hits, misses, and cached queries in the query cache. +query_cache.
    hit_count | Integer | The total number of hits in the query cache. +query_cache.
    miss_count | Integer | The total number of misses in the query cache. +query_cache.
    cache_size | Integer | The size of the query cache, in bytes. +query_cache.
    cache_count | Integer | The number of queries in the query cache. +query_cache.
    evictions | Integer | The number of evictions in the query cache. +fielddata | Object | Statistics about the field data cache for all shards in the node. +fielddata.
    memory_size_in_bytes | Integer | The total amount of memory used for the field data cache for all shards in the node. +fielddata.
    evictions | Integer | The number of evictions in the field data cache. +fielddata.
    fields | Object | Contains all field data fields. +completion | Object | Statistics about completions for all shards in the node. +completion.
    size_in_bytes | Integer | The total amount of memory used for completion for all shards in the node, in bytes. +completion.
    fields | Object | Contains completion fields. +segments | Object | Statistics about segments for all shards in the node. +segments.
    count | Integer | The total number of segments. +segments.
    memory_in_bytes | Integer | The total amount of memory, in bytes. +segments.
    terms_memory_in_bytes | Integer | The total amount of memory used for terms, in bytes. +segments.
    stored_fields_memory_in_bytes | Integer | The total amount of memory used for stored fields, in bytes. +segments.
    term_vectors_memory_in_bytes | Integer | The total amount of memory used for term vectors, in bytes. +segments.
    norms_memory_in_bytes | Integer | The total amount of memory used for normalization factors, in bytes. +segments.
    points_memory_in_bytes | Integer | The total amount of memory used for points, in bytes. +segments.
    doc_values_memory_in_bytes | Integer | The total amount of memory used for doc values, in bytes. +segments.
    index_writer_memory_in_bytes | Integer | The total amount of memory used by all index writers, in bytes. +segments.
    version_map_memory_in_bytes | Integer | The total amount of memory used by all version maps, in bytes. +segments.
    fixed_bit_set_memory_in_bytes | Integer | The total amount of memory used by fixed bit sets, in bytes. Fixed bit sets are used for nested objects and join fields. +segments.
    max_unsafe_auto_id_timestamp | Integer | The timestamp for the most recently retired indexing request, in milliseconds since the epoch. +segments.
    file_sizes | Integer | Statistics about the size of the segment files. +translog | Object | Statistics about transaction log operations for the node. +translog.
    operations | Integer | The number of translog operations. +translog.
    size_in_bytes | Integer | The size of the translog, in bytes. +translog.
    uncommitted_operations | Integer | The number of uncommitted translog operations. +translog.
    uncommitted_size_in_bytes | Integer | The size of uncommitted translog operations, in bytes. +translog.
    earliest_last_modified_age | Integer | The earliest last modified age for the translog. +request_cache | Object | Statistics about the request cache for the node. +request_cache.
    memory_size_in_bytes | Integer | The memory size used by the request cache, in bytes. +request_cache.
    evictions | Integer | The number of request cache evictions. +request_cache.
    hit_count | Integer | The number of request cache hits. +request_cache.
    miss_count | Integer | The number of request cache misses. +recovery | Object | Statistics about recovery operations for the node. +recovery.
    current_as_source | Integer | The number of recovery operations that have used an index shard as a source. +recovery.
    current_as_target | Integer | The number of recovery operations that have used an index shard as a target. +recovery.
    throttle_time_in_millis | Integer | The delay of recovery operations due to throttling, in milliseconds. + + +### `os` + +The `os` object has the OS statistics for the node and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +timestamp | Integer | The last refresh time for the OS statistics, in milliseconds since the epoch. +cpu | Object | Statistics about the node's CPU usage. +cpu.percent | Integer | Recent CPU usage for the system. +cpu.load_average | Object | Statistics about load averages for the system. +cpu.load_average.1m | Float | The load average for the system for the time period of one minute. +cpu.load_average.5m | Float | The load average for the system for the time period of five minutes. +cpu.load_average.15m | Float | The load average for the system for the time period of 15 minutes. +cpu.mem | Object | Statistics about memory usage for the node. +cpu.mem.    total_in_bytes | Integer | The total amount of physical memory, in bytes. +cpu.mem.    free_in_bytes | Integer | The total amount of free physical memory, in bytes. +cpu.mem.    used_in_bytes | Integer | The total amount of used physical memory, in bytes. +cpu.mem.    free_percent | Integer | The percentage of memory that is free. +cpu.mem.    used_percent | Integer | The percentage of memory that is used. +cpu.swap | Object | Statistics about swap space for the node. +cpu.swap.    total_in_bytes | Integer | The total amount of swap space, in bytes. +cpu.swap.free_in_bytes | Integer | The total amount of free swap space, in bytes. +cpu.swap.    used_in_bytes | Integer | The total amount of used swap space, in bytes. +cpu.cgroup | Object | Contains cgroup statistics for the node. Returned for Linux only. +cpu.cgroup.cpuacct | Object | Statistics about the cpuacct control group for the node. +cpu.cgroup.    cpu | Object | Statistics about the CPU control group for the node. +cpu.cgroup.    memory | Object | Statistics about the memory control group for the node. + +### `process` + +The `process` object contains process statistics for the node and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +timestamp | Integer | The last refresh time for the process statistics, in milliseconds since the epoch. +open_file_descriptors | Integer | The number of opened file descriptors associated with the current process. +max_file_descriptors | Integer | The maximum number of file descriptors for the system. +cpu | Object | Statistics about the CPU for the node. +cpu.percent | Integer | The percentage of CPU usage for the process. +cpu.total_in_millis | Integer | The total CPU time used by the process on which the JVM is running, in milliseconds. +mem | Object | Statistics about the memory for the node. +mem.total_virtual_in_bytes | Integer | The total amount of virtual memory that is guaranteed to be available to the process that is currently running, in bytes. + + +### `jvm` + +The `jvm` object contains statistics about the JVM for the node and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +timestamp | Integer | The last refresh time for the JVM statistics, in milliseconds since the epoch. +uptime_in_millis | Integer | The JVM uptime, in milliseconds. +mem | Object | Statistics for the JVM memory usage on the node. +mem.heap_used_in_bytes | Integer | The amount of memory that is currently being used, in bytes. +mem.heap_used_percent | Integer | The percentage of memory that is currently used by the heap. +mem.heap_committed_in_bytes | Integer | The amount of memory available for use by the heap, in bytes. +mem.heap_max_in_bytes | Integer | The maximum amount of memory available for use by the heap, in bytes. +mem.non_heap_used_in_bytes | Integer | The amount of non-heap memory that is currently used, in bytes. +mem.non_heap_committed_in_bytes | Integer | The maximum amount of non-heap memory available for use, in bytes. +mem.pools | Object | Statistics about heap memory usage for the node. +mem.pools.young | Object | Statistics about the young generation heap memory usage for the node. Contains the amount of memory used, the maximum amount of memory available, and the peak amount of memory used. +mem.pools.old | Object | Statistics about the old generation heap memory usage for the node. Contains the amount of memory used, the maximum amount of memory available, and the peak amount of memory used. +mem.pools.survivor | Object | Statistics about the survivor space memory usage for the node. Contains the amount of memory used, the maximum amount of memory available, and the peak amount of memory used. +threads | Object | Statistics about the JVM thread usage for the node. +threads.count | Integer | The number of threads that are currently active in the JVM. +threads.peak_count | Integer | The maximum number of threads in the JVM. +gc.collectors | Object | Statistics about the JVM garbage collectors for the node. +gc.collectors.young | Integer | Statistics about JVM garbage collectors that collect young generation objects. +gc.collectors.young.collection_count | Integer | The number of garbage collectors that collect young generation objects. +gc.collectors.young.collection_time_in_millis | Integer | The total time spent on garbage collection of young generation objects, in milliseconds. +gc.collectors.old | Integer | Statistics about JVM garbage collectors that collect old generation objects. +gc.collectors.old.collection_count | Integer | The number of garbage collectors that collect old generation objects. +gc.collectors.old.collection_time_in_millis | Integer | The total time spent on garbage collection of old generation objects, in milliseconds. +buffer_pools | Object | Statistics about the JVM buffer pools for the node. +buffer_pools.mapped | Object | Statistics about the mapped JVM buffer pools for the node. +buffer_pools.mapped.count | Integer | The number of mapped buffer pools. +buffer_pools.mapped.used_in_bytes | Integer | The amount of memory used by mapped buffer pools, in bytes. +buffer_pools.mapped.total_capacity_in_bytes | Integer | The total capacity of the mapped buffer pools, in bytes. +buffer_pools.direct | Object | Statistics about the direct JVM buffer pools for the node. +buffer_pools.direct.count | Integer | The number of direct buffer pools. +buffer_pools.direct.used_in_bytes | Integer | The amount of memory used by direct buffer pools, in bytes. +buffer_pools.direct.total_capacity_in_bytes | Integer | The total capacity of the direct buffer pools, in bytes. +classes | Object | Statistics about the classes loaded by the JVM for the node. +classes.current_loaded_count | Integer | The number of classes currently loaded by the JVM. +classes.total_loaded_count | Integer | The total number of classes loaded by the JVM since it started. +classes.total_unloaded_count | Integer | The total number of classes unloaded by the JVM since it started. + +### `thread_pool` + +The `thread_pool` object contains a list of all thread pools. Each thread pool is a nested object specified by its ID with the properties listed below. + +Field | Field type | Description +:--- | :--- | :--- +threads | Integer | The number of threads in the pool. +queue | Integer | The number of threads in queue. +active | Integer | The number of active threads in the pool. +rejected | Integer | The number of tasks that have been rejected. +largest | Integer | The peak number of threads in the pool. +completed | Integer | The number of tasks completed. + +### `fs` + +The `fs` object represents statistics about the file stores for the node. It has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +timestamp | Integer | The last refresh time for the file store statistics, in milliseconds since the epoch. +total | Object | Statistics for all file stores of the node. +total.total_in_bytes | Integer | The total memory size of all file stores, in bytes. +total.free_in_bytes | Integer | The total unallocated disk space in all file stores, in bytes. +total.available_in_bytes | Integer | The total disk space available to the JVM on all file stores. Represents the actual amount of memory, in bytes, that OpenSearch can use. +data | Array | The list of all file stores. Each file store has the properties listed below. +data.path | String | The path to the file store. +data.mount | String | The mount point of the file store. +data.type | String | The type of the file store (for example, overlay). +data.total_in_bytes | Integer | The total size of the file store, in bytes. +data.free_in_bytes | Integer | The total unallocated disk space in the file store, in bytes. +data.available_in_bytes | Integer | The total amount of disk space available to the JVM for the file store, in bytes. +io_stats | Object | I/O statistics for the node (Linux only). Includes devices, read and write operations, and the I/O operation time. + +### `transport` + +The `transport` object has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +server_open | Integer | The number of open inbound TCP connections that OpenSearch nodes use for internal communication. +total_outbound_connections | Integer | The total number of outbound transport connections that the node has opened since it started. +rx_count | Integer | The total number of RX (receive) packets the node received during internal communication. +rx_size_in_bytes | Integer | The total size of RX packets the node received during internal communication, in bytes. +tx_count | Integer | The total number of TX (transmit) packets the node sent during internal communication. +tx_size_in_bytes | Integer | The total size of TX (transmit) packets the node sent during internal communication, in bytes. + +### `breakers` + +The `breakers` object contains statistics about the circuit breakers for the node. Each circuit breaker is a nested object listed by name and contains the following properties. + +Field | Field type | Description +:--- | :--- | :--- +limit_size_in_bytes | Integer | The memory limit for the circuit breaker, in bytes. +limit_size | Byte value | The memory limit for the circuit breaker in human-readable format (for example, `307.1mb`). +estimated_size_in_bytes | Integer | The estimated memory used for the operation, in bytes. +estimated_size | Byte value | The estimated memory used for the operation in human-readable format (for example, `356b`). +overhead | Float | A factor that all estimates are multiplied by to calculate the final estimate. +tripped | Integer | The total number of times the circuit breaker has been activated to prevent an out-of-memory error. + +### `script` and `script_cache` + +The `script` and `script_cache` objects have the following properties. + +Field | Field type | Description +:--- | :--- | :--- +script | Object | Script statistics for the node. +script.compilations | Integer | The total number of script compilations for the node. +script.cache_evictions| Integer | The total number of times the script cache has purged old data. +script.compilation_limit_triggered | Integer | The total number of times script compilation was limited by a circuit breaker. +script_cache | Object | Script cache statistics for the node. +script_cache.sum.compilations | Integer | The total number of script compilations in the cache for the node. +script_cache.sum.cache_evictions| Integer | The total number of times the script cache has purged old data. +script_cache.sum.compilation_limit_triggered | Integer | The total number of times script compilation in the cache was limited by a circuit breaker. +script_cache.contexts | Array of objects | The list of contexts for the script cache. Each context contains its name, the number of compilations, the number of cache evictions, and the number of times the script was limited by a circuit breaker. + +### `discovery` + +The `discovery` object contains the node discovery statistics and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +cluster_state_queue | Object | Cluster state queue statistics for the node. +cluster_state_queue.total | Integer | The total number of cluster states in the queue. +cluster_state_queue.pending | Integer | The number of pending cluster states in the queue. +cluster_state_queue.committed | Integer | The number of committed cluster states in the queue. +published_cluster_states | Object | Statistics for the published cluster states for the node. +published_cluster_states.full_states | Integer | The number of published cluster states. +published_cluster_states.incompatible_diffs | Integer | The number of incompatible differences between published cluster states. +published_cluster_states.compatible_diffs | Integer | The number of compatible differences between published cluster states. + +### `ingest` + +The `ingest` object contains the ingest statistics and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +total | Integer | Ingest statistics for the node's lifetime. +total.count | Integer | The total number of documents ingested by the node. +total.time_in_millis | Integer | The total amount of time for preprocessing ingest documents, in milliseconds. +total.current | Integer | The total number of documents that are currently being ingested by the node. +total.failed | Integer | The total number of failed ingestions for the node. +pipelines | Object | Ingest pipeline statistics for the node. Each pipeline is a nested object specified by its ID with the properties listed below. +pipelines._id_.count | Integer | The number of documents preprocessed by the ingest pipeline. +pipelines._id_.time_in_millis | Integer | The total amount of time for preprocessing documents in the ingest pipeline, in milliseconds. +pipelines._id_.failed | Integer | The total number of failed ingestions for the ingest pipeline. +pipelines._id_.processors | Array of objects | Statistics for the ingest processors. Includes the number of documents that are currently transformed, the total number of transformed documents, the number of failed transformations, and the time spent transforming documents. + +### `adaptive_selection` + +The `adaptive_selection` object contains the adaptive selection statistics. Each entry is specified by the node ID and has the properties listed below. + +Field | Field type | Description +:--- | :--- | :--- +outgoing_searches | Integer | The number of outgoing search requests for the node. +avg_queue_size | Integer | The rolling average queue size of search requests for the node (exponentially weighted). +avg_service_time_ns | Integer | The rolling average service time for search requests, in nanoseconds (exponentially weighted). +avg_response_time_ns | Integer | The rolling average response time for search requests, in nanoseconds (exponentially weighted). +rank | String | The node's rank that is used to select shards when routing requests. + +### `indexing_pressure` + +The `indexing_pressure` object contains the indexing pressure statistics and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +memory | Object | Statistics related to memory consumption for the indexing load. +memory.
    current | Object | Statistics related to memory consumption for the current indexing load. +memory.
    current.
        combined_coordinating_and_primary_in_bytes | Integer | The total memory used by indexing requests in the coordinating or primary stages, in bytes. A node can reuse the coordinating memory if the primary stage is run locally, so the total memory does not necessarily equal the sum of the coordinating and primary stage memory usage. +memory.
    current.
        coordinating_in_bytes | The total memory consumed by indexing requests in the coordinating stage, in bytes. +memory.
    current.
        primary_in_bytes | Integer | The total memory consumed by indexing requests in the primary stage, in bytes. +memory.
    current.
        replica_in_bytes | Integer | The total memory consumed by indexing requests in the replica stage, in bytes. +memory.
    current.
        all_in_bytes | Integer | The total memory consumed by indexing requests in the coordinating, primary, or replica stages. + +### `shard_indexing_pressure` + +The `shard_indexing_pressure` object contains the [shard indexing pressure]({{site.url}}{{site.baseurl}}/opensearch/shard-indexing-backpressure) statistics and has the following properties. + +Field | Field type | Description +:--- | :--- | :--- +[stats]({{site.url}}{{site.baseurl}}/opensearch/stats-api/) | Object | Statistics about shard indexing pressure. +total_rejections_breakup_shadow_mode | Object | If running in shadow mode, the `total_rejections_breakup_shadow_mode` object contains statistics about the request rejection criteria of all shards in the node. +total_rejections_breakup_shadow_mode.
    node_limits | Integer | The total number of rejections due to the node memory limit. When all shards reach the memory limit assigned to the node (for example, 10% of heap size), the shard is unable to take in more traffic on the node, and the indexing request is rejected. +total_rejections_breakup_shadow_mode.
    no_successful_request_limits | Integer | The total number of rejections when the node occupancy level is breaching its soft limit and the shard has multiple outstanding requests that are waiting to be executed. In this case, additional indexing requests are rejected until the system recovers. +total_rejections_breakup_shadow_mode.
    throughput_degradation_limits | Integer | The total number of rejections when the node occupancy level is breaching its soft limit and there is a constant deterioration in the request turnaround at the shard level. In this case, additional indexing requests are rejected until the system recovers. +enabled | Boolean | Specifies whether the shard indexing pressure feature is turned on for the node. +enforced | Boolean | If true, the shard indexing pressure runs in enforced mode (there are rejections). If false, the shard indexing pressure runs in shadow mode (there are no rejections, but statistics are recorded and can be retrieved in the `total_rejections_breakup_shadow_mode` object). Only applicable if shard indexing pressure is enabled. + +## Required permissions + +If you use the security plugin, make sure you have the appropriate permissions: `cluster:monitor/nodes/stats`. diff --git a/_search-plugins/knn/approximate-knn.md b/_search-plugins/knn/approximate-knn.md index 776586d235..722e74a5a2 100644 --- a/_search-plugins/knn/approximate-knn.md +++ b/_search-plugins/knn/approximate-knn.md @@ -269,39 +269,39 @@ GET my-knn-index-1/_search A space corresponds to the function used to measure the distance between two points in order to determine the k-nearest neighbors. From the k-NN perspective, a lower score equates to a closer and better result. This is the opposite of how OpenSearch scores results, where a greater score equates to a better result. To convert distances to OpenSearch scores, we take 1 / (1 + distance). The k-NN plugin the spaces the plugin supports are below. Not every method supports each of these spaces. Be sure to check out [the method documentation]({{site.url}}{{site.baseurl}}/search-plugins/knn/knn-index#method-definitions) to make sure the space you are interested in is supported. - + - + - - - + + + - - - + + + - - + + - - + + - - + +
spaceTypeDistance FunctionDistance Function (d) OpenSearch Score
l2\[ Distance(X, Y) = \sum_{i=1}^n (X_i - Y_i)^2 \]1 / (1 + Distance Function)l1\[ d(\mathbf{x}, \mathbf{y}) = \sum_{i=1}^n |x_i - y_i| \]\[ score = {1 \over 1 + d } \]
l1\[ Distance(X, Y) = \sum_{i=1}^n (X_i - Y_i) \]1 / (1 + Distance Function)l2\[ d(\mathbf{x}, \mathbf{y}) = \sum_{i=1}^n (x_i - y_i)^2 \]\[ score = {1 \over 1 + d } \]
linf\[ Distance(X, Y) = Max(X_i - Y_i) \]1 / (1 + Distance Function)\[ d(\mathbf{x}, \mathbf{y}) = max(|x_i - y_i|) \]\[ score = {1 \over 1 + d } \]
cosinesimil\[ 1 - {A · B \over \|A\| · \|B\|} = 1 - - {\sum_{i=1}^n (A_i · B_i) \over \sqrt{\sum_{i=1}^n A_i^2} · \sqrt{\sum_{i=1}^n B_i^2}}\] - where \(\|A\|\) and \(\|B\|\) represent normalized vectors.1 / (1 + Distance Function)\[ d(\mathbf{x}, \mathbf{y}) = 1 - cos { \theta } = 1 - {\mathbf{x} · \mathbf{y} \over \|\mathbf{x}\| · \|\mathbf{y}\|}\]\[ = 1 - + {\sum_{i=1}^n x_i y_i \over \sqrt{\sum_{i=1}^n x_i^2} · \sqrt{\sum_{i=1}^n y_i^2}}\] + where \(\|\mathbf{x}\|\) and \(\|\mathbf{y}\|\) represent normalized vectors.\[ score = {1 \over 1 + d } \]
innerproduct\[ Distance(X, Y) = - {A · B} \]if (Distance Function >= 0) 1 / (1 + Distance Function) else -Distance Function + 1\[ d(\mathbf{x}, \mathbf{y}) = - {\mathbf{x} · \mathbf{y}} = - \sum_{i=1}^n x_i y_i \]\[ \text{If} d \ge 0, \] \[score = {1 \over 1 + d }\] \[\text{If} d < 0, score = −d + 1\]
diff --git a/_search-plugins/knn/knn-score-script.md b/_search-plugins/knn/knn-score-script.md index a02e85defd..1f77d8ff4f 100644 --- a/_search-plugins/knn/knn-score-script.md +++ b/_search-plugins/knn/knn-score-script.md @@ -282,44 +282,44 @@ GET my-long-index/_search A space corresponds to the function used to measure the distance between two points in order to determine the k-nearest neighbors. From the k-NN perspective, a lower score equates to a closer and better result. This is the opposite of how OpenSearch scores results, where a greater score equates to a better result. The following table illustrates how OpenSearch converts spaces to scores: - + - + - - - + + + - - - + + + - - + + - - + + - - - + + + - - + +
spaceTypeDistance FunctionDistance Function (d) OpenSearch Score
l2\[ Distance(X, Y) = \sum_{i=1}^n (X_i - Y_i)^2 \]1 / (1 + Distance Function)l1\[ d(\mathbf{x}, \mathbf{y}) = \sum_{i=1}^n |x_i - y_i| \]\[ score = {1 \over 1 + d } \]
l1\[ Distance(X, Y) = \sum_{i=1}^n (X_i - Y_i) \]1 / (1 + Distance Function)l2\[ d(\mathbf{x}, \mathbf{y}) = \sum_{i=1}^n (x_i - y_i)^2 \]\[ score = {1 \over 1 + d } \]
linf\[ Distance(X, Y) = Max(X_i - Y_i) \]1 / (1 + Distance Function)\[ d(\mathbf{x}, \mathbf{y}) = max(|x_i - y_i|) \]\[ score = {1 \over 1 + d } \]
cosinesimil\[ {A · B \over \|A\| · \|B\|} = - {\sum_{i=1}^n (A_i · B_i) \over \sqrt{\sum_{i=1}^n A_i^2} · \sqrt{\sum_{i=1}^n B_i^2}}\] - where \(\|A\|\) and \(\|B\|\) represent normalized vectors.1 + Distance Function\[ d(\mathbf{x}, \mathbf{y}) = cos \theta = {\mathbf{x} · \mathbf{y} \over \|\mathbf{x}\| · \|\mathbf{y}\|}\]\[ = + {\sum_{i=1}^n x_i y_i \over \sqrt{\sum_{i=1}^n x_i^2} · \sqrt{\sum_{i=1}^n y_i^2}}\] + where \(\|\mathbf{x}\|\) and \(\|\mathbf{y}\|\) represent normalized vectors.\[ score = 1 + d \]
innerproduct\[ Distance(X, Y) = \sum_{i=1}^n X_iY_i \]1 / (1 + Distance Function)innerproduct (not supported for Lucene)\[ d(\mathbf{x}, \mathbf{y}) = - {\mathbf{x} · \mathbf{y}} = - \sum_{i=1}^n x_i y_i \]\[ \text{If} d \ge 0, \] \[score = {1 \over 1 + d }\] \[\text{If} d < 0, score = −d + 1\]
hammingbitDistance = countSetBits(X \(\oplus\) Y) 1 / (1 + Distance Function)\[ d(\mathbf{x}, \mathbf{y}) = \text{countSetBits}(\mathbf{x} \oplus \mathbf{y})\]\[ score = {1 \over 1 + d } \]