Add indexing pressure documentation

docs/reference/cluster/nodes-stats.asciidoc

@@ -58,6 +58,10 @@ using metrics.
   `http`::
       HTTP connection information.
 
+  `indexing_pressure`::
+        Indexing pressure statistics about total and current indexing load and
+        indexing rejections.
+
   `indices`::
       Indices stats about size, document count, indexing and deletion times,
       search times, field cache size, merges and flushes.
@@ -2099,6 +2103,62 @@ Number of failed operations for the processor.
 =======
 ======
 
+[[cluster-nodes-stats-api-response-body-indexing-pressure]]
+`indexing_pressure`::
+(object)
+Contains indexing pressure statistics for the node.
++
+.Properties of `indexing_pressure`
+[%collapsible%open]
+======
+`total`::
+(object)
+Contains statistics for cumulative indexing load since the node started.
++
+.Properties of `<total>`
+[%collapsible%open]
+=======
+`coordinating_and_primary_bytes`::
+(integer)
+Bytes consumed by indexing requests in the coordinating or primary stage.
+
+`replica_bytes`::
+(integer)
+Bytes consumed by indexing requests in the replica stage.
+
+`all_bytes`::
+(integer)
+Bytes consumed by indexing requests in the coordinating, primary, or replica stage.
+
+`coordinating_and_primary_memory_limit_rejections`::
+(integer)
+Rejections of indexing requests in the coordinating or primary stage.
+
+`replica_memory_limit_rejections`::
+(integer)
+Rejections of indexing requests in the replica stage.
+=======
+`current`::
+(object)
+Contains statistics for current indexing load.
++
+.Properties of `<current>`
+[%collapsible%open]
+=======
+`coordinating_and_primary_bytes`::
+(integer)
+Bytes consumed by indexing requests in the coordinating or primary stage.
+
+`replica_bytes`::
+(integer)
+Bytes consumed by indexing requests in the replica stage.
+
+`all_bytes`::
+(integer)
+Bytes consumed by indexing requests in the coordinating, primary, or replica stage.
+=======
+======
+
 [[cluster-nodes-stats-api-response-body-adaptive-selection]]
 `adaptive_selection`::
 (object)

docs/reference/index-modules/indexing-pressure.asciidoc

@@ -0,0 +1,56 @@
+[[index-modules-indexing-pressure]]
+== Indexing Pressure
+
+Indexing documents into Elasticsearch introduces system load in the form of
+memory and CPU load. Each indexing operation includes coordinating, primary, and
+replica stages. These stages can be performed across multiple nodes in the
+cluster. If too much indexing work is introduced into the system, the cluster
+can become saturated.
+
+Indexing pressure is primarily generated by external operations such as indexing
+requests or CCR follow tasks. However, there is some internal indexing pressure
+during a shard recovery or primary failover.
+
+Elasticsearch internally monitors indexing load. When the load exceeds
+certain limits, new indexing work will be rejected.
+
+[float]
+=== Indexing Stages
+
+External indexing operations go through three stages. The node receiving the
+indexing request is the coordinating node. In the coordinating stage the node
+performs any configured ingest pipelines, separates the request into individual
+shard requests, and dispatches the requests to the primary shards.
+
+In the primary stage, the primary shard node ensures that it is still the primary
+(otherwise it reroutes the request to the new primary), waits on active shards,
+indexes the documents into its <<index-modules-translog,Translog>> and Lucene, and
+dispatches to the replicas.
+
+Finally, in the replica stage, every available replica node indexes the documents
+into its <<index-modules-translog,Translog>> and Lucene.
+
+
+[float]
+=== Memory Limits
+
+Elasticsearch exposes a node setting `indexing_pressure.memory.limit` which
+restricts the number of bytes for outstanding indexing requests. Be default,
+this setting is configured to be 10% of the heap.
+
+A node will start rejecting new indexing work at the coordinating or primary
+stage when the number of outstanding coordinating, primary and replica indexing
+requests is greater than the configured limit.
+
+A node will start rejecting new indexing work at the replica stage when the
+number of outstanding replica indexing requests is greater than 1.5x the
+configured limit. This design means that as indexing pressure builds on nodes,
+they will naturally stop accepting coordinating and primary work in favor of
+outstanding replica work.
+
+
+[float]
+=== Monitoring
+
+Indexing pressure metrics are exposed by the
+<<cluster-nodes-stats-api-response-body-indexing-pressure,Node Stats API>>.