Merge branch 'main' into esql-reconnect

docs/changelog/114813.yaml

@@ -0,0 +1,5 @@
+pr: 114813
+summary: Retry `S3BlobContainer#getRegister` on all exceptions
+area: Snapshot/Restore
+type: enhancement
+issues: []

docs/changelog/114837.yaml

@@ -0,0 +1,5 @@
+pr: 114837
+summary: Add warning headers for ingest pipelines containing special characters
+area: Ingest Node
+type: bug
+issues: [ 104411 ]

docs/changelog/114856.yaml

@@ -0,0 +1,5 @@
+pr: 114856
+summary: "OTel mappings: avoid metrics to be rejected when attributes are malformed"
+area: Data streams
+type: bug
+issues: []

docs/changelog/114888.yaml

@@ -0,0 +1,6 @@
+pr: 114888
+summary: Fix ST_CENTROID_AGG when no records are aggregated
+area: ES|QL
+type: bug
+issues:
+ - 106025

docs/plugins/mapper-annotated-text.asciidoc

@@ -167,8 +167,16 @@ duplicates removed. So:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "text": {
         "type": "annotated_text",
@@ -215,8 +223,16 @@ are preserved.
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "text": { "type": "annotated_text", "store": true }
     }

docs/reference/cat/indices.asciidoc

@@ -6,8 +6,8 @@
 
 [IMPORTANT]
 ====
-cat APIs are only intended for human consumption using the command line or {kib} 
-console. They are _not_ intended for use by applications. For application 
+cat APIs are only intended for human consumption using the command line or {kib}
+console. They are _not_ intended for use by applications. For application
 consumption, use the <<indices-get-index,get index API>>.
 ====
 
@@ -50,6 +50,10 @@ indexing and search. As a result, all document counts include hidden
 To get an accurate count of {es} documents, use the <<cat-count,cat count>> or
 <<search-count,count>> APIs.
 
+Note that information such as document count, deleted document count and store size are not shown for
+indices restored from <<snapshots-source-only-repository,source-only snapshots>> since these indices
+do not contain the relevant data structures to retrieve this information from.
+
 
 [[cat-indices-api-path-params]]
 ==== {api-path-parms-title}

docs/reference/how-to/size-your-shards.asciidoc

@@ -208,6 +208,7 @@ index can be <<indices-delete-index,removed>>. You may then consider setting
 <<indices-add-alias,Create Alias>> against the destination index for the source 
 index's name to point to it for continuity. 
 
+See this https://www.youtube.com/watch?v=sHyNYnwbYro[fixing shard sizes video] for an example troubleshooting walkthrough.
 
 [discrete]
 [[shard-count-recommendation]]
@@ -571,6 +572,8 @@ PUT _cluster/settings
 }
 ----
 
+For more information, see <<troubleshooting-shards-capacity-issues,Troubleshooting shards capacity>>.
+
 [discrete]
 [[troubleshooting-max-docs-limit]]
 ==== Number of documents in the shard cannot exceed [2147483519]

docs/reference/inference/service-elser.asciidoc

@@ -80,12 +80,13 @@ Must be a power of 2. Max allowed value is 32.
 [[inference-example-elser]]
 ==== ELSER service example
 
-The following example shows how to create an {infer} endpoint called
-`my-elser-model` to perform a `sparse_embedding` task type.
+The following example shows how to create an {infer} endpoint called `my-elser-model` to perform a `sparse_embedding` task type.
 Refer to the {ml-docs}/ml-nlp-elser.html[ELSER model documentation] for more info.
 
-The request below will automatically download the ELSER model if it isn't
-already downloaded and then deploy the model.
+NOTE: If you want to optimize your ELSER endpoint for ingest, set the number of threads to `1` (`"num_threads": 1`).
+If you want to optimize your ELSER endpoint for search, set the number of threads to greater than `1`.
+
+The request below will automatically download the ELSER model if it isn't already downloaded and then deploy the model.
 
 [source,console]
 ------------------------------------------------------------
@@ -100,7 +101,6 @@ PUT _inference/sparse_embedding/my-elser-model
 ------------------------------------------------------------
 // TEST[skip:TBD]
 
-
 Example response:
 
 [source,console-result]
@@ -130,12 +130,12 @@ If using the Python client, you can set the `timeout` parameter to a higher valu
 [[inference-example-elser-adaptive-allocation]]
 ==== Setting adaptive allocation for the ELSER service
 
-The following example shows how to create an {infer} endpoint called
-`my-elser-model` to perform a `sparse_embedding` task type and configure
-adaptive allocations.
+NOTE: For more information on how to optimize your ELSER endpoints, refer to {ml-docs}/ml-nlp-elser.html#elser-recommendations[the ELSER recommendations] section in the model documentation.
+To learn more about model autoscaling, refer to the {ml-docs}/ml-nlp-auto-scale.html[trained model autoscaling] page.
+
+The following example shows how to create an {infer} endpoint called `my-elser-model` to perform a `sparse_embedding` task type and configure adaptive allocations.
 
-The request below will automatically download the ELSER model if it isn't
-already downloaded and then deploy the model.
+The request below will automatically download the ELSER model if it isn't already downloaded and then deploy the model.
 
 [source,console]
 ------------------------------------------------------------

docs/reference/mapping/fields/synthetic-source.asciidoc

@@ -2,7 +2,7 @@
 ==== Synthetic `_source`
 
 IMPORTANT: Synthetic `_source` is Generally Available only for TSDB indices
-(indices that have `index.mode` set to `time_series`). For other indices
+(indices that have `index.mode` set to `time_series`). For other indices,
 synthetic `_source` is in technical preview. Features in technical preview may
 be changed or removed in a future release. Elastic will work to fix
 any issues, but features in technical preview are not subject to the support SLA
@@ -11,15 +11,19 @@ of official GA features.
 Though very handy to have around, the source field takes up a significant amount
 of space on disk. Instead of storing source documents on disk exactly as you
 send them, Elasticsearch can reconstruct source content on the fly upon retrieval.
-Enable this by setting `mode: synthetic` in `_source`:
+Enable this by using the value `synthetic` for the index setting `index.mapping.source.mode`:
 
 [source,console,id=enable-synthetic-source-example]
 ----
 PUT idx
 {
-  "mappings": {
-    "_source": {
-      "mode": "synthetic"
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
     }
   }
 }
@@ -38,7 +42,7 @@ properties when used with synthetic `_source`.
 <<synthetic-source-fields-native-list, Most field types>> construct synthetic `_source` using existing data, most
 commonly <<doc-values,`doc_values`>> and <<stored-fields, stored fields>>. For these field types, no additional space
 is needed to store the contents of `_source` field. Due to the storage layout of <<doc-values,`doc_values`>>, the
-generated `_source` field undergoes <<synthetic-source-modifications, modifications>> compared to original document.
+generated `_source` field undergoes <<synthetic-source-modifications, modifications>> compared to the original document.
 
 For all other field types, the original value of the field is stored as is, in the same way as the `_source` field in
 non-synthetic mode. In this case there are no modifications and field data in `_source` is the same as in the original
@@ -227,10 +231,16 @@ For instance:
 ----
 PUT idx_keep
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": {
-      "mode": "synthetic"
-    },
     "properties": {
       "path": {
         "type": "object",

docs/reference/mapping/types/aggregate-metric-double.asciidoc

@@ -267,8 +267,16 @@ For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "agg_metric": {
         "type": "aggregate_metric_double",

docs/reference/mapping/types/boolean.asciidoc

@@ -249,8 +249,16 @@ Synthetic source always sorts `boolean` fields. For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "bool": { "type": "boolean" }
     }

docs/reference/mapping/types/date.asciidoc

@@ -130,7 +130,7 @@ The following parameters are accepted by `date` fields:
 <<ignore-malformed,`ignore_malformed`>>::
 
     If `true`, malformed numbers are ignored. If `false` (default), malformed
-    numbers throw an exception and reject the whole document.  Note that this
+    numbers throw an exception and reject the whole document. Note that this
     cannot be set if the `script` parameter is used.
 
 <<mapping-index,`index`>>::
@@ -248,8 +248,16 @@ Synthetic source always sorts `date` fields. For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "date": { "type": "date" }
     }

docs/reference/mapping/types/date_nanos.asciidoc

@@ -160,8 +160,16 @@ Synthetic source always sorts `date_nanos` fields. For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "date": { "type": "date_nanos" }
     }

docs/reference/mapping/types/flattened.asciidoc

@@ -334,8 +334,16 @@ For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "flattened": { "type": "flattened" }
     }
@@ -367,8 +375,16 @@ For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "flattened": { "type": "flattened" }
     }
@@ -407,8 +423,16 @@ For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "flattened": { "type": "flattened" }
     }

docs/reference/mapping/types/geo-point.asciidoc

@@ -229,8 +229,16 @@ longitude) and reduces them to their stored precision. For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "point": { "type": "geo_point" }
     }

docs/reference/mapping/types/ip.asciidoc

@@ -170,8 +170,16 @@ Synthetic source always sorts `ip` fields and removes duplicates. For example:
 ----
 PUT idx
 {
+  "settings": {
+    "index": {
+      "mapping": {
+        "source": {
+          "mode": "synthetic"
+        }
+      }
+    }
+  },
   "mappings": {
-    "_source": { "mode": "synthetic" },
     "properties": {
       "ip": { "type": "ip" }
     }