Merge branch 'main' into esql-migrate-binary-comparisons-optimizations-2

 Conflicts:
	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/OptimizerRules.java
	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/OptimizerRulesTests.java

.github/workflows/gradle-wrapper-validation.yml

@@ -10,4 +10,4 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2
-      - uses: gradle/wrapper-validation-action@v1
+      - uses: gradle/wrapper-validation-action@699bb18358f12c5b78b37bb0111d3a0e2276e0e2 # Release v2.1.1

docs/changelog/105860.yaml

@@ -0,0 +1,5 @@
+pr: 105860
+summary: "ESQL: Re-enable logical dependency check"
+area: ES|QL
+type: bug
+issues: []

docs/changelog/106306.yaml

@@ -0,0 +1,6 @@
+pr: 99961
+summary: "added fix for inconsistent text trimming in Unified Highlighter"
+area: Highlighting
+type: bug
+issues:
+ - 101803

docs/changelog/106511.yaml

@@ -0,0 +1,5 @@
+pr: 106511
+summary: Wait indefintely for http connections on shutdown by default
+area: Infra/Node Lifecycle
+type: bug
+issues: []

docs/changelog/106654.yaml

@@ -0,0 +1,6 @@
+pr: 106654
+summary: "ES|QL: Fix usage of IN operator with TEXT fields"
+area: ES|QL
+type: bug
+issues:
+ - 105379

docs/changelog/106655.yaml

@@ -0,0 +1,5 @@
+pr: 106655
+summary: Fix Array out of bounds exception in the XLM Roberta tokenizer
+area: Machine Learning
+type: bug
+issues: []

docs/internal/DistributedArchitectureGuide.md

@@ -10,20 +10,70 @@
 
 ### ActionListener
 
-`ActionListener`s are a means off injecting logic into lower layers of the code. They encapsulate a block of code that takes a response
-value -- the `onResponse()` method --, and then that block of code (the `ActionListener`) is passed into a function that will eventually
-execute the code (call `onResponse()`) when a response value is available. `ActionListener`s are used to pass code down to act on a result,
-rather than lower layers returning a result back up to be acted upon by the caller. One of three things can happen to a listener: it can be
-executed in the same thread — e.g. `ActionListener.run()` --; it can be passed off to another thread to be executed; or it can be added to
-a list someplace, to eventually be executed by some service. `ActionListener`s also define `onFailure()` logic, in case an error is
-encountered before a result can be formed.
+Callbacks are used extensively throughout Elasticsearch because they enable us to write asynchronous and nonblocking code, i.e. code which
+doesn't necessarily compute a result straight away but also doesn't block the calling thread waiting for the result to become available.
+They support several useful control flows:
+
+- They can be completed immediately on the calling thread.
+- They can be completed concurrently on a different thread.
+- They can be stored in a data structure and completed later on when the system reaches a particular state.
+- Most commonly, they can be passed on to other methods that themselves require a callback.
+- They can be wrapped in another callback which modifies the behaviour of the original callback, perhaps adding some extra code to run
+  before or after completion, before passing them on.
+
+`ActionListener` is a general-purpose callback interface that is used extensively across the Elasticsearch codebase. `ActionListener` is
+used pretty much everywhere that needs to perform some asynchronous and nonblocking computation. The uniformity makes it easier to compose
+parts of the system together without needing to build adapters to convert back and forth between different kinds of callback. It also makes
+it easier to develop the skills needed to read and understand all the asynchronous code, although this definitely takes practice and is
+certainly not easy in an absolute sense. Finally, it has allowed us to build a rich library for working with `ActionListener` instances
+themselves, creating new instances out of existing ones and completing them in interesting ways. See for instance:
+
+- all the static methods on [ActionListener](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/ActionListener.java) itself
+- [`ThreadedActionListener`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/support/ThreadedActionListener.java) for forking work elsewhere
+- [`RefCountingListener`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/support/RefCountingListener.java) for running work in parallel
+- [`SubscribableListener`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/support/SubscribableListener.java) for constructing flexible workflows
+
+Callback-based asynchronous code can easily call regular synchronous code, but synchronous code cannot run callback-based asynchronous code
+without blocking the calling thread until the callback is called back. This blocking is at best undesirable (threads are too expensive to
+waste with unnecessary blocking) and at worst outright broken (the blocking can lead to deadlock). Unfortunately this means that most of our
+code ends up having to be written with callbacks, simply because it's ultimately calling into some other code that takes a callback. The
+entry points for all Elasticsearch APIs are callback-based (e.g. REST APIs all start at
+[`org.elasticsearch.rest.BaseRestHandler#prepareRequest`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/rest/BaseRestHandler.java#L158-L171),
+and transport APIs all start at
+[`org.elasticsearch.action.support.TransportAction#doExecute`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/support/TransportAction.java#L65))
+and the whole system fundamentally works in terms of an event loop (a `io.netty.channel.EventLoop`) which processes network events via
+callbacks.
+
+`ActionListener` is not an _ad-hoc_ invention. Formally speaking, it is our implementation of the general concept of a continuation in the
+sense of [_continuation-passing style_](https://en.wikipedia.org/wiki/Continuation-passing_style) (CPS): an extra argument to a function
+which defines how to continue the computation when the result is available. This is in contrast to _direct style_ which is the more usual
+style of calling methods that return values directly back to the caller so they can continue executing as normal. There's essentially two
+ways that computation can continue in Java (it can return a value or it can throw an exception) which is why `ActionListener` has both an
+`onResponse()` and an `onFailure()` method.
+
+CPS is strictly more expressive than direct style: direct code can be mechanically translated into continuation-passing style, but CPS also
+enables all sorts of other useful control structures such as forking work onto separate threads, possibly to be executed in parallel,
+perhaps even across multiple nodes, or possibly collecting a list of continuations all waiting for the same condition to be satisfied before
+proceeding (e.g.
+[`SubscribableListener`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/support/SubscribableListener.java)
+amongst many others). Some languages have first-class support for continuations (e.g. the `async` and `await` primitives in C#) allowing the
+programmer to write code in direct style away from those exotic control structures, but Java does not. That's why we have to manipulate all
+the callbacks ourselves.
+
+Strictly speaking, CPS requires that a computation _only_ continues by calling the continuation. In Elasticsearch, this means that
+asynchronous methods must have `void` return type and may not throw any exceptions. This is mostly the case in our code as written today,
+and is a good guiding principle, but we don't enforce void exceptionless methods and there are some deviations from this rule. In
+particular, it's not uncommon to permit some methods to throw an exception, using things like
+[`ActionListener#run`](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/ActionListener.java#L381-L390)
+(or an equivalent `try ... catch ...` block) further up the stack to handle it. Some methods also take (and may complete) an
+`ActionListener` parameter, but still return a value separately for other local synchronous work.
 
 This pattern is often used in the transport action layer with the use of the
-[ChannelActionListener]([url](https://github.com/elastic/elasticsearch/blob/8.12/server/src/main/java/org/elasticsearch/action/support/ChannelActionListener.java))
+[ChannelActionListener](https://github.com/elastic/elasticsearch/blob/v8.12.2/server/src/main/java/org/elasticsearch/action/support/ChannelActionListener.java)
 class, which wraps a `TransportChannel` produced by the transport layer. `TransportChannel` implementations can hold a reference to a Netty
-channel with which to pass the response back to the network caller. Netty has a many-to-one association of network callers to channels, so
-a call taking a long time generally won't hog resources: it's cheap. A transport action can take hours to respond and that's alright,
-barring caller timeouts.
+channel with which to pass the response back to the network caller. Netty has a many-to-one association of network callers to channels, so a
+call taking a long time generally won't hog resources: it's cheap. A transport action can take hours to respond and that's alright, barring
+caller timeouts.
 
 (TODO: add useful starter references and explanations for a range of Listener classes. Reference the Netty section.)
 

docs/reference/connector/apis/update-connector-filtering-api.asciidoc

@@ -55,32 +55,32 @@ Contains the set of rules that are actively used for sync jobs. The `active` obj
     The value to be used in conjunction with the rule for matching the contents of the document's field.
     ** `order` (Required, number) +
     The order in which the rules are applied. The first rule to match has its policy applied.
-    ** `created_at` (Optional, datetime) +
+    ** `created_at` (Required, datetime) +
     The timestamp when the rule was added.
-    ** `updated_at` (Optional, datetime) +
+    ** `updated_at` (Required, datetime) +
     The timestamp when the rule was last edited.
 
-  * `advanced_snippet` (Optional, object) +
+  * `advanced_snippet` (Required, object) +
   Used for {enterprise-search-ref}/sync-rules.html#sync-rules-advanced[advanced filtering] at query time, with the following sub-attributes:
     ** `value` (Required, object) +
     A JSON object passed directly to the connector for advanced filtering.
-    ** `created_at` (Optional, datetime) +
+    ** `created_at` (Required, datetime) +
     The timestamp when this JSON object was created.
-    ** `updated_at` (Optional, datetime) +
+    ** `updated_at` (Required, datetime) +
     The timestamp when this JSON object was last edited.
 
-  * `validation` (Optional, object) +
+  * `validation` (Required, object) +
   Provides validation status for the rules, including:
     ** `state` (Required, string) +
     Indicates the validation state: "edited", "valid", or "invalid".
-    ** `errors` (Optional, object) +
+    ** `errors` (Required, object) +
     Contains details about any validation errors, with sub-attributes:
       *** `ids` (Required, string) +
       The ID(s) of any rules deemed invalid.
       *** `messages` (Required, string) +
       Messages explaining what is invalid about the rules.
 
-- `draft` (Optional, object) +
+- `draft` (Required, object) +
 An object identical in structure to the `active` object, but used for drafting and editing filtering rules before they become active.
 
 

docs/reference/search/search-your-data/ccs-version-compat-matrix.asciidoc

@@ -1,22 +1,24 @@
-[cols="^,^,^,^,^,^,^,^,^,^,^,^,^,^,^,^,^"]
+[cols="^,^,^,^,^,^,^,^,^,^,^,^,^,^,^,^,^,^"]
 |====
-| 16+^h| Remote cluster version
+| 17+^h| Remote cluster version
 h| Local cluster version
-            |  6.8        | 7.1–7.16   | 7.17       | 8.0        | 8.1        | 8.2        | 8.3       | 8.4       | 8.5       |8.6         |8.7         |8.8         |8.9         |8.10        |8.11       |8.12 
-| 6.8       |  {yes-icon} | {yes-icon} | {yes-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} | {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} 
-| 7.1–7.16  |  {yes-icon} | {yes-icon} | {yes-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} | {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} 
-| 7.17      |  {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} 
-| 8.0       |  {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.1       |  {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.2       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.3       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon}|{yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.4       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} |{yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.5       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  |{yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.6       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.7       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.8       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.9       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.10      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.11      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon}| {yes-icon}
-| 8.12      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon}| {yes-icon}
-|====
+            |  6.8        | 7.1–7.16   | 7.17       | 8.0        | 8.1        | 8.2        | 8.3       | 8.4       | 8.5       |8.6         |8.7         |8.8         |8.9         |8.10        |8.11       |8.12       |8.13
+| 6.8       |  {yes-icon} | {yes-icon} | {yes-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} | {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} | {no-icon}
+| 7.1–7.16  |  {yes-icon} | {yes-icon} | {yes-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} | {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {no-icon} | {no-icon}
+| 7.17      |  {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon}
+| 8.0       |  {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon}
+| 8.1       |  {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon}
+| 8.2       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon}
+| 8.3       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon}|{yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon}
+| 8.4       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} |{yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon}
+| 8.5       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.6       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.7       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.8       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.9       |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.10      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.11      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon} | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.12      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {yes-icon}| {yes-icon} | {yes-icon} | {yes-icon}
+| 8.13      |  {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}| {no-icon} |  {no-icon} | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon}  | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon}
+|====
+