Merge remote-tracking branch 'upstream/main' into 10512

Signed-off-by: Ashish Singh <[email protected]>

CHANGELOG.md

@@ -54,8 +54,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Change http code on create index API with bad input raising NotXContentException from 500 to 400 ([#4773](https://github.com/opensearch-project/OpenSearch/pull/4773))
 - Improve summary error message for invalid setting updates ([#4792](https://github.com/opensearch-project/OpenSearch/pull/4792))
 - Return 409 Conflict HTTP status instead of 503 on failure to concurrently execute snapshots ([#8986](https://github.com/opensearch-project/OpenSearch/pull/5855))
+- Add task completion count in search backpressure stats API ([#10028](https://github.com/opensearch-project/OpenSearch/pull/10028/))
 - Performance improvement for Datetime field caching ([#4558](https://github.com/opensearch-project/OpenSearch/issues/4558))
 
+
 ### Deprecated
 
 ### Removed
@@ -119,6 +121,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Bump `peter-evans/create-pull-request` from 3 to 5 ([#10301](https://github.com/opensearch-project/OpenSearch/pull/10301))
 - Bump `org.apache.avro:avro` from 1.11.2 to 1.11.3 ([#10210](https://github.com/opensearch-project/OpenSearch/pull/10210))
 - Bump `org.bouncycastle:bc-fips` from 1.0.2.3 to 1.0.2.4 ([#10297](https://github.com/opensearch-project/OpenSearch/pull/10297))
+- Bump `org.apache.zookeeper:zookeeper` from 3.9.0 to 3.9.1 ([#10506](https://github.com/opensearch-project/OpenSearch/pull/10506))
+- Bump `de.thetaphi:forbiddenapis` from 3.5.1 to 3.6 ([#10508](https://github.com/opensearch-project/OpenSearch/pull/10508))
 
 ### Changed
 - Add instrumentation in rest and network layer. ([#9415](https://github.com/opensearch-project/OpenSearch/pull/9415))
@@ -149,8 +153,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Fix circular dependency in Settings initialization ([10194](https://github.com/opensearch-project/OpenSearch/pull/10194))
 - Fix registration and initialization of multiple extensions ([10256](https://github.com/opensearch-project/OpenSearch/pull/10256))
 - Fix Segment Replication ShardLockObtainFailedException bug during index corruption ([10370](https://github.com/opensearch-project/OpenSearch/pull/10370))
+- Fix some test methods in SimulatePipelineRequestParsingTests never run and fix test failure ([#10496](https://github.com/opensearch-project/OpenSearch/pull/10496))
 
 ### Security
 
 [Unreleased 3.0]: https://github.com/opensearch-project/OpenSearch/compare/2.x...HEAD
-[Unreleased 2.x]: https://github.com/opensearch-project/OpenSearch/compare/2.11...2.x
+[Unreleased 2.x]: https://github.com/opensearch-project/OpenSearch/compare/2.11...2.x

buildSrc/build.gradle

@@ -114,7 +114,7 @@ dependencies {
   api 'com.github.johnrengelman:shadow:8.1.1'
   api 'org.jdom:jdom2:2.0.6.1'
   api "org.jetbrains.kotlin:kotlin-stdlib-jdk8:${props.getProperty('kotlin')}"
-  api 'de.thetaphi:forbiddenapis:3.5.1'
+  api 'de.thetaphi:forbiddenapis:3.6'
   api 'com.avast.gradle:gradle-docker-compose-plugin:0.16.12'
   api "org.yaml:snakeyaml:${props.getProperty('snakeyaml')}"
   api 'org.apache.maven:maven-model:3.9.4'

buildSrc/reaper/src/main/java/org/opensearch/gradle/reaper/Reaper.java

@@ -45,17 +45,16 @@
 
 /**
  * A standalone process that will reap external services after a build dies.
- *
  * <h2>Input</h2>
  * Since how to reap a given service is platform and service dependent, this tool
  * operates on system commands to execute. It takes a single argument, a directory
  * that will contain files with reaping commands. Each line in each file will be
  * executed with {@link Runtime#exec(String)}.
- *
+ * <p>
  * The main method will wait indefinitely on the parent process (Gradle) by
  * reading from stdin. When Gradle shuts down, whether normally or abruptly, the
  * pipe will be broken and read will return.
- *
+ * <p>
  * The reaper will then iterate over the files in the configured directory,
  * and execute the given commands. If any commands fail, a failure message is
  * written to stderr. Otherwise, the input file will be deleted. If no inputs

buildSrc/src/main/java/org/opensearch/gradle/BwcVersions.java

@@ -52,15 +52,15 @@
 
 /**
  * A container for opensearch supported version information used in BWC testing.
- *
+ * <p>
  * Parse the Java source file containing the versions declarations and use the known rules to figure out which are all
  * the version the current one is wire and index compatible with.
  * On top of this, figure out which of these are unreleased and provide the branch they can be built from.
- *
+ * <p>
  * Note that in this context, currentVersion is the unreleased version this build operates on.
  * At any point in time there will surely be four such unreleased versions being worked on,
  * thus currentVersion will be one of these.
- *
+ * <p>
  * Considering:
  * <dl>
  *     <dt>M, M &gt; 0</dt>
@@ -84,7 +84,7 @@
  * Each build is only concerned with versions before it, as those are the ones that need to be tested
  * for backwards compatibility. We never look forward, and don't add forward facing version number to branches of previous
  * version.
- *
+ * <p>
  * Each branch has a current version, and expected compatible versions are parsed from the server code's Version` class.
  * We can reliably figure out which the unreleased versions are due to the convention of always adding the next unreleased
  * version number to server in all branches when a version is released.

buildSrc/src/main/java/org/opensearch/gradle/LoggingOutputStream.java

@@ -38,7 +38,7 @@
 
 /**
  * Writes data passed to this stream as log messages.
- *
+ * <p>
  * The stream will be flushed whenever a newline is detected.
  * Allows setting an optional prefix before each line of output.
  */

buildSrc/src/main/java/org/opensearch/gradle/precommit/TestingConventionRule.java

@@ -40,7 +40,7 @@
 
 /**
  * Represent rules for tests enforced by the @{link {@link TestingConventionsTasks}}
- *
+ * <p>
  * Rules are identified by name, tests must have this name as a suffix and implement one of the base classes
  * and be part of all the specified tasks.
  */

buildSrc/src/main/java/org/opensearch/gradle/tar/SymbolicLinkPreservingTar.java

@@ -61,7 +61,7 @@
 
 /**
  * A custom archive task that assembles a tar archive that preserves symbolic links.
- *
+ * <p>
  * This task is necessary because the built-in task {@link org.gradle.api.tasks.bundling.Tar} does not preserve symbolic links.
  */
 public class SymbolicLinkPreservingTar extends Tar {

buildSrc/src/main/java/org/opensearch/gradle/test/rest/RestResourcesPlugin.java

@@ -65,7 +65,7 @@
  * <strong>Rest YAML tests :</strong> <br>
  * When the {@link RestResourcesPlugin} has been applied the {@link CopyRestTestsTask} will copy the Rest YAML tests if explicitly
  * configured with `includeCore` through the `restResources.restTests` extension.
- *
+ * <p>
  * Additionally you can specify which sourceSetName resources should be copied to. The default is the yamlRestTest source set.
  * @see CopyRestApiTask
  * @see CopyRestTestsTask

buildSrc/src/main/java/org/opensearch/gradle/vagrant/VagrantMachine.java

@@ -53,7 +53,7 @@
 
 /**
  * An helper to manage a vagrant box.
- *
+ * <p>
  * This is created alongside a {@link VagrantExtension} for a project to manage starting and
  * stopping a single vagrant box.
  */
@@ -185,7 +185,7 @@ public void setArgs(String... args) {
 
         /**
          * A function to translate output from the vagrant command execution to the progress line.
-         *
+         * <p>
          * The function takes the current line of output from vagrant, and returns a new
          * progress line, or {@code null} if there is no update.
          */

buildSrc/src/main/java/org/opensearch/gradle/vagrant/VagrantShellTask.java

@@ -47,7 +47,7 @@
 
 /**
  * A shell script to run within a vagrant VM.
- *
+ * <p>
  * The script is run as root within the VM.
  */
 public abstract class VagrantShellTask extends DefaultTask {

buildSrc/src/test/java/org/opensearch/gradle/pluginzip/PublishTests.java

@@ -58,7 +58,7 @@ public void tearDown() {
      * This test is used to verify that adding the 'opensearch.pluginzip' to the project
      * adds some other transitive plugins and tasks under the hood. This is basically
      * a behavioral test of the {@link Publish#apply(Project)} method.
-     *
+     * <p>
      * This is equivalent of having a build.gradle script with just the following section:
      * <pre>
      *     plugins {
@@ -202,7 +202,7 @@ public void useDefaultValues() throws IOException, URISyntaxException, XmlPullPa
         GradleRunner runner = prepareGradleRunnerFromTemplate("useDefaultValues.gradle", "build", ZIP_PUBLISH_TASK);
         BuildResult result = runner.build();
 
-        /** Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
+        /* Check if build and ZIP_PUBLISH_TASK tasks have run well */
         assertEquals(SUCCESS, result.task(":" + "build").getOutcome());
         assertEquals(SUCCESS, result.task(":" + ZIP_PUBLISH_TASK).getOutcome());
 
@@ -277,7 +277,7 @@ public void allProjectsGroup() throws IOException, URISyntaxException, XmlPullPa
         GradleRunner runner = prepareGradleRunnerFromTemplate("allProjectsGroup.gradle", "build", ZIP_PUBLISH_TASK);
         BuildResult result = runner.build();
 
-        /** Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
+        /* Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
         assertEquals(SUCCESS, result.task(":" + "build").getOutcome());
         assertEquals(SUCCESS, result.task(":" + ZIP_PUBLISH_TASK).getOutcome());
 
@@ -312,7 +312,7 @@ public void groupPriorityLevel() throws IOException, URISyntaxException, XmlPull
         GradleRunner runner = prepareGradleRunnerFromTemplate("groupPriorityLevel.gradle", "build", ZIP_PUBLISH_TASK);
         BuildResult result = runner.build();
 
-        /** Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
+        /* Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
         assertEquals(SUCCESS, result.task(":" + "build").getOutcome());
         assertEquals(SUCCESS, result.task(":" + ZIP_PUBLISH_TASK).getOutcome());
 
@@ -348,7 +348,7 @@ public void missingPOMEntity() throws IOException, URISyntaxException, XmlPullPa
         GradleRunner runner = prepareGradleRunnerFromTemplate("missingPOMEntity.gradle", "build", ZIP_PUBLISH_TASK);
         BuildResult result = runner.build();
 
-        /** Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
+        /* Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
         assertEquals(SUCCESS, result.task(":" + "build").getOutcome());
         assertEquals(SUCCESS, result.task(":" + ZIP_PUBLISH_TASK).getOutcome());
 
@@ -395,7 +395,7 @@ public void customizedGroupValue() throws IOException, URISyntaxException, XmlPu
         GradleRunner runner = prepareGradleRunnerFromTemplate("customizedGroupValue.gradle", "build", ZIP_PUBLISH_TASK);
         BuildResult result = runner.build();
 
-        /** Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
+        /* Check if build and {@value ZIP_PUBLISH_TASK} tasks have run well */
         assertEquals(SUCCESS, result.task(":" + "build").getOutcome());
         assertEquals(SUCCESS, result.task(":" + ZIP_PUBLISH_TASK).getOutcome());
 

buildSrc/src/testFixtures/java/org/opensearch/gradle/test/GradleThreadsFilter.java

@@ -36,7 +36,7 @@
 
 /**
  * Filter out threads controlled by gradle that may be created during unit tests.
- *
+ * <p>
  * Currently this includes pooled threads for Exec as well as file system event watcher threads.
  */
 public class GradleThreadsFilter implements ThreadFilter {

buildSrc/src/testFixtures/java/org/opensearch/gradle/test/JUnit3MethodProvider.java

@@ -43,7 +43,7 @@
 
 /**
  * Backwards compatible test* method provider (public, non-static).
- *
+ * <p>
  * copy of org.apache.lucene.util.LuceneJUnit3MethodProvider to avoid a dependency between build and test fw.
  */
 public final class JUnit3MethodProvider implements TestMethodProvider {

client/benchmark/src/main/java/org/opensearch/client/benchmark/metrics/SampleRecorder.java

@@ -37,7 +37,7 @@
 
 /**
  * Stores measurement samples.
- *
+ * <p>
  * This class is NOT threadsafe.
  */
 public final class SampleRecorder {

client/rest-high-level/src/main/java/org/opensearch/client/ClusterClient.java

@@ -170,8 +170,8 @@ public ClusterHealthResponse health(ClusterHealthRequest healthRequest, RequestO
 
     /**
      * Asynchronously get cluster health using the Cluster Health API.
-     *
      * If timeout occurred, {@link ClusterHealthResponse} will have isTimedOut() == true and status() == RestStatus.REQUEST_TIMEOUT
+     *
      * @param healthRequest the request
      * @param options the request options (e.g. headers), use {@link RequestOptions#DEFAULT} if nothing needs to be customized
      * @param listener the listener to be notified upon request completion

client/rest-high-level/src/main/java/org/opensearch/client/RequestConverters.java

@@ -451,9 +451,9 @@ static void addSearchRequestParams(Params params, SearchRequest searchRequest) {
             params.withIndicesOptions(searchRequest.indicesOptions());
         }
         params.withSearchType(searchRequest.searchType().name().toLowerCase(Locale.ROOT));
-        /**
-         * Merging search responses as part of CCS flow to reduce roundtrips is not supported for point in time -
-         * refer to org.opensearch.action.search.SearchResponseMerger
+        /*
+          Merging search responses as part of CCS flow to reduce roundtrips is not supported for point in time -
+          refer to org.opensearch.action.search.SearchResponseMerger
          */
         if (searchRequest.pointInTimeBuilder() != null) {
             params.putParam("ccs_minimize_roundtrips", "false");

client/rest-high-level/src/main/java/org/opensearch/client/TimedRequest.java

@@ -37,7 +37,7 @@
 
 /**
  * A base request for any requests that supply timeouts.
- *
+ * <p>
  * Please note, any requests that use a ackTimeout should set timeout as they
  * represent the same backing field on the server.
  */

client/rest-high-level/src/main/java/org/opensearch/client/indices/CreateIndexRequest.java

@@ -156,7 +156,7 @@ public MediaType mappingsMediaType() {
 
     /**
      * Adds mapping that will be added when the index gets created.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      *
      * @param source The mapping source
@@ -168,7 +168,7 @@ public CreateIndexRequest mapping(String source, MediaType mediaType) {
 
     /**
      * Adds mapping that will be added when the index gets created.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      *
      * @param source The mapping source
@@ -179,7 +179,7 @@ public CreateIndexRequest mapping(XContentBuilder source) {
 
     /**
      * Adds mapping that will be added when the index gets created.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      *
      * @param source The mapping source
@@ -196,7 +196,7 @@ public CreateIndexRequest mapping(Map<String, ?> source) {
 
     /**
      * Adds mapping that will be added when the index gets created.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      *
      * @param source The mapping source
@@ -282,7 +282,7 @@ public CreateIndexRequest aliases(Collection<Alias> aliases) {
 
     /**
      * Sets the settings and mappings as a single source.
-     *
+     * <p>
      * Note that the mapping definition should *not* be nested under a type name.
      */
     public CreateIndexRequest source(String source, MediaType mediaType) {
@@ -291,7 +291,7 @@ public CreateIndexRequest source(String source, MediaType mediaType) {
 
     /**
      * Sets the settings and mappings as a single source.
-     *
+     * <p>
      * Note that the mapping definition should *not* be nested under a type name.
      */
     public CreateIndexRequest source(XContentBuilder source) {
@@ -300,7 +300,7 @@ public CreateIndexRequest source(XContentBuilder source) {
 
     /**
      * Sets the settings and mappings as a single source.
-     *
+     * <p>
      * Note that the mapping definition should *not* be nested under a type name.
      */
     public CreateIndexRequest source(BytesReference source, MediaType mediaType) {
@@ -311,7 +311,7 @@ public CreateIndexRequest source(BytesReference source, MediaType mediaType) {
 
     /**
      * Sets the settings and mappings as a single source.
-     *
+     * <p>
      * Note that the mapping definition should *not* be nested under a type name.
      */
     @SuppressWarnings("unchecked")

client/rest-high-level/src/main/java/org/opensearch/client/indices/PutMappingRequest.java

@@ -105,7 +105,7 @@ public MediaType mediaType() {
 
     /**
      * The mapping source definition.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      */
     public PutMappingRequest source(Map<String, ?> mappingSource) {
@@ -120,7 +120,7 @@ public PutMappingRequest source(Map<String, ?> mappingSource) {
 
     /**
      * The mapping source definition.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      */
     public PutMappingRequest source(String mappingSource, MediaType mediaType) {
@@ -131,7 +131,7 @@ public PutMappingRequest source(String mappingSource, MediaType mediaType) {
 
     /**
      * The mapping source definition.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      */
     public PutMappingRequest source(XContentBuilder builder) {
@@ -142,7 +142,7 @@ public PutMappingRequest source(XContentBuilder builder) {
 
     /**
      * The mapping source definition.
-     *
+     * <p>
      * Note that the definition should *not* be nested under a type name.
      */
     public PutMappingRequest source(BytesReference source, MediaType mediaType) {

client/rest-high-level/src/main/java/org/opensearch/client/tasks/TaskGroup.java

@@ -38,7 +38,6 @@
 
 /**
  * Client side counterpart of server side version.
- *
  * {@link org.opensearch.action.admin.cluster.node.tasks.list.TaskGroup}
  */
 public class TaskGroup {

client/rest-high-level/src/test/java/org/opensearch/client/AbstractRequestTestCase.java

@@ -44,7 +44,7 @@
 
 /**
  * Base class for HLRC request parsing tests.
- *
+ * <p>
  * This case class facilitates generating client side request test instances and
  * verifies that they are correctly parsed into server side request instances.
  *

client/rest-high-level/src/test/java/org/opensearch/client/AbstractResponseTestCase.java

@@ -44,7 +44,7 @@
 
 /**
  * Base class for HLRC response parsing tests.
- *
+ * <p>
  * This case class facilitates generating server side response test instances and
  * verifies that they are correctly parsed into HLRC response instances.
  *