Merge branch 'main' into support_flat_object_docvalue

Signed-off-by: kkewwei <[email protected]>

.ci/bwcVersions

@@ -39,4 +39,5 @@ BWC_VERSION:
   - "2.16.1"
   - "2.17.0"
   - "2.17.1"
+  - "2.17.2"
   - "2.18.0"

CHANGELOG.md

@@ -11,15 +11,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Fallback to Remote cluster-state on Term-Version check mismatch - ([#15424](https://github.com/opensearch-project/OpenSearch/pull/15424))
 - Implement WithFieldName interface in ValuesSourceAggregationBuilder & FieldSortBuilder ([#15916](https://github.com/opensearch-project/OpenSearch/pull/15916))
 - Add successfulSearchShardIndices in searchRequestContext ([#15967](https://github.com/opensearch-project/OpenSearch/pull/15967))
-- Remove identity-related feature flagged code from the RestController ([#15430](https://github.com/opensearch-project/OpenSearch/pull/15430))
 - Add support for msearch API to pass search pipeline name - ([#15923](https://github.com/opensearch-project/OpenSearch/pull/15923))
 - Add _list/indices API as paginated alternate to _cat/indices ([#14718](https://github.com/opensearch-project/OpenSearch/pull/14718))
+- Add success and failure metrics for async shard fetch ([#15976](https://github.com/opensearch-project/OpenSearch/pull/15976))
+- Add new metric REMOTE_STORE to NodeStats API response ([#15611](https://github.com/opensearch-project/OpenSearch/pull/15611))
+- [S3 Repository] Change default retry mechanism of s3 clients to Standard Mode ([#15978](https://github.com/opensearch-project/OpenSearch/pull/15978))
+- Add changes to block calls in cat shards, indices and segments based on dynamic limit settings ([#15986](https://github.com/opensearch-project/OpenSearch/pull/15986))
+- New `phone` & `phone-search` analyzer + tokenizer ([#15915](https://github.com/opensearch-project/OpenSearch/pull/15915))
 - Flat object field use IndexOrDocValuesQuery to optimize query ([#14383](https://github.com/opensearch-project/OpenSearch/issues/14383))
 
 ### Dependencies
 - Bump `com.azure:azure-identity` from 1.13.0 to 1.13.2 ([#15578](https://github.com/opensearch-project/OpenSearch/pull/15578))
 - Bump `protobuf` from 3.22.3 to 3.25.4 ([#15684](https://github.com/opensearch-project/OpenSearch/pull/15684))
-- Bump `org.apache.logging.log4j:log4j-core` from 2.23.1 to 2.24.0 ([#15858](https://github.com/opensearch-project/OpenSearch/pull/15858))
+- Bump `org.apache.logging.log4j:log4j-core` from 2.23.1 to 2.24.1 ([#15858](https://github.com/opensearch-project/OpenSearch/pull/15858), [#16134](https://github.com/opensearch-project/OpenSearch/pull/16134))
 - Bump `peter-evans/create-pull-request` from 6 to 7 ([#15863](https://github.com/opensearch-project/OpenSearch/pull/15863))
 - Bump `com.nimbusds:oauth2-oidc-sdk` from 11.9.1 to 11.19.1 ([#15862](https://github.com/opensearch-project/OpenSearch/pull/15862))
 - Bump `com.microsoft.azure:msal4j` from 1.17.0 to 1.17.1 ([#15945](https://github.com/opensearch-project/OpenSearch/pull/15945))
@@ -30,9 +34,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Bump `actions/github-script` from 5 to 7 ([#16039](https://github.com/opensearch-project/OpenSearch/pull/16039))
 - Bump `dnsjava:dnsjava` from 3.6.1 to 3.6.2 ([#16041](https://github.com/opensearch-project/OpenSearch/pull/16041))
 - Bump `com.maxmind.geoip2:geoip2` from 4.2.0 to 4.2.1 ([#16042](https://github.com/opensearch-project/OpenSearch/pull/16042))
+- Bump `com.maxmind.db:maxmind-db` from 3.1.0 to 3.1.1 ([#16137](https://github.com/opensearch-project/OpenSearch/pull/16137))
+- Bump `com.azure:azure-core-http-netty` from 1.15.3 to 1.15.4 ([#16133](https://github.com/opensearch-project/OpenSearch/pull/16133))
+- Bump `org.jline:jline` from 3.26.3 to 3.27.0 ([#16135](https://github.com/opensearch-project/OpenSearch/pull/16135))
+- Bump `netty` from 4.1.112.Final to 4.1.114.Final ([#16182](https://github.com/opensearch-project/OpenSearch/pull/16182))
 
 ### Changed
 - Add support for docker compose v2 in TestFixturesPlugin ([#16049](https://github.com/opensearch-project/OpenSearch/pull/16049))
+- Remove identity-related feature flagged code from the RestController ([#15430](https://github.com/opensearch-project/OpenSearch/pull/15430))
+- Remove Identity FeatureFlag ([#16024](https://github.com/opensearch-project/OpenSearch/pull/16024))
+- Ensure RestHandler.Wrapper delegates all implementations to the wrapped handler ([#16154](https://github.com/opensearch-project/OpenSearch/pull/16154))
 
 
 ### Deprecated
@@ -47,6 +58,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Avoid infinite loop when `flat_object` field contains invalid token ([#15985](https://github.com/opensearch-project/OpenSearch/pull/15985))
 - Fix infinite loop in nested agg ([#15931](https://github.com/opensearch-project/OpenSearch/pull/15931))
 - Fix race condition in node-join and node-left ([#15521](https://github.com/opensearch-project/OpenSearch/pull/15521))
+- Streaming bulk request hangs ([#16158](https://github.com/opensearch-project/OpenSearch/pull/16158))
+- Fix warnings from SLF4J on startup when repository-s3 is installed ([#16194](https://github.com/opensearch-project/OpenSearch/pull/16194))
 
 ### Security
 

README.md

@@ -8,7 +8,7 @@
 [![Open Issues](https://img.shields.io/github/issues/opensearch-project/OpenSearch)](https://github.com/opensearch-project/OpenSearch/issues)
 [![Open Pull Requests](https://img.shields.io/github/issues-pr/opensearch-project/OpenSearch)](https://github.com/opensearch-project/OpenSearch/pulls)
 [![2.18.0 Open Issues](https://img.shields.io/github/issues/opensearch-project/OpenSearch/v2.18.0)](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aissue+is%3Aopen+label%3A"v2.18.0")
-[![2.17.1 Open Issues](https://img.shields.io/github/issues/opensearch-project/OpenSearch/v2.17.1)](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aissue+is%3Aopen+label%3A"v2.17.1")
+[![2.17.2 Open Issues](https://img.shields.io/github/issues/opensearch-project/OpenSearch/v2.17.2)](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aissue+is%3Aopen+label%3A"v2.17.2")
 [![3.0.0 Open Issues](https://img.shields.io/github/issues/opensearch-project/OpenSearch/v3.0.0)](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aissue+is%3Aopen+label%3A"v3.0.0")
 [![GHA gradle check](https://github.com/opensearch-project/OpenSearch/actions/workflows/gradle-check.yml/badge.svg)](https://github.com/opensearch-project/OpenSearch/actions/workflows/gradle-check.yml)
 [![GHA validate pull request](https://github.com/opensearch-project/OpenSearch/actions/workflows/wrapper.yml/badge.svg)](https://github.com/opensearch-project/OpenSearch/actions/workflows/wrapper.yml)

buildSrc/src/testKit/thirdPartyAudit/sample_jars/build.gradle

@@ -17,7 +17,7 @@ repositories {
 }
 
 dependencies {
-  implementation "org.apache.logging.log4j:log4j-core:2.24.0"
+  implementation "org.apache.logging.log4j:log4j-core:2.24.1"
 }
 
 ["0.0.1", "0.0.2"].forEach { v ->

buildSrc/version.properties

@@ -29,7 +29,7 @@ hdrhistogram      = 2.2.2
 # when updating the JNA version, also update the version in buildSrc/build.gradle
 jna               = 5.13.0
 
-netty             = 4.1.112.Final
+netty             = 4.1.114.Final
 joda              = 2.12.7
 
 # project reactor

client/rest-high-level/src/test/java/org/opensearch/client/OpenSearchRestHighLevelClientTestCase.java

@@ -90,6 +90,7 @@
 public abstract class OpenSearchRestHighLevelClientTestCase extends OpenSearchRestTestCase {
 
     protected static final String CONFLICT_PIPELINE_ID = "conflict_pipeline";
+    protected static final double DOUBLE_DELTA = 0.000001;
 
     private static RestHighLevelClient restHighLevelClient;
     private static boolean async = Booleans.parseBoolean(System.getProperty("tests.rest.async", "false"));

client/rest-high-level/src/test/java/org/opensearch/client/RankEvalIT.java

@@ -158,7 +158,7 @@ public void testMetrics() throws IOException {
 
             RankEvalRequest rankEvalRequest = new RankEvalRequest(spec, new String[] { "index", "index2" });
             RankEvalResponse response = execute(rankEvalRequest, highLevelClient()::rankEval, highLevelClient()::rankEvalAsync);
-            assertEquals(expectedScores[i], response.getMetricScore(), Double.MIN_VALUE);
+            assertEquals(expectedScores[i], response.getMetricScore(), DOUBLE_DELTA);
             i++;
         }
     }

gradle/missing-javadoc.gradle

@@ -127,6 +127,7 @@ configure([
   project(":plugins:analysis-icu"),
   project(":plugins:analysis-kuromoji"),
   project(":plugins:analysis-nori"),
+  project(":plugins:analysis-phonenumber"),
   project(":plugins:analysis-phonetic"),
   project(":plugins:analysis-smartcn"),
   project(":plugins:analysis-stempel"),

libs/core/src/main/java/org/opensearch/Version.java

@@ -110,6 +110,7 @@ public class Version implements Comparable<Version>, ToXContentFragment {
     public static final Version V_2_16_1 = new Version(2160199, org.apache.lucene.util.Version.LUCENE_9_11_1);
     public static final Version V_2_17_0 = new Version(2170099, org.apache.lucene.util.Version.LUCENE_9_11_1);
     public static final Version V_2_17_1 = new Version(2170199, org.apache.lucene.util.Version.LUCENE_9_11_1);
+    public static final Version V_2_17_2 = new Version(2170299, org.apache.lucene.util.Version.LUCENE_9_11_1);
     public static final Version V_2_18_0 = new Version(2180099, org.apache.lucene.util.Version.LUCENE_9_11_1);
     public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_12_0);
     public static final Version CURRENT = V_3_0_0;

libs/core/src/main/java/org/opensearch/core/common/Strings.java

@@ -815,4 +815,17 @@ public static String toLowercaseAscii(String in) {
         }
         return out.toString();
     }
+
+    /**
+     * Check whether every single character in the string is a digit.
+     *
+     * <p>An empty string returns {@code false}.</p>
+     *
+     * @param s the string, must not be null.
+     * @return {@code true} if the string only contains digits, {@code false} otherwise.
+     */
+    public static boolean isDigits(final String s) {
+        return !s.isEmpty() && s.chars().allMatch(Character::isDigit);
+    }
+
 }

libs/core/src/test/java/org/opensearch/core/common/StringsTests.java

@@ -114,4 +114,15 @@ public void testToStringToXContentWithOrWithoutParams() {
             containsString("\"color_from_param\":\"blue\"")
         );
     }
+
+    public void testIsDigits() {
+        assertTrue(Strings.isDigits("1"));
+        assertTrue(Strings.isDigits("123"));
+        assertFalse(Strings.isDigits(""));
+        assertFalse(Strings.isDigits("abc"));
+        assertFalse(Strings.isDigits("123a"));
+        assertFalse(Strings.isDigits("0x123"));
+        assertFalse(Strings.isDigits("123.4"));
+        assertFalse(Strings.isDigits("123f"));
+    }
 }

modules/ingest-geoip/build.gradle

@@ -41,7 +41,7 @@ opensearchplugin {
 dependencies {
   api('com.maxmind.geoip2:geoip2:4.2.1')
   // geoip2 dependencies:
-  api('com.maxmind.db:maxmind-db:3.1.0')
+  api('com.maxmind.db:maxmind-db:3.1.1')
   api("com.fasterxml.jackson.core:jackson-annotations:${versions.jackson}")
   api("com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}")
 

modules/ingest-geoip/licenses/maxmind-db-3.1.1.jar.sha1

@@ -0,0 +1 @@
+84a2a5d322fef1f1e82f05095683c8d53902baf1

modules/transport-netty4/licenses/netty-buffer-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+f1d77d15c0b781cd9395a2a956262766fd0c7602

modules/transport-netty4/licenses/netty-codec-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+5a49dfa2828d64bf756f670e63259115332744cf

modules/transport-netty4/licenses/netty-codec-http-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+fbce5a53884275662e68aaad70f88bf7e5d04164

modules/transport-netty4/licenses/netty-codec-http2-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+19ae07fdf99142a70338f8cda70a3d2edbc8e80a

modules/transport-netty4/licenses/netty-common-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+862712e292b162c8ccaa7847a6a54df8178f77e5

modules/transport-netty4/licenses/netty-handler-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+e56fbde4b9aa628eed15a5dbfbeb97877db88146

modules/transport-netty4/licenses/netty-resolver-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+10b23784b23d6a948930f52ba82874f1291b5873

modules/transport-netty4/licenses/netty-transport-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+e0225a575f487904be8517092cbd74e01913533c

modules/transport-netty4/licenses/netty-transport-native-unix-common-4.1.114.Final.jar.sha1

@@ -0,0 +1 @@
+d1171bb99411f282068f49d780cedf8c9adeabfd

plugins/analysis-phonenumber/build.gradle

@@ -0,0 +1,21 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ *
+ * Modifications Copyright OpenSearch Contributors. See
+ * GitHub history for details.
+ */
+
+apply plugin: 'opensearch.yaml-rest-test'
+
+opensearchplugin {
+  description 'Adds an analyzer for phone numbers to OpenSearch.'
+  classname 'org.opensearch.analysis.phone.PhoneNumberAnalysisPlugin'
+}
+
+dependencies {
+  implementation group: 'com.googlecode.libphonenumber', name: 'libphonenumber', version: '8.13.45'
+}

plugins/analysis-phonenumber/licenses/libphonenumber-8.13.45.jar.sha1

@@ -0,0 +1 @@
+bfac00f71616796abc7d8b135dda12558a0ccee2