Skip to content

Commit

Permalink
Merge branch 'main' into dependabot/gradle/plugins/discovery-gce/com.…
Browse files Browse the repository at this point in the history
…google.apis-google-api-services-compute-v1-rev20241015-2.0.0

Signed-off-by: gaobinlong <[email protected]>
  • Loading branch information
gaobinlong authored Oct 30, 2024
2 parents 8053d61 + 1e7c122 commit e17e5a4
Show file tree
Hide file tree
Showing 12 changed files with 498 additions and 181 deletions.
85 changes: 3 additions & 82 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,105 +5,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),

## [Unreleased 2.x]
### Added
- Add support for async deletion in S3BlobContainer ([#15621](https://github.com/opensearch-project/OpenSearch/pull/15621))
- MultiTermQueries in keyword fields now default to `indexed` approach and gated behind cluster setting ([#15637](https://github.com/opensearch-project/OpenSearch/pull/15637))
- [Workload Management] Add Integration Tests for Workload Management CRUD APIs ([#15955](https://github.com/opensearch-project/OpenSearch/pull/15955))
- [Workload Management] QueryGroup resource cancellation framework changes ([#15651](https://github.com/opensearch-project/OpenSearch/pull/15651))
- [Workload Management] Add orchestrator for wlm resiliency (QueryGroupService) ([#15925](https://github.com/opensearch-project/OpenSearch/pull/15925))
- [Workload Management] Add QueryGroup Stats API Logic ([15777](https://github.com/opensearch-project/OpenSearch/pull/15777))
- Fallback to Remote cluster-state on Term-Version check mismatch - ([#15424](https://github.com/opensearch-project/OpenSearch/pull/15424))
- Implement WithFieldName interface in ValuesSourceAggregationBuilder & FieldSortBuilder ([#15916](https://github.com/opensearch-project/OpenSearch/pull/15916))
- Add successfulSearchShardIndices in searchRequestContext ([#15967](https://github.com/opensearch-project/OpenSearch/pull/15967), [#16110](https://github.com/opensearch-project/OpenSearch/pull/16110))
- [Tiered Caching] Segmented cache changes ([#16047](https://github.com/opensearch-project/OpenSearch/pull/16047))
- Add support for msearch API to pass search pipeline name - ([#15923](https://github.com/opensearch-project/OpenSearch/pull/15923))
- Add _list/indices API as paginated alternate to _cat/indices ([#14718](https://github.com/opensearch-project/OpenSearch/pull/14718))
- Add success and failure metrics for async shard fetch ([#15976](https://github.com/opensearch-project/OpenSearch/pull/15976))
- Add new metric REMOTE_STORE to NodeStats API response ([#15611](https://github.com/opensearch-project/OpenSearch/pull/15611))
- Add support to dynamically resize threadpools size. ([#16236](https://github.com/opensearch-project/OpenSearch/pull/16236))
- [S3 Repository] Change default retry mechanism of s3 clients to Standard Mode ([#15978](https://github.com/opensearch-project/OpenSearch/pull/15978))
- Add changes to block calls in cat shards, indices and segments based on dynamic limit settings ([#15986](https://github.com/opensearch-project/OpenSearch/pull/15986))
- New `phone` & `phone-search` analyzer + tokenizer ([#15915](https://github.com/opensearch-project/OpenSearch/pull/15915))
- Add _list/shards API as paginated alternate to _cat/shards ([#14641](https://github.com/opensearch-project/OpenSearch/pull/14641))
- Latency and Memory allocation improvements to Multi Term Aggregation queries ([#14993](https://github.com/opensearch-project/OpenSearch/pull/14993))
- Flat object field use IndexOrDocValuesQuery to optimize query ([#14383](https://github.com/opensearch-project/OpenSearch/issues/14383))
- Add support for renaming aliases during snapshot restore ([#16292](https://github.com/opensearch-project/OpenSearch/pull/16292))
- Add method to return dynamic SecureTransportParameters from SecureTransportSettingsProvider interface ([#16387](https://github.com/opensearch-project/OpenSearch/pull/16387))
- URI path filtering support in cluster stats API ([#15938](https://github.com/opensearch-project/OpenSearch/pull/15938))
- [Star Tree - Search] Add support for metric aggregations with/without term query ([15289](https://github.com/opensearch-project/OpenSearch/pull/15289))
- Add support for restoring from snapshot with search replicas ([#16111](https://github.com/opensearch-project/OpenSearch/pull/16111))
- Add logic in master service to optimize performance and retain detailed logging for critical cluster operations. ([#14795](https://github.com/opensearch-project/OpenSearch/pull/14795))
- Add Setting to adjust the primary constraint weights ([#16471](https://github.com/opensearch-project/OpenSearch/pull/16471))
- Switch from `buildSrc/version.properties` to Gradle version catalog (`gradle/libs.versions.toml`) to enable dependabot to perform automated upgrades on common libs ([#16284](https://github.com/opensearch-project/OpenSearch/pull/16284))

### Dependencies
- Bump `com.azure:azure-identity` from 1.13.0 to 1.13.2 ([#15578](https://github.com/opensearch-project/OpenSearch/pull/15578))
- Bump `protobuf` from 3.22.3 to 3.25.4 ([#15684](https://github.com/opensearch-project/OpenSearch/pull/15684))
- Bump `org.apache.logging.log4j:log4j-core` from 2.23.1 to 2.24.1 ([#15858](https://github.com/opensearch-project/OpenSearch/pull/15858), [#16134](https://github.com/opensearch-project/OpenSearch/pull/16134))
- Bump `peter-evans/create-pull-request` from 6 to 7 ([#15863](https://github.com/opensearch-project/OpenSearch/pull/15863))
- Bump `com.nimbusds:oauth2-oidc-sdk` from 11.9.1 to 11.19.1 ([#15862](https://github.com/opensearch-project/OpenSearch/pull/15862))
- Bump `com.microsoft.azure:msal4j` from 1.17.0 to 1.17.2 ([#15945](https://github.com/opensearch-project/OpenSearch/pull/15945), [#16406](https://github.com/opensearch-project/OpenSearch/pull/16406))
- Bump `ch.qos.logback:logback-core` from 1.5.6 to 1.5.10 ([#15946](https://github.com/opensearch-project/OpenSearch/pull/15946), [#16307](https://github.com/opensearch-project/OpenSearch/pull/16307))
- Update protobuf from 3.25.4 to 3.25.5 ([#16011](https://github.com/opensearch-project/OpenSearch/pull/16011))
- Bump `org.roaringbitmap:RoaringBitmap` from 1.2.1 to 1.3.0 ([#16040](https://github.com/opensearch-project/OpenSearch/pull/16040))
- Bump `com.nimbusds:nimbus-jose-jwt` from 9.40 to 9.41.1 ([#16038](https://github.com/opensearch-project/OpenSearch/pull/16038))
- Bump `actions/github-script` from 5 to 7 ([#16039](https://github.com/opensearch-project/OpenSearch/pull/16039))
- Bump `dnsjava:dnsjava` from 3.6.1 to 3.6.2 ([#16041](https://github.com/opensearch-project/OpenSearch/pull/16041))
- Bump `com.maxmind.geoip2:geoip2` from 4.2.0 to 4.2.1 ([#16042](https://github.com/opensearch-project/OpenSearch/pull/16042))
- Bump `com.maxmind.db:maxmind-db` from 3.1.0 to 3.1.1 ([#16137](https://github.com/opensearch-project/OpenSearch/pull/16137))
- Bump Apache lucene from 9.11.1 to 9.12.0 ([#15333](https://github.com/opensearch-project/OpenSearch/pull/15333))
- Bump `com.azure:azure-core-http-netty` from 1.15.3 to 1.15.5 ([#16133](https://github.com/opensearch-project/OpenSearch/pull/16133), [#16311](https://github.com/opensearch-project/OpenSearch/pull/16311))
- Bump `org.jline:jline` from 3.26.3 to 3.27.0 ([#16135](https://github.com/opensearch-project/OpenSearch/pull/16135))
- Bump `netty` from 4.1.112.Final to 4.1.114.Final ([#16182](https://github.com/opensearch-project/OpenSearch/pull/16182))
- Bump `com.google.api-client:google-api-client` from 2.2.0 to 2.7.0 ([#16216](https://github.com/opensearch-project/OpenSearch/pull/16216))
- Bump `com.azure:azure-json` from 1.1.0 to 1.3.0 ([#16217](https://github.com/opensearch-project/OpenSearch/pull/16217))
- Bump `io.grpc:grpc-api` from 1.57.2 to 1.68.0 ([#16213](https://github.com/opensearch-project/OpenSearch/pull/16213))
- Bump `com.squareup.okio:okio` from 3.9.0 to 3.9.1 ([#16212](https://github.com/opensearch-project/OpenSearch/pull/16212))
- Bump `me.champeau.gradle.japicmp` from 0.4.3 to 0.4.4 ([#16309](https://github.com/opensearch-project/OpenSearch/pull/16309))
- Bump `com.google.oauth-client:google-oauth-client` from 1.35.0 to 1.36.0 ([#16306](https://github.com/opensearch-project/OpenSearch/pull/16306))
- Bump `lycheeverse/lychee-action` from 1.10.0 to 2.0.2 ([#16310](https://github.com/opensearch-project/OpenSearch/pull/16310))
- Bump `com.google.code.gson:gson` from 2.10.1 to 2.11.0 ([#16308](https://github.com/opensearch-project/OpenSearch/pull/16308))
- Bump `com.google.apis:google-api-services-compute` from v1-rev20240407-2.0.0 to v1-rev20241015-2.0.0 ([#16502](https://github.com/opensearch-project/OpenSearch/pull/16502))

### Changed
- Add support for docker compose v2 in TestFixturesPlugin ([#16049](https://github.com/opensearch-project/OpenSearch/pull/16049))
- Remove identity-related feature flagged code from the RestController ([#15430](https://github.com/opensearch-project/OpenSearch/pull/15430))
- Remove Identity FeatureFlag ([#16024](https://github.com/opensearch-project/OpenSearch/pull/16024))
- Ensure RestHandler.Wrapper delegates all implementations to the wrapped handler ([#16154](https://github.com/opensearch-project/OpenSearch/pull/16154))
- Enable coordinator search.request_stats_enabled by default ([#16290](https://github.com/opensearch-project/OpenSearch/pull/16290))
- Code cleanup: Remove ApproximateIndexOrDocValuesQuery ([#16273](https://github.com/opensearch-project/OpenSearch/pull/16273))
- Optimise clone operation for incremental full cluster snapshots ([#16296](https://github.com/opensearch-project/OpenSearch/pull/16296))
- Update last seen cluster state in the commit phase ([#16215](https://github.com/opensearch-project/OpenSearch/pull/16215))
- Make multiple settings dynamic for tuning on larger clusters([#16347](https://github.com/opensearch-project/OpenSearch/pull/16347))

### Deprecated

### Removed

### Fixed
- Fix wildcard query containing escaped character ([#15737](https://github.com/opensearch-project/OpenSearch/pull/15737))
- Fix case-insensitive query on wildcard field ([#15882](https://github.com/opensearch-project/OpenSearch/pull/15882))
- Add validation for the search backpressure cancellation settings ([#15501](https://github.com/opensearch-project/OpenSearch/pull/15501))
- Fix search_as_you_type not supporting multi-fields ([#15988](https://github.com/opensearch-project/OpenSearch/pull/15988))
- Avoid infinite loop when `flat_object` field contains invalid token ([#15985](https://github.com/opensearch-project/OpenSearch/pull/15985))
- Fix infinite loop in nested agg ([#15931](https://github.com/opensearch-project/OpenSearch/pull/15931))
- Fix update settings with null replica not honoring cluster setting bug ([#14948](https://github.com/opensearch-project/OpenSearch/pull/14948))
- Fix race condition in node-join and node-left ([#15521](https://github.com/opensearch-project/OpenSearch/pull/15521))
- Streaming bulk request hangs ([#16158](https://github.com/opensearch-project/OpenSearch/pull/16158))
- Fix warnings from SLF4J on startup when repository-s3 is installed ([#16194](https://github.com/opensearch-project/OpenSearch/pull/16194))
- Fix protobuf-java leak through client library dependencies ([#16254](https://github.com/opensearch-project/OpenSearch/pull/16254))
- Fix get index settings API doesn't show `number_of_routing_shards` setting when it was explicitly set ([#16294](https://github.com/opensearch-project/OpenSearch/pull/16294))
- Fix multi-search with template doesn't return status code ([#16265](https://github.com/opensearch-project/OpenSearch/pull/16265))
- [Streaming Indexing] Fix intermittent 'The bulk request must be terminated by a newline [\n]' failures [#16337](https://github.com/opensearch-project/OpenSearch/pull/16337))
- Fix wrong default value when setting `index.number_of_routing_shards` to null on index creation ([#16331](https://github.com/opensearch-project/OpenSearch/pull/16331))
- [Workload Management] Make query groups persistent across process restarts ([#16370](https://github.com/opensearch-project/OpenSearch/pull/16370))
- [Workload Management] Enhance rejection mechanism in workload management ([#16417](https://github.com/opensearch-project/OpenSearch/pull/16417))
- Fix inefficient Stream API call chains ending with count() ([#15386](https://github.com/opensearch-project/OpenSearch/pull/15386))
- Fix array hashCode calculation in ResyncReplicationRequest ([#16378](https://github.com/opensearch-project/OpenSearch/pull/16378))
- Fix missing fields in task index mapping to ensure proper task result storage ([#16201](https://github.com/opensearch-project/OpenSearch/pull/16201))
- Fix typo super->sb in method toString() of RemoteStoreNodeAttribute ([#15362](https://github.com/opensearch-project/OpenSearch/pull/15362))
- [Workload Management] Fixing Create/Update QueryGroup TransportActions to execute from non-cluster manager nodes ([16422](https://github.com/opensearch-project/OpenSearch/pull/16422))
- Fix flaky test in `testApproximateRangeWithSizeOverDefault` by adjusting totalHits assertion logic ([#16434](https://github.com/opensearch-project/OpenSearch/pull/16434#pullrequestreview-2386999409))
- Revert changes to upload remote state manifest using minimum codec version([#16403](https://github.com/opensearch-project/OpenSearch/pull/16403))
- Ensure index templates are not applied to system indices ([#16418](https://github.com/opensearch-project/OpenSearch/pull/16418))

### Security

[Unreleased 2.x]: https://github.com/opensearch-project/OpenSearch/compare/2.17...2.x
[Unreleased 2.x]: https://github.com/opensearch-project/OpenSearch/compare/2.18...2.x
20 changes: 12 additions & 8 deletions buildSrc/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ if (project == rootProject) {
// we update the version property to reflect if we are building a snapshot or a release build
// we write this back out below to load it in the Build.java which will be shown in rest main action
// to indicate this being a snapshot build or a release build.
Properties props = VersionPropertiesLoader.loadBuildSrcVersion(project.file('version.properties'))
Properties props = VersionPropertiesLoader.loadBuildSrcVersion(project)
version = props.getProperty("opensearch")

def generateVersionProperties = tasks.register("generateVersionProperties", WriteProperties) {
Expand Down Expand Up @@ -287,15 +287,19 @@ if (project != rootProject) {
}
}

// Define this here because we need it early.
// Define this here because we need it early. It uses VersionCatalogsExtension to extract all versions
// and converts to a Java Properties object
class VersionPropertiesLoader {
static Properties loadBuildSrcVersion(File input) throws IOException {
static Properties loadBuildSrcVersion(Project project) throws IOException {
Properties props = new Properties();
InputStream is = new FileInputStream(input)
try {
props.load(is)
} finally {
is.close()

var catalogs = project.extensions.getByType(VersionCatalogsExtension)
var libs = catalogs.named("libs")
libs.getVersionAliases().forEach {
libs.findVersion(it).ifPresent { v ->
// Gradle replaces '_' with '.' so 'google_http_client' becomes 'google.http.client' instead
props.setProperty(it.replaceAll("[.]", "_"), v.requiredVersion)
}
}
loadBuildSrcVersion(props, System.getProperties())
return props
Expand Down
8 changes: 8 additions & 0 deletions buildSrc/settings.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
*/

include 'reaper'

dependencyResolutionManagement {
versionCatalogs {
libs {
from(files("../gradle/libs.versions.toml"))
}
}
}
79 changes: 1 addition & 78 deletions buildSrc/version.properties
Original file line number Diff line number Diff line change
@@ -1,79 +1,2 @@
# Please use ../gradle/libs.versions.toml for dependency management
opensearch = 3.0.0
lucene = 9.12.0

bundled_jdk_vendor = adoptium
bundled_jdk = 23.0.1+11

# optional dependencies
spatial4j = 0.7
jts = 1.15.0
jackson = 2.17.2
jackson_databind = 2.17.2
snakeyaml = 2.1
icu4j = 75.1
supercsv = 2.4.0
log4j = 2.21.0
slf4j = 1.7.36
asm = 9.7
jettison = 1.5.4
woodstox = 6.4.0
kotlin = 1.7.10
antlr4 = 4.13.1
guava = 32.1.1-jre
protobuf = 3.25.5
jakarta_annotation = 1.3.5
google_http_client = 1.44.1
tdigest = 3.3
hdrhistogram = 2.2.2
grpc = 1.68.0

# when updating the JNA version, also update the version in buildSrc/build.gradle
jna = 5.13.0

netty = 4.1.114.Final
joda = 2.12.7

# project reactor
reactor_netty = 1.1.23
reactor = 3.5.20

# client dependencies
httpclient5 = 5.3.1
httpcore5 = 5.2.5
httpclient = 4.5.14
httpcore = 4.4.16
httpasyncclient = 4.1.5
commonslogging = 1.2
commonscodec = 1.16.1
commonslang = 3.14.0
commonscompress = 1.26.1
commonsio = 2.16.0
# plugin dependencies
aws = 2.20.86
reactivestreams = 1.0.4

# when updating this version, you need to ensure compatibility with:
# - plugins/ingest-attachment (transitive dependency, check the upstream POM)
# - distribution/tools/plugin-cli
bouncycastle=1.78
# test dependencies
randomizedrunner = 2.7.1
junit = 4.13.2
hamcrest = 2.1
mockito = 5.14.1
objenesis = 3.2
bytebuddy = 1.15.4

# benchmark dependencies
jmh = 1.35

# compression
zstd = 1.5.5-5

jzlib = 1.1.3

resteasy = 6.2.4.Final

# opentelemetry dependencies
opentelemetry = 1.41.0
opentelemetrysemconv = 1.27.0-alpha
80 changes: 80 additions & 0 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
[versions]
opensearch = "3.0.0"
lucene = "9.12.0"

bundled_jdk_vendor = "adoptium"
bundled_jdk = "23.0.1+11"

# optional dependencies
spatial4j = "0.7"
jts = "1.15.0"
jackson = "2.17.2"
jackson_databind = "2.17.2"
snakeyaml = "2.1"
icu4j = "75.1"
supercsv = "2.4.0"
log4j = "2.21.0"
slf4j = "1.7.36"
asm = "9.7"
jettison = "1.5.4"
woodstox = "6.4.0"
kotlin = "1.7.10"
antlr4 = "4.13.1"
guava = "32.1.1-jre"
protobuf = "3.25.5"
jakarta_annotation = "1.3.5"
google_http_client = "1.44.1"
tdigest = "3.3"
hdrhistogram = "2.2.2"
grpc = "1.68.0"

# when updating the JNA version, also update the version in buildSrc/build.gradle
jna = "5.13.0"

netty = "4.1.114.Final"
joda = "2.12.7"

# project reactor
reactor_netty = "1.1.23"
reactor = "3.5.20"

# client dependencies
httpclient5 = "5.3.1"
httpcore5 = "5.2.5"
httpclient = "4.5.14"
httpcore = "4.4.16"
httpasyncclient = "4.1.5"
commonslogging = "1.2"
commonscodec = "1.16.1"
commonslang = "3.14.0"
commonscompress = "1.26.1"
commonsio = "2.16.0"
# plugin dependencies
aws = "2.20.86"
reactivestreams = "1.0.4"

# when updating this version, you need to ensure compatibility with:
# - plugins/ingest-attachment (transitive dependency, check the upstream POM)
# - distribution/tools/plugin-cli
bouncycastle="1.78"
# test dependencies
randomizedrunner = "2.7.1"
junit = "4.13.2"
hamcrest = "2.1"
mockito = "5.14.1"
objenesis = "3.2"
bytebuddy = "1.15.4"

# benchmark dependencies
jmh = "1.35"

# compression
zstd = "1.5.5-5"

jzlib = "1.1.3"

resteasy = "6.2.4.Final"

# opentelemetry dependencies
opentelemetry = "1.41.0"
opentelemetrysemconv = "1.27.0-alpha"
Loading

0 comments on commit e17e5a4

Please sign in to comment.