2.18.0

2024-10-28 Version 2.18.0 Release Notes

[2.18.0]

Added

• [Workload Management] Add orchestrator for wlm resiliency (QueryGroupService) (#15925)
• [Offline Nodes] Adds offline-tasks library containing various interfaces to be used for Offline Background Tasks. (#13574)
• Add support for async deletion in S3BlobContainer (#15621)
• [Workload Management] QueryGroup resource cancellation framework changes (#15651)
• [Workload Management] Add QueryGroup Stats API Logic (15777)
• Implement WithFieldName interface in ValuesSourceAggregationBuilder & FieldSortBuilder (#15916)
• Add successfulSearchShardIndices in searchRequestContext (#15967, #16110)
• Fallback to Remote cluster-state on Term-Version check mismatch - (#15424)
• [Tiered Caching] Segmented cache changes (#16047)
• Add support for msearch API to pass search pipeline name - (#15923)
• Add success and failure metrics for async shard fetch (#15976)
• Add support to dynamically resize threadpools size. (#16236)
• [S3 Repository] Change default retry mechanism of s3 clients to Standard Mode (#15978)
• [Workload Management] Add Integration Tests for Workload Management CRUD APIs (#15955)
• Add new metric REMOTE_STORE to NodeStats API response (#15611)
• New phone & phone-search analyzer + tokenizer (#15915)
• Add _list/indices API as paginated alternate to _cat/indices (#14718)
• Add changes to block calls in cat shards, indices and segments based on dynamic limit settings (#15986)
• Flat object field use IndexOrDocValuesQuery to optimize query (#14383)
• Add method to return dynamic SecureTransportParameters from SecureTransportSettingsProvider interface (#16387
• Add _list/shards API as paginated alternate to _cat/shards (#14641)
• [Star Tree - Search] Add support for metric aggregations with/without term query (15289)
• URI path filtering support in cluster stats API (#15938)
• Add support for renaming aliases during snapshot restore (#16292)

Dependencies

• Bump org.apache.logging.log4j:log4j-core from 2.23.1 to 2.24.0 (#15858)
• Bump com.azure:azure-identity from 1.13.0 to 1.13.2 (#15578)
• Bump protobuf from 3.22.3 to 3.25.4 (#15684)
• Bump peter-evans/create-pull-request from 6 to 7 (#15863)
• Bump com.nimbusds:oauth2-oidc-sdk from 11.9.1 to 11.19.1 (#15862)
• Bump com.microsoft.azure:msal4j from 1.17.0 to 1.17.2 (#15945, #16406)
• Bump ch.qos.logback:logback-core from 1.5.6 to 1.5.10 (#15946, #16307)
• Update protobuf from 3.25.4 to 3.25.5 (#16011)
• Bump org.roaringbitmap:RoaringBitmap from 1.2.1 to 1.3.0 (#16040)
• Bump com.nimbusds:nimbus-jose-jwt from 9.40 to 9.41.1 (#16038)
• Bump actions/github-script from 5 to 7 (#16039)
• Bump dnsjava:dnsjava from 3.6.1 to 3.6.2 (#16041)
• Bump com.maxmind.geoip2:geoip2 from 4.2.0 to 4.2.1 (#16042)
• Bump com.maxmind.db:maxmind-db from 3.1.0 to 3.1.1 (#16137)
• Bump Apache lucene from 9.11.1 to 9.12.0 (#15333)
• Bump com.azure:azure-core-http-netty from 1.15.3 to 1.15.5 (#16133, #16311)
• Bump netty from 4.1.112.Final to 4.1.114.Final (#16182)
• Bump com.google.api-client:google-api-client from 2.2.0 to 2.7.0 (#16216)
• Bump com.azure:azure-json from 1.1.0 to 1.3.0 (#16217)
• Bump io.grpc:grpc-api from 1.57.2 to 1.68.0 (#16213)
• Bump org.jline:jline from 3.26.3 to 3.27.0 (#16135)
• Bump com.squareup.okio:okio from 3.9.0 to 3.9.1 (#16212)
• Bump lycheeverse/lychee-action from 1.10.0 to 2.0.2 (#16310)
• Bump com.google.code.gson:gson from 2.10.1 to 2.11.0 (#16308)
• Bump io.grpc:grpc-api from 1.57.2 to 1.68.0 (#16213)
• Bump me.champeau.gradle.japicmp from 0.4.3 to 0.4.4 (#16309)
• Bump com.google.oauth-client:google-oauth-client from 1.35.0 to 1.36.0 (#16306)

Changed

• Add support for docker compose v2 in TestFixturesPlugin (#16049)
• Remove identity-related feature flagged code from the RestController (#15430)
• Remove Identity FeatureFlag (#16024)
• Ensure RestHandler.Wrapper delegates all implementations to the wrapped handler (#16154)
• Enable coordinator search.request_stats_enabled by default (#16290)
• Optimise clone operation for incremental full cluster snapshots (#16296)
• Code cleanup: Remove ApproximateIndexOrDocValuesQuery (#16273)
• Update last seen cluster state in the commit phase (#16215)
• Make multiple settings dynamic for tuning on larger clusters(#16347)

Fixed

• Fix wildcard query containing escaped character (#15737)
• Fix case-insensitive query on wildcard field (#15882)
• Add validation for the search backpressure cancellation settings (#15501)
• Fix search_as_you_type not supporting multi-fields (#15988)
• Avoid infinite loop when flat_object field contains invalid token (#15985)
• Fix infinite loop in nested agg (#15931)
• Fix update settings with null replica not honoring cluster setting bug (#14948)
• Fix race condition in node-join and node-left (#15521)
• Streaming bulk request hangs (#16158)
• Fix warnings from SLF4J on startup when repository-s3 is installed (#16194)
• Fix protobuf-java leak through client library dependencies (#16254)
• Fix multi-search with template doesn't return status code (#16265)
• [Streaming Indexing] Fix intermittent 'The bulk request must be terminated by a newline [\n]' fa...

2.17.1

2024-10-01 Version 2.17.1 Release Notes

[2.17.1]

Added

• Add path prefix support to hashed prefix snapshots (#15664)
• Memory optimisations in _cluster/health API (#15492)

Dependencies

Changed

Deprecated

Removed

Fixed

2.17.0

2024-09-17 Version 2.17.0 Release Notes

[2.17.0]

Added

• [Workload Management] Add Settings for Workload Management feature (#15028)
• Fix for hasInitiatedFetching to fix allocation explain and manual reroute APIs ((#14972)
• [Workload Management] Add queryGroupId to Task (14708)
• Add setting to ignore throttling nodes for allocation of unassigned primaries in remote restore (#14991)
• [Workload Management] Add Delete QueryGroup API Logic (#14735)
• [Streaming Indexing] Enhance RestClient with a new streaming API support (#14437)
• Add basic aggregation support for derived fields (#14618)
• [Workload Management] Add Create QueryGroup API Logic (#14680)- [Workload Management] Add Create QueryGroup API Logic (#14680)
• Add ThreadContextPermission for markAsSystemContext and allow core to perform the method (#15016)
• Add ThreadContextPermission for stashAndMergeHeaders and stashWithOrigin (#15039)
• [Concurrent Segment Search] Support composite aggregations with scripting (#15072)
• Add rangeQuery and regexpQuery for constant_keyword field type (#14711)
• Add took time to request nodes stats (#15054)
• [Workload Management] Add Get QueryGroup API Logic (14709)
• [Workload Management] Add Update QueryGroup API Logic (#14775)
• [Workload Management] QueryGroup resource tracking framework changes (#13897)
• Support filtering on a large list encoded by bitmap (#14774)
• Add slice execution listeners to SearchOperationListener interface (#15153)
• Make balanced shards allocator timebound (#15239)
• Add allowlist setting for ingest-geoip and ingest-useragent (#15325)
• Adding access to noSubMatches and noOverlappingMatches in Hyphenation (#13895)
• Star tree mapping changes (#14605)
• Add support for index level max slice count setting for concurrent segment search (#15336)
• Support cancellation for cat shards and node stats API.(#13966)
• [Streaming Indexing] Introduce bulk HTTP API streaming flavor (#15381)
• Add support for centralize snapshot creation with pinned timestamp (#15124)
• Add concurrent search support for Derived Fields (#15326)
• [Workload Management] Add query group stats constructs (#15343))
• Add limit on number of processors for Ingest pipeline(#15460).
• Add runAs to Subject interface and introduce IdentityAwarePlugin extension point (#14630)
• [Workload Management] Add rejection logic for co-ordinator and shard level requests (#15428))
• Adding translog durability validation in index templates (#15494)
• [Range Queries] Add new approximateable query framework to short-circuit range queries (#13788)
• [Workload Management] Add query group level failure tracking (#15227)
• [Reader Writer Separation] Add experimental search replica shard type to achieve reader writer separation (#15237)
• Add index creation using the context field (#15290)
• [Remote Publication] Add remote download stats (#15291)
• Add support to upload snapshot shard blobs with hashed prefix (#15426)
• Add prefix support to hashed prefix & infix path types on remote store (#15557)
• Add canRemain method to TargetPoolAllocationDecider to move shards from local to remote pool for hot to warm tiering (#15010)
• Add support for pluggable deciders for concurrent search (#15363)
• Optimise snapshot deletion to speed up snapshot deletion and creation (#15568)
• [Remote Publication] Added checksum validation for cluster state behind a cluster setting (#15218)
• Optimize NodeIndicesStats output behind flag (#14454)
• Add support for comma-separated list of index names to be used with Snapshot Status API (#15409)[SnapshotV2] Snapshot Status API changes (#15409))
• ClusterManagerTaskThrottler Improvements (#15508)
• Relax the join validation for Remote State publication (#15471)
• Reset DiscoveryNodes in all transport node actions request (#15131)
• MultiTermQueries in keyword fields now default to indexed approach and gated behind cluster setting (#15637)
• Adding WithFieldName interface for QueryBuilders with fieldName (#15705)
• Making _cat/allocation API use indexLevelStats (#15292)
• Static RemotePublication setting added, removed experimental feature flag (#15478)
• [Remote Publication] Upload incremental cluster state on master re-election (#15145)

Dependencies

• Bump netty from 4.1.111.Final to 4.1.112.Final (#15081)
• Bump org.apache.commons:commons-lang3 from 3.14.0 to 3.15.0 (#14861)
• OpenJDK Update (July 2024 Patch releases) (#14998)
• Bump com.microsoft.azure:msal4j from 1.16.1 to 1.17.0 (#14995, #15420)
• Bump actions/github-script from 6 to 7 (#14997)
• Bump org.tukaani:xz from 1.9 to 1.10 (#15110)
• Bump org.apache.avro:avro from 1.11.3 to 1.12.0 in /plugins/repository-hdfs (#15119)
• Bump org.bouncycastle:bcpg-fips from 1.0.7.1 to 2.0.9 (#15103, #15299)
• Bump com.azure:azure-core from 1.49.1 to 1.51.0 (#15111)
• Bump org.xerial.snappy:snappy-java from 1.1.10.5 to 1.1.10.6 (#15207)
• Bump com.azure:azure-xml from 1.0.0 to 1.1.0 (#15206)
• Bump reactor from 3.5.19 to 3.5.20 (#15262)
• Bump reactor-netty from 1.1.21 to 1.1.22 (#15262)
• Bump org.apache.kerby:kerb-admin from 2.0.3 to 2.1.0 (#15301)
• Bump com.azure:azure-core-http-netty from 1.15.1 to 1.15.3 (#15300)
• Bump com.gradle.develocity from 3.17.6 to 3.18 (#15297)
• Bump commons-cli:commons-cli from 1.8.0 to 1.9.0 (#15298)
• Bump opentelemetry from 1.40.0 to 1.41.0 ([#15361](https://github.com/...

2.16.0

2024-07-24 Version 2.16.0 Release Notes

[2.16.0]

Added

• Add fingerprint ingest processor (#13724)
• [Remote Store] Rate limiter for remote store low priority uploads (#14374)
• Apply the date histogram rewrite optimization to range aggregation (#13865)
• [Writable Warm] Add composite directory implementation and integrate it with FileCache (12782)
• [Workload Management] Add QueryGroup schema (13669)
• Add batching supported processor base type AbstractBatchingProcessor (#14554)
• Fix race condition while parsing derived fields from search definition (14445)
• Add strict_allow_templates dynamic mapping option (#14555)
• Add allowlist setting for ingest-common and search-pipeline-common processors (#14439)
• [Workload Management] add queryGroupId header propagator across requests and nodes (#14614)
• Create SystemIndexRegistry with helper method matchesSystemIndex (#14415)
• Print reason why parent task was cancelled (#14604)
• Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750)
• Add Plugin interface for loading application based configuration templates ((#14659))
• Refactor remote-routing-table service inline with remote state interfaces(#14668)
• Add shard-diff path to diff manifest to reduce number of read calls remote store ((#14684))
• Add SortResponseProcessor to Search Pipelines ((#14785))
• Add prefix mode verification setting for repository verification ((#14790))
• Add SplitResponseProcessor to Search Pipelines ((#14800))
• Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call (14749)
• Reduce logging in DEBUG for MasterService:run (#14795)
• Refactor remote-routing-table service inline with remote state interfaces(#14668)
• Add rest, transport layer changes for hot to warm tiering - dedicated setup ((#13980)
• Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273)
• Optimize Cluster Stats Indices to precomute node level stats (#14426)
• Create listener to refresh search thread resource usage (#14832)
• Add logic to create index templates (v2) using context field (#14811)

Dependencies

• Update to Apache Lucene 9.11.1 (#14042, #14576)
• Bump netty from 4.1.110.Final to 4.1.111.Final (#14356)
• Bump org.wiremock:wiremock-standalone from 3.3.1 to 3.6.0 (#14361)
• Bump reactor from 3.5.17 to 3.5.19 (#14395, #14697)
• Bump reactor-netty from 1.1.19 to 1.1.21 (#14395, #14697)
• Bump commons-net:commons-net from 3.10.0 to 3.11.1 (#14396)
• Bump com.nimbusds:nimbus-jose-jwt from 9.37.3 to 9.40 (#14398)
• Bump org.apache.commons:commons-configuration2 from 2.10.1 to 2.11.0 (#14399)
• Bump com.gradle.develocity from 3.17.4 to 3.17.5 (#14397)
• Bump opentelemetry from 1.36.0 to 1.40.0 (#14457, #14674)
• Bump opentelemetry-semconv from 1.25.0-alpha to 1.26.0-alpha (#14674)
• Bump azure-identity from 1.11.4 to 1.13.0, Bump msal4j from 1.14.3 to 1.15.1, Bump msal4j-persistence-extension from 1.2.0 to 1.3.0 (#14506)
• Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 (#14517)
• Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 (#14610)
• Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 (#14672)
• Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 (#14673)
• Bump jackson from 2.17.1 to 2.17.2 (#14687)
• Bump net.minidev:json-smart from 2.5.0 to 2.5.1 (#14748)

Changed

• [Tiered Caching] Move query recomputation logic outside write lock (#14187)
• unsignedLongRangeQuery now returns MatchNoDocsQuery if the lower bounds are greater than the upper bounds (#14416)
• Make the class CommunityIdProcessor final (#14448)
• Updated the indices.query.bool.max_clause_count setting from being static to dynamically updateable (#13568)
• Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (#14575)
• Add @internalapi annotation to japicmp exclusions (#14597)
• Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)
• Make reroute iteration time-bound for large shard allocations (#14848)

Deprecated

• Deprecate batch_size parameter on bulk API (#14725)

Removed

• Remove query categorization changes (#14759)

Fixed

• Fix bug in SBP cancellation logic (#13259)
• Fix handling of Short and Byte data types in ScriptProcessor ingest pipeline (#14379)
• Switch to iterative version of WKT format parser (#14086)
• Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959)
• Fix the computed max shards of cluster to avoid int overflow (#14155)
• Fixed rest-high-level client searchTemplate & mtermVectors endpoints to have a leading slash (#14465)
• Write shard level metadata blob when snapshotting searchable snapshot indexes (#13190)
• Fix aggs result of NestedAggregator with sub NestedAggregator (#13324)
• Fix fs info reporting negative available size (#11573)
• Add ListPitInfo::getKeepAlive() getter (#14495)
• Fix FuzzyQuery in keyword field will use IndexOrDocValuesQuery when both of index and doc_value are true (#14378)
• Fix file cache initialization (#14004)
• Handle NPE in GetResult if "found" field is missing (#14552)
• Fix create or update alias API doesn't throw exception for unsupported parameters (#14719)
• Refactoring FilterPath.parse by using an iterative approach ([#14200...

1.3.18

2024-07-09 Version 1.3.18 Release Notes

Upgrades

• Bump netty from 4.1.110.Final to 4.1.111.Final (#14356)

2.15.0

2024-06-12 Version 2.15.0 Release Notes

[2.15.0]

Added

• Add leader and follower check failure counter metrics (#12439)
• Add latency metrics for instrumenting critical clusterManager code paths (#12333)
• Add support for Azure Managed Identity in repository-azure (#12423)
• Add useCompoundFile index setting (#13478)
• Make outbound side of transport protocol dependent (#13293)
• [Remote Store] Upload translog checkpoint as object metadata to translog.tlog(#13637)
• [Remote Store] Add dynamic cluster settings to set timeout for segments upload to Remote Store (#13679)
• Add getMetadataFields to MapperService (#13819)
• Add "wildcard" field type that supports efficient wildcard, prefix, and regexp queries (#13461)
• Allow setting query parameters on requests (#13776)
• Add dynamic action retry timeout setting (#14022)
• Add capability to disable source recovery_source for an index (#13590)
• Add remote routing table for remote state publication with experimental feature flag (#13304)
• Add upload flow for writing routing table to remote store (#13870)
• Add dynamic action retry timeout setting (#14022)
• [Remote Store] Add support to disable flush based on translog reader count (#14027)
• Add recovery chunk size setting (#13997)
• [Query Insights] Add exporter support for top n queries (#12982)
• [Query Insights] Add X-Opaque-Id to search request metadata for top n queries (#13374)
• [Streaming Indexing] Enhance RestAction with request / response streaming support (#13772)
• Move Remote Store Migration from DocRep to GA and modify remote migration settings name (#14100)
• [Remote State] Add async remote state deletion task running on an interval, configurable by a setting (#13995)
• Add remote routing table for remote state publication with experimental feature flag (#13304)
• Add support for query level resource usage tracking (#13172)
• [Query Insights] Add cpu and memory metrics to top n queries (#13739)
• Derived field object type support (#13720)
• Support Dynamic Pruning in Cardinality Aggregation (#13821)

Dependencies

• Bump com.github.spullara.mustache.java:compiler from 0.9.10 to 0.9.13 (#13329, #13559)
• Bump org.apache.commons:commons-text from 1.11.0 to 1.12.0 (#13557)
• Bump org.hdrhistogram:HdrHistogram from 2.1.12 to 2.2.2 (#13556, #13986)
• Bump com.gradle.enterprise from 3.17.2 to 3.17.4 (#13641, #13753)
• Bump org.apache.hadoop:hadoop-minicluster from 3.3.6 to 3.4.0 (#13642)
• Bump mockito from 5.11.0 to 5.12.0 (#13665)
• Bump com.google.code.gson:gson from 2.10.1 to 2.11.0 (#13752)
• Bump ch.qos.logback:logback-core from 1.5.3 to 1.5.6 (#13756)
• Bump netty from 4.1.109.Final to 4.1.110.Final (#13802)
• Bump jackson from 2.17.0 to 2.17.1 (#13817)
• Bump reactor from 3.5.15 to 3.5.17 (#13825)
• Bump reactor-netty from 1.1.17 to 1.1.19 (#13825)
• Bump commons-cli:commons-cli from 1.7.0 to 1.8.0 (#13840)
• Bump org.apache.xmlbeans:xmlbeans from 5.2.0 to 5.2.1 (#13839)
• Bump actions/checkout from 3 to 4 (#13935)
• Bump com.netflix.nebula.ospackage-base from 11.9.0 to 11.9.1 (#13933)
• Bump com.azure:azure-core-http-netty from 1.12.8 to 1.15.1 (#14128)
• Bump tim-actions/get-pr-commits from 1.1.0 to 1.3.1 (#14126)

Changed

• Add ability for Boolean and date field queries to run when only doc_values are enabled (#11650)
• Refactor implementations of query phase searcher, allow QueryCollectorContext to have zero collectors (#13481)
• Adds support to inject telemetry instances to plugins (#13636)
• Adds support to provide tags with value in Gauge metric. (#13994)
• Move cache removal notifications outside lru lock (#14017)

Removed

• Remove handling of index.mapper.dynamic in AutoCreateIndex(#13067)

Fixed

• Fix get field mapping API returns 404 error in mixed cluster with multiple versions (#13624)
• Allow clearing remote_store.compatibility_mode setting (#13646)
• Painless: ensure type "UnmodifiableMap" for params (#13885)
• Don't return negative scores from multi_match query with cross_fields type (#13829)
• Pass parent filter to inner hit query (#13903)
• Fix NPE on restore searchable snapshot (#13911)
• Fix double invocation of postCollection when MultiBucketCollector is present (#14015)
• Fix ReplicaShardBatchAllocator to batch shards without duplicates (#13710)
• Java high-level REST client bulk() is not respecting the bulkRequest.requireAlias(true) method call (#14146)
• Fix ShardNotFoundException during request cache clean up (#14219)
• Fix the rewrite method for MatchOnlyText field query (#14248)
• Fix Concurrent Modification Exception in Indices Request Cache(#14032)

1.3.17

2024-05-30 Version 1.3.17 Release Notes

Upgrades

• OpenJDK Update (April 2024 Patch releases), update to Eclipse Temurin 11.0.23+9 (#13406)
• Upgrade BouncyCastle dependencies from 1.75 to 1.78.1 resolving [CVE-2024-30172], [CVE-2024-30171] and [CVE-2024-29857]
• Bump netty from 4.1.109.Final to 4.1.110.Final (#13802)

2.14.0

2024-05-07 Version 2.14.0 Release Notes

[2.14.0]

Added

• Add explicit dependency to validatePom and generatePom tasks (#12909)
• Constant Keyword Field (#12285)
• [Concurrent Segment Search] Perform buildAggregation concurrently and support Composite Aggregations (#12697)
• Convert ingest processor supports ip type (#12818)
• Allow setting KEYSTORE_PASSWORD through env variable (#12865)
• Add a counter to node stat (and _cat/shards) api to track shard going from idle to non-idle (#12768)
• [Concurrent Segment Search] Disable concurrent segment search for system indices and throttled requests (#12954)
• Rename ingest processor supports overriding target field if exists (#12990)
• [Tiered Caching] Make took time caching policy setting dynamic (#13063)
• Detect breaking changes on pull requests (#9044)
• Add cluster primary balance contraint for rebalancing with buffer (#12656)
• [Remote Store] Make translog transfer timeout configurable (#12704)
• Derived fields support to derive field values at query time without indexing (#12569)
• Add support for more than one protocol for transport (#12967)
• [Tiered Caching] Add dimension-based stats to ICache implementations. (#12531)
• Add changes for overriding remote store and replication settings during snapshot restore. (#11868)
• Reject Resize index requests (i.e, split, shrink and clone), While DocRep to SegRep migration is in progress.(#12686)
• Add an individual setting of rate limiter for segment replication (#12959)
• [Tiered Caching] Add dimension-based stats to TieredSpilloverCache (#13236)
• [Tiered Caching] Expose new cache stats API (#13237)
• [Streaming Indexing] Ensure support of the new transport by security plugin (#13174)
• Add cluster setting to dynamically configure the buckets for filter rewrite optimization. (#13179)
• [Tiered caching] Make Indices Request Cache Stale Key Mgmt Threshold setting dynamic (#12941)
• Make search query counters dynamic to support all query types (#12601)
• [Tiered Caching] Gate new stats logic behind FeatureFlags.PLUGGABLE_CACHE (#13238)
• [Tiered Caching] Add a dynamic setting to disable/enable disk cache. (#13373)
• Batch mode for async fetching shard information in GatewayAllocator for unassigned shards (#8746)
• [Remote Store] Add settings for remote path type and hash algorithm (#13225)
• [Remote Store] Upload remote paths during remote enabled index creation (#13386)
• [Search Pipeline] Handle default pipeline for multiple indices (#13276)
• [Batch Ingestion] Add batch_size to _bulk API. (#12457)
• [Remote Store] Add capability of doing refresh as determined by the translog (#12992)
• Support multi ranges traversal when doing date histogram rewrite optimization. (#13317)

Dependencies

• Bump org.apache.commons:commons-configuration2 from 2.10.0 to 2.10.1 (#12896)
• Bump asm from 9.6 to 9.7 (#12908)
• Bump net.minidev:json-smart from 2.5.0 to 2.5.1 (#12893, #13117)
• Bump netty from 4.1.107.Final to 4.1.109.Final (#12924, #13233)
• Bump commons-io:commons-io from 2.15.1 to 2.16.0 (#12996, #12998, #12999)
• Bump org.apache.commons:commons-compress from 1.24.0 to 1.26.1 (#12627)
• Bump org.apache.commons:commonscodec from 1.15 to 1.16.1 (#12627)
• Bump org.apache.commons:commonslang from 3.13.0 to 3.14.0 (#12627)
• Bump Apache Tika from 2.6.0 to 2.9.2 (#12627)
• Bump com.gradle.enterprise from 3.16.2 to 3.17.2 (#13116, #13191, #13327)
• Bump gradle/wrapper-validation-action from 2 to 3 (#13192)
• Bump joda from 2.12.2 to 2.12.7 (#13193)
• Bump bouncycastle from 1.77 to 1.78 (#13243)
• Update google dependencies in repository-gcs and discovery-gce (#13213)
• Bump com.google.apis:google-api-services-compute from v1-rev235-1.25.0 to v1-rev20240407-2.0.0 (#13333)
• Bump com.github.spullara.mustache.java:compiler from 0.9.10 to 0.9.11 (#13329)
• Bump commons-cli:commons-cli from 1.6.0 to 1.7.0 (#13331)
• Bump jakarta.enterprise:jakarta.enterprise.cdi-api from 4.0.1 to 4.1.0 (#13328)
• Bump com.google.api.grpc:proto-google-iam-v1 from 0.12.0 to 1.33.0 (#13332)
• Bump com.squareup.okio:okio from 3.8.0 to 3.9.0 (#12997)
• OpenJDK Update (April 2024 Patch releases), update to Eclipse Temurin 21.0.3+9 (#13389)
• Bump org.bouncycastle:bc-fips from 1.0.2.4 to 1.0.2.5 (#13446)
• Bump lycheeverse/lychee-action from 1.9.3 to 1.10.0 (#13447)
• Bump com.netflix.nebula.ospackage-base from 11.8.1 to 11.9.0 (#13440)

Changed

• [BWC and API enforcement] Enforcing the presence of API annotations at build time (#12872)
• Improve built-in secure transports support (#12907)
• Update links to documentation in rest-api-spec (#13043)
• Ignoring unavailable shards during search request execution with ignore_available parameter (#13298)
• Refactoring globMatch using simpleMatchWithNormalizedStrings from Regex (#13104)
• [BWC and API enforcement] Reconsider the breaking changes check policy to detect breaking changes against released versions (#13292)
• Switch to macos-13 runner for precommit and assemble github actions due to macos-latest is now arm64 (#13412)
• [Revert] Prevent unnecessary fetch sub phase processor initialization during fetch phase execution (#12503)

Fixed

• Fix bulk API ignores ingest pipeline for upsert (#12883)
• Fix issue with feature flags where default value may not be honored (#12849)
• Fix UOE While building Exists query for nested search_as_y...

1.3.16

2024-04-18 Version 1.3.16 Release Notes

Upgrades

• Bump netty from 4.1.107.Final to 4.1.109.Final (#12924, #13233)

2.13.0

2024-03-21 Version 2.13.0 Release Notes

[2.13.0]

Added

• [Tiered caching] Introducing cache plugins and exposing Ehcache as one of the pluggable disk cache option (#11874)
• Add support for dependencies in plugin descriptor properties with semver range (#11441)
• Add community_id ingest processor (#12121)
• Introduce query level setting index.query.max_nested_depth limiting nested queries (#3268
• Add toString methods to MultiSearchRequest, MultiGetRequest and CreateIndexRequest (#12163)
• Fix error in RemoteSegmentStoreDirectory when debug logging is enabled (#12328)
• Support for returning scores in matched queries (#11626)
• Add shard id property to SearchLookup for use in field types provided by plugins (#1063)
• [Tiered caching] Make IndicesRequestCache implementation configurable [EXPERIMENTAL] (#12533)
• Force merge API supports performing on primary shards only (#11269)
• Add kuromoji_completion analyzer and filter (#4835)
• [Admission Control] Integrate IO Usage Tracker to the Resource Usage Collector Service and Emit IO Usage Stats (#11880)
• The org.opensearch.bootstrap.Security should support codebase for JAR files with classifiers (#12586)
• Remote reindex: Add support for configurable retry mechanism (#12561)
• Tracing for deep search path (#12103)
• [Metrics Framework] Adds support for asynchronous gauge metric type. (#12642)
• [Tiered caching] Add Stale keys Management and CacheCleaner to IndicesRequestCache (#12625)
• Make search query counters dynamic to support all query types (#12601)
• [Tiered caching] Add policies controlling which values can enter pluggable caches [EXPERIMENTAL] (#12542)
• [Tiered caching] Add serializer integration to allow ehcache disk cache to use non-primitive values (#12709)
• [Admission Control] Integrated IO Based AdmissionController to AdmissionControl Framework (#12583)
• Add Remote Store Migration Experimental flag and allow mixed mode clusters under same (#11986)
• Built-in secure transports support (#12435)
• Lightweight Transport action to verify local term before fetching cluster-state from remote (#12252)
• Integrate with admission controller for cluster-manager Read API. (#12496)
• Introduce a new setting index.check_pending_flush.enabled to expose the ability to disable the check for pending flushes by write threads (#12710)

Dependencies

• Bump com.squareup.okio:okio from 3.7.0 to 3.8.0 (#12290)
• Bump org.bouncycastle:bcprov-jdk15to18 to org.bouncycastle:bcprov-jdk18on version 1.77 (#12326)
• Bump org.bouncycastle:bcmail-jdk15to18 to org.bouncycastle:bcmail-jdk18on version 1.77 (#12326)
• Bump org.bouncycastle:bcpkix-jdk15to18 to org.bouncycastle:bcpkix-jdk18on version 1.77 (#12326)
• Bump gradle/wrapper-validation-action from 1 to 2 (#12367)
• Bump netty from 4.1.106.Final to 4.1.107.Final (#12372)
• Bump opentelemetry from 1.34.1 to 1.36.0 (#12388, #12618)
• Bump Apache Lucene from 9.9.2 to 9.10.0 (#12392)
• Bump org.apache.logging.log4j:log4j-core from 2.22.1 to 2.23.1 (#12464, #12587)
• Bump antlr4 from 4.11.1 to 4.13.1 (#12445)
• Bump com.netflix.nebula.ospackage-base from 11.8.0 to 11.8.1 (#12461)
• Bump peter-evans/create-or-update-comment from 3 to 4 (#12462)
• Bump lycheeverse/lychee-action from 1.9.1 to 1.9.3 (#12521)
• Bump com.azure:azure-core from 1.39.0 to 1.47.0 (#12520)
• Bump ch.qos.logback:logback-core from 1.2.13 to 1.5.3 (#12519)
• Bump codecov/codecov-action from 3 to 4 (#12585)
• Bump org.apache.zookeeper:zookeeper from 3.9.1 to 3.9.2 (#12580)
• Bump org.codehaus.woodstox:stax2-api from 4.2.1 to 4.2.2 (#12579)
• Bump Jackson version from 2.16.1 to 2.17.0 (#12611, #12662)
• Bump reactor-netty from 1.1.15 to 1.1.17 (#12633)
• Bump reactor from 3.5.14 to 3.5.15 (#12633)
• Bump aws-sdk-java from 2.20.55 to 2.20.86 (#12251)

Changed

• Allow composite aggregation to run under a parent filter aggregation (#11499)
• Quickly compute terms aggregations when the top-level query is functionally match-all for a segment (#11643)
• Mark fuzzy filter GA and remove experimental setting (12631)
• Keep the election scheduler open until cluster state has been applied (#11699)

Fixed

• [Revert] [Bug] Check phase name before SearchRequestOperationsListener onPhaseStart (#12035)
• Add support of special WrappingSearchAsyncActionPhase so the onPhaseStart() will always be followed by onPhaseEnd() within AbstractSearchAsyncAction (#12293)
• Add a system property to configure YamlParser codepoint limits (#12298)
• Prevent read beyond slice boundary in ByteArrayIndexInput (#10481)
• Fix the "highlight.max_analyzer_offset" request parameter with "plain" highlighter (#10919)
• Prevent unnecessary fetch sub phase processor initialization during fetch phase execution (#12503)
• Fix terms query on float field when doc_values are turned off by reverting back to FloatPoint from FloatField (#12499)
• Fix get task API does not refresh resource stats (#11531)
• Fix for deserilization bug in weighted round-robin metadata (#11679)
• onShardResult and onShardFailure are executed on one shard causes opensearch jvm crashed (#12158)
• Avoid overflow when sorting missing last on epoch_millis datetime field (#12676)