Skip to content

Commit

Permalink
Rebase throttling branch with main (#4650)
Browse files Browse the repository at this point in the history
* [BUG] Running "opensearch-service.bat start" and "opensearch-service.bat manager" (#4289)

* [BUG] Update opensearch-service-x64.exe parameters to //ES for Execute Service. Update opensearch-service-mgr.exe parameters to //ES for Edit Service. Add code comments for the Apache Commons Daemon.

Signed-off-by: Alex Burck <[email protected]>

* update changelog with pull request link

Signed-off-by: Alex Burck <[email protected]>

Signed-off-by: Alex Burck <[email protected]>

* Removing dead code in RecoveryTarget. (#4278)

* Removing dead code in RecoveryTarget.

This code in RecoveryTarget is not invoked, all of these methods are implemented by the parent ReplicationTarget with the same behavior.

Signed-off-by: Marc Handalian <[email protected]>

* PR Comments.

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

* Update the head ref to changelog verifier (#4296)

* Update changelog contribution guide

Signed-off-by: Kunal Kotwani <[email protected]>

* Fix reference to pull request

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Add 2.x version to CHANGELOG (#4297)

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Do not fail replica shard due to primary closure (#4133)

This commit prevents a replica shard from being failed in the case that
a replication action to a replica is canceled due to the primary shard
being closed.

Signed-off-by: Andrew Ross <[email protected]>

Signed-off-by: Andrew Ross <[email protected]>

* Some dependency updates (#4308)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses (#4307)

* Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses

Signed-off-by: Tianli Feng <[email protected]>

* Add changelog

Signed-off-by: Tianli Feng <[email protected]>

Signed-off-by: Tianli Feng <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>

* [BUG] Create logs directory before running OpenSearch on Windows (#4305)

* [BUG] Create logs directory before running OpenSearch on Windows

Signed-off-by: Alex Burck <[email protected]>

* update changlog pr link

Signed-off-by: Alex Burck <[email protected]>

Signed-off-by: Alex Burck <[email protected]>

* Use RemoteSegmentStoreDirectory instead of RemoteDirectory (#4240)

* Use RemoteSegmentStoreDirectory instead of RemoteDirectory

Signed-off-by: Sachin Kale <[email protected]>

* ZIP publication groupId value is configurable (#4156)

When publishing Zip POM the groupId value was hard-coded to `org.opensearch.plugin` value which worked fine for existing core plugins but is not convenient for other plugins (such as community plugins maintained in independent repositories).

This PR changes the sources of the ZIP publishing groupId value.

Specifically, there are two ways to set the value:
1) It is automatically inherited from the Gradle "project.group"
2) It can be manually specified in the ZIP publication POM object

This PR also brings a major rework of tests in PublishTests class. Individual testing scenarios are driven by "real" gradle building scripts (utilizing `java-gradle-plugin` gradle plugin).

Closes #3692

Signed-off-by: Lukáš Vlček <[email protected]>

Signed-off-by: Lukáš Vlček <[email protected]>

* [Segment Replication] Add timeout on Mockito.verify to reduce flakyness in testReplicationOnDone test (#4314)

* [Segment Replication] testReplicationOnDone Add timeout to allow time for verify call

Signed-off-by: Suraj Singh <[email protected]>

* Update changelog

Signed-off-by: Suraj Singh <[email protected]>

* Add change log entry

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* Adding @dreamer-89 to Opensearch maintainers. (#4342)

Signed-off-by: Kartik Ganesh <[email protected]>

Signed-off-by: Kartik Ganesh <[email protected]>

* [CVE] Update snakeyaml dependency (#4341)

The package `org.yaml:snakeyaml` before version 1.31 are vulnerable to Denial of Service (DoS) due missing to nested depth limitation for collections.

Details at https://nvd.nist.gov/vuln/detail/CVE-2022-25857

Signed-off-by: Rabi Panda <[email protected]>

* Fixed commit workflow for dependabot PR helper (#4331)

* Fixed label for dependabot PR helper

Signed-off-by: Kunal Kotwani <[email protected]>

* Update autocommit workflow for dependabot changelog

Signed-off-by: Kunal Kotwani <[email protected]>

* Add version config for dependabot changelog helper

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Add release notes for patch release 1.3.5 (#4343)


Signed-off-by: Rabi Panda <[email protected]>

* Add release notes for patch release 2.2.1 (#4344)


Signed-off-by: Rabi Panda <[email protected]>

* Add label configuration for dependabot PRs (#4348)

* Support for HTTP/2 (server-side)  (#3847)

* Support for HTTP/2 (server-side)

Signed-off-by: Andriy Redko <[email protected]>

* Addressing code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Added HTTP/1.1 channel configuration

Signed-off-by: Andriy Redko <[email protected]>

* Addressing code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Update pul request URL in CHANGELOG.md

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Fix token usage for changelog helper (#4351)

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Revert "Fix token usage for changelog helper (#4351)" (#4361)

This reverts commit c28221e.

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Segment Replication - Implement segment replication event cancellation. (#4225)

* Segment Replication.  Fix Cancellation of replication events.

This PR updates segment replication paths to correctly cancel replication events on the primary and replica.
In the source service, any ongoing event for a primary that is sending to a replica that shuts down or is promoted as a new primary are cancelled.
In the target service, any ongoing event for a replica that is promoted as a new primary or is fetching from a primary that shuts down.
It wires up SegmentReplicationSourceService as an IndexEventListener so that it can respond to events and cancel any ongoing transfer state.
This change also includes some test cleanup for segment replication to rely on actual components over mocks.

Signed-off-by: Marc Handalian <[email protected]>

Fix to not start/stop SegmentReplicationSourceService as a lifecycle component with feature flag off.

Signed-off-by: Marc Handalian <[email protected]>

Update logic to properly mark SegmentReplicationTarget as cancelled when cancel initiated by primary.

Signed-off-by: Marc Handalian <[email protected]>

Minor updates from self review.

Signed-off-by: Marc Handalian <[email protected]>

* Add missing changelog entry.

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

* Bug fixes for dependabot changelog verifier (#4364)

* Fix token usage for changelog helper

Signed-off-by: Kunal Kotwani <[email protected]>

* Add conditional check for dependabot steps

Signed-off-by: Kunal Kotwani <[email protected]>

* Add dependency section

Signed-off-by: Kunal Kotwani <[email protected]>

* Bug fixes for dependabot changelog verifier

Signed-off-by: Kunal Kotwani <[email protected]>

* Update the changelog

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Add changes for Create PIT and Delete PIT rest layer and rest high level client (#4064)

* Create and delete PIT search rest layer changes

Signed-off-by: Bharathwaj G <[email protected]>

* Bump com.diffplug.spotless from 6.9.1 to 6.10.0 (#4319)

* Update to Netty 4.1.80.Final (#4359)

Signed-off-by: Andriy Redko <[email protected]>

* Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment (#4354)

* Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment

Bumps xmlbeans from 5.1.0 to 5.1.1.

---
updated-dependencies:
- dependency-name: org.apache.xmlbeans:xmlbeans
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Fix randomized test failure NRTReplicationEngineTests.testUpdateSegments (#4352)

Overload `generateHistoryOnReplica` to be able to generate only a specific `Engine.Operation.TYPE` operations as required by the `testUpdateSegments` test

Signed-off-by: Rabi Panda <[email protected]>

Signed-off-by: Rabi Panda <[email protected]>

* [AUTO] [main] Added bwc version 2.2.2. (#4383)

* Added bwc version 2.2.2

* Add changelog

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Kunal Kotwani <[email protected]>

* [Segment Replication] Bump segment infos counter before commit during replica promotion (#4365)

* [Segment Replication] Bump segment infos counter before commit during replica promotion

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica (#4363)

* [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments 2

Signed-off-by: Suraj Singh <[email protected]>

* Test failures

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* Adding support for labels on version bump PRs, skip label support for… (#4391)

* Adding support for labels on version bump PRs, skip label support for changelog verifier

Signed-off-by: Kunal Kotwani <[email protected]>

* Add changelog

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* [Segment Replication] Extend FileChunkWriter to allow cancel on transport client (#4386)

* [Segment Replication] Extend FileChunkWriter to allow cancel on retryable transport client

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments

Signed-off-by: Suraj Singh <[email protected]>

* Integration test

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* Added RestLayer Changes for PIT stats (#4217)

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Modified cat shards test for pit stats (#4408)

Signed-off-by: Ajay Kumar Movva <[email protected]>

* [Remote Store] Add index specific setting for remote repository (#4253)

* Add index specific setting for remote repository
* Fix for failover incremental uploads

Signed-off-by: Sachin Kale <[email protected]>

* [Semgnet Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test (#4414)

* [Semgnet Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry

Signed-off-by: Suraj Singh <[email protected]>

* Update changelog entry

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards. (#4366)

* Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards.

This change fixes the errors that occur when computing metadata snapshots on primary shards from the latest in-memory SegmentInfos.  The error occurs when a segments_N file that is referenced by the in-memory infos is deleted as part of a concurrent commit.  The segments themselves are incref'd by IndexWriter.incRefDeleter but the commit file (Segments_N) is not.  This change resolves this by ignoring the segments_N file when computing metadata for CopyState and only sending incref'd segment files to replicas.

Signed-off-by: Marc Handalian <[email protected]>

* Fix spotless.

Signed-off-by: Marc Handalian <[email protected]>

* Update StoreTests.testCleanupAndPreserveLatestCommitPoint to assert additional segments are deleted.

Signed-off-by: Marc Handalian <[email protected]>

* Rename snapshot to metadataMap in CheckpointInfoResponse.

Signed-off-by: Marc Handalian <[email protected]>

* Refactor segmentReplicationDiff method to compute off two maps instead of MetadataSnapshots.

Signed-off-by: Marc Handalian <[email protected]>

* Fix spotless.

Signed-off-by: Marc Handalian <[email protected]>

* Revert catchall in SegmentReplicationSourceService.

Signed-off-by: Marc Handalian <[email protected]>

* Revert log lvl change.

Signed-off-by: Marc Handalian <[email protected]>

* Fix SegmentReplicationTargetTests

Signed-off-by: Marc Handalian <[email protected]>

* Cleanup unused logger.

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>
Co-authored-by: Suraj Singh <[email protected]>

* Bump org.gradle.test-retry from 1.4.0 to 1.4.1 (#4411)

* Bump org.gradle.test-retry from 1.4.0 to 1.4.1

Bumps org.gradle.test-retry from 1.4.0 to 1.4.1.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Revert to Netty 4.1.79.Final (#4428)

Signed-off-by: Craig Perkins <[email protected]>

Signed-off-by: Craig Perkins <[email protected]>

* [Segment Replication] Fix timeout issue by calculating time needed to process getSegmentFiles. (#4426)

* Fix timeout issue by calculating time needed to process getSegmentFiles.

Signed-off-by: Rishikesh1159 <[email protected]>

* Formatting sizeOfSegmentFiles for time calculation.

Signed-off-by: Rishikesh1159 <[email protected]>

* Addressing comments and applying spotless check.

Signed-off-by: Rishikesh1159 <[email protected]>

Signed-off-by: Rishikesh1159 <[email protected]>

* [Bug]: gradle check failing with java heap OutOfMemoryError (#4328)

* [Bug]: gradle check failing with java heap OutOfMemoryError

Signed-off-by: Andriy Redko <[email protected]>

* Fork JavaCompile task

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Bump azure-core-http-netty from 1.12.0 to 1.12.4 in /plugins/repository-azure (#4160)

* Bump azure-core-http-netty in /plugins/repository-azure

Bumps [azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.12.0 to 1.12.4.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-core_1.12.0...azure-core-http-netty_1.12.4)

---
updated-dependencies:
- dependency-name: com.azure:azure-core-http-netty
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Added missing forbidden classes

Signed-off-by: Owais Kazi <[email protected]>

* update version of azure-storage-common and azure-core-http-netty

Signed-off-by: Xue Zhou <[email protected]>

* update version of azure-storage-common and azure-core-http-netty

Signed-off-by: Xue Zhou <[email protected]>

* adding changelog information

Signed-off-by: Xue Zhou <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Owais Kazi <[email protected]>
Signed-off-by: Xue Zhou <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owais Kazi <[email protected]>
Co-authored-by: Xue Zhou <[email protected]>

* Bugfix: Allow opensearch.bat file and opensearch-env.bat files to run when install path includes a space. (#4362)

* Bugfix: Prevent escaping of in `else-if` statement by setting variable without double quotes.

Signed-off-by: Mike Wilkinson <[email protected]>

* Add changelog entry for fix

Signed-off-by: Mike Wilkinson <[email protected]>

* Escape double quotes for environment variables set by `opensearch-env.bat`. Explicitly apply quotes where those environment variables are invoked.

Signed-off-by: Mike Wilkinson <[email protected]>

Signed-off-by: Mike Wilkinson <[email protected]>
Co-authored-by: Mike Wilkinson <[email protected]>

* [Segment Replication] - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. (#4402)

* Segment Replication - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards.

This change updates replicas to commit SegmentInfos before the shard is closed, on receiving a new commit point from a primary, and when a new primary is detected. This change also makes the public commitSegmentInfos on NRTEngine obsolete, refactoring IndexShard to simply call reset on the engine.

Signed-off-by: Marc Handalian <[email protected]>

* Remove noise & extra log statement.

Signed-off-by: Marc Handalian <[email protected]>

* PR feedback.

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

* Bump reactor-netty-core from 1.0.19 to 1.0.22 in /plugins/repository-azure (#4447)

* Bump reactor-netty-core in /plugins/repository-azure

Bumps [reactor-netty-core](https://github.com/reactor/reactor-netty) from 1.0.19 to 1.0.22.
- [Release notes](https://github.com/reactor/reactor-netty/releases)
- [Commits](reactor/reactor-netty@v1.0.19...v1.0.22)

---
updated-dependencies:
- dependency-name: io.projectreactor.netty:reactor-netty-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add bwcVersion 2.4.0 (#4455)

* Add bwcVersion 2.4.0

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* add support for s390x architecture (#4001)

* add s390x support in systemcallfilter

#4000
Signed-off-by: Andrija Pantovic <[email protected]>

* add gradle cfg for s390x

Signed-off-by: Andrija Pantovic <[email protected]>

* change assertion for architecture

Signed-off-by: Andrija Pantovic <[email protected]>

* change assertion for architecture

Signed-off-by: Andrija Pantovic <[email protected]>

* update changelog

Signed-off-by: Andrija Pantovic <[email protected]>

Signed-off-by: Andrija Pantovic <[email protected]>
Signed-off-by: Andrija Pantovic <[email protected]>

* 2.3.0 release notes (#4457) (#4464)

* 2.3.0 release notes

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* Add getHistoryOperationsFromTranslog method to fetch the history snapshot from translogs (#3948)

* Add getHistoryOperationsFromTranslog method to fetch the hostory snapshot from translogs

Signed-off-by: Ankit Kala <[email protected]>

* Fixed flaky test: ResourceAwareTasksTests.testTaskIdPersistsInThreadContext (#4484)

Signed-off-by: Ketan Verma <[email protected]>

* Getting security exception due to access denied 'java.lang.RuntimePermission' 'accessDeclaredMembers' when trying to get snapshot with S3 IRSA (#4469)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: Suraj Singh <[email protected]>

* [Javadoc] add missing javadocs for :distribution:tools modules (#4483)

* Add javadocs for distribution:tools:java-version-checker

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs for distribution:tools:keystore-cli

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs for distribution:tools:plugin-cli

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs for distribution:tools:launchers

Signed-off-by: Daniel Widdis <[email protected]>

* Change log

Signed-off-by: Daniel Widdis <[email protected]>

Signed-off-by: Daniel Widdis <[email protected]>

* Bump reactive-streams from 1.0.3 to 1.0.4 in /plugins/repository-azure (#4488)

* Bump reactive-streams from 1.0.3 to 1.0.4 in /plugins/repository-azure

Bumps [reactive-streams](https://github.com/reactive-streams/reactive-streams) from 1.0.3 to 1.0.4.
- [Release notes](https://github.com/reactive-streams/reactive-streams/releases)
- [Changelog](https://github.com/reactive-streams/reactive-streams-jvm/blob/master/RELEASE-NOTES.md)
- [Commits](reactive-streams/reactive-streams-jvm@v1.0.3...v1.0.4)

---
updated-dependencies:
- dependency-name: org.reactivestreams:reactive-streams
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Ignore all malformed objects when ignore_malformed is true (#4494)

Fixes a bug to not fail the entire document when "ignore_malformed" is set to true. Allowing the valid fields 
to be indexed and ignore only the malformed fields.

Signed-off-by: Hauck <[email protected]>

* [AUTO] [main] Added bwc version 2.3.1. (#4513)

* Added bwc version 2.3.1
* Add changelog of adding bwc version 2.3.1

Signed-off-by: Tianli Feng <[email protected]>

* [Segment Replication] Add snapshot and restore tests for segment replication feature (#3993)

* [Segment Replication] Add snapshots tests with segment replication enabled

Signed-off-by: Suraj Singh <[email protected]>

* Fix spotless failures

Signed-off-by: Suraj Singh <[email protected]>

* Add changelog entry, address review comments, add failover test

Signed-off-by: Suraj Singh <[email protected]>

* Fix spotless failures

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments 2

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* [Remote Store] Change behaviour in replica recovery for remote translog enabled indices (#4318)

Signed-off-by: Ashish Singh <[email protected]>

* Weighted round-robin scheduling policy for shard coordination traffic… (#4241)

* Weighted round-robin scheduling policy for shard coordination traffic routing

Signed-off-by: Anshu Agarwal <[email protected]>

* [Javadoc] add missing javadocs for :example-plugin modules (#4540)

* Add javadocs for example-plugins:custom-settings module

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs for example-plugins:custom-significance-heuristic

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs for example-plugins:custom-suggester

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs to example-plugins:painless-allowlist

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs to example-plugins:rescore

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs to example-plugins:rest-handler

Signed-off-by: Daniel Widdis <[email protected]>

* Add javadocs to example-plugins:script-expert-scoring

Signed-off-by: Daniel Widdis <[email protected]>

* Remove exclusions for module which aren't giving javadoc errors

Signed-off-by: Daniel Widdis <[email protected]>

* CI testing apparently needs to instantiate a no-op class

Signed-off-by: Daniel Widdis <[email protected]>

Signed-off-by: Daniel Widdis <[email protected]>

* Add GeoBounds aggregation on GeoShape field type.(#3980) (#4266)

Enables geo_bounds aggregation to work with geo_shape field types.

This enhancement includes:
    * Addition of Doc values on the GeoShape Field.
    * Addition of GeoShape ValueSource level code interfaces for accessing the DocValues.
    * Addition of Missing Value feature in the GeoShape Aggregations.

Signed-off-by: Navneet Verma <[email protected]>

* Updates jackson and snakeyml dependencies (#4556)

* Upgrade jackson and snakeyml dependencies

Signed-off-by: Owais Kazi <[email protected]>

* Updated changelog

Signed-off-by: Owais Kazi <[email protected]>

Signed-off-by: Owais Kazi <[email protected]>

* Bump com.diffplug.spotless from 6.10.0 to 6.11.0 (#4547)

* Bump com.diffplug.spotless from 6.10.0 to 6.11.0

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure (#4548)

* Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure

Bumps [reactor-core](https://github.com/reactor/reactor-core) from 3.4.18 to 3.4.23.
- [Release notes](https://github.com/reactor/reactor-core/releases)
- [Commits](reactor/reactor-core@v3.4.18...v3.4.23)

---
updated-dependencies:
- dependency-name: io.projectreactor:reactor-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment (#4550)

* Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment

Bumps jempbox from 1.8.16 to 1.8.17.

---
updated-dependencies:
- dependency-name: org.apache.pdfbox:jempbox
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump log4j-core from 2.18.0 to 2.19.0 in /buildSrc/src/testKit/thirdPartyAudit (#4549)

* Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit

Bumps log4j-core from 2.18.0 to 2.19.0.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Ignore lock file when testing cleanupAndPreserveLatestCommitPoint (#4544)

Signed-off-by: Daniel Widdis <[email protected]>

Signed-off-by: Daniel Widdis <[email protected]>

* Remove deprecated code to add node name into log pattern of log4j property file (#4568)

Signed-off-by: Tianli Feng <[email protected]>

* Unmute RelocationIT.testRelocationWhileIndexingRandom. (#4580)

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

* Add DecommissionService and helper to execute awareness attribute decommissioning (#4084)

* Add Executor to decommission node attribute
* Decommission service implementation with cluster metadata
* Master abdication changes to decommission local awareness leader
* Update join validator changes to validate decommissioned node join request

Signed-off-by: Rishab Nahata <[email protected]>

* Added rest layer changes for List all PITs and PIT segments (#4388)

* Changes for list all and pit segments

Signed-off-by: Bharathwaj G <[email protected]>

* Add PUT api to update shard routing weights (#4272)

* Add PUT api to update shard routing weights

Signed-off-by: Anshu Agarwal <[email protected]>

* Further simplification of the ZIP publication implementation (#4360)

A follow-up of PR #4156 that brings in further simplification of the ZIP publication implementation and new tests.

It is possible to remove some of the initialization code because the work is already handled by other library under the hood (most likely by NebulaPublishPlugin).

For instance the condition to test the `groupId` presence: `if (groupId == null)` was pointless. It was never `true`. If the `project.group` is not defined the publication task fails with an exception, if there is a custom `groupId` value setup in publication config then it overrides the `project.group` as well. Tests are provided to cover these use cases.

As for the new tests they cover the following cases:

- verify that the task "publishToMavenLocal" gets expected results. The inspiration for this comes from opensearch-project/opensearch-plugin-template-java#35

- verify that applying only the 'opensearch.pluginzip' is enough, because both the NebulaPublish and MavenPublishPlugin plugins are added explicitly and tasks are chained correctly

- verify that if the plugin is applied but no publication is defined then a message is printed for the user

Signed-off-by: Lukáš Vlček <[email protected]>

Signed-off-by: Lukáš Vlček <[email protected]>

* Add missing Javadoc tag descriptions in unit tests (#4629)

* Add missing Javadoc tag descriptions in unit tests

Using JDK14 (downloaded from https://jdk.java.net/archive/), I tried
running ./gradlew check, and received errors like:

server/src/test/java/org/opensearch/indices/replication/SegmentReplicationTargetTests.java:440:
warning: no description for @throws

The build then failed with "warnings found and -Werror specified".

I tried again with JDK17 and the build passed.

According to DEVELOPER_GUIDE.md, we should be able to build with JDK14,
I added the missing Javadoc tag descriptions.

Signed-off-by: Michael Froh <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Michael Froh <[email protected]>

Signed-off-by: Michael Froh <[email protected]>

* Bump reactor-netty-http from 1.0.18 to 1.0.23 in /plugins/repository-azure (#4594)

* Bump reactor-netty-http in /plugins/repository-azure

Bumps [reactor-netty-http](https://github.com/reactor/reactor-netty) from 1.0.18 to 1.0.23.
- [Release notes](https://github.com/reactor/reactor-netty/releases)
- [Commits](reactor/reactor-netty@v1.0.18...v1.0.23)

---
updated-dependencies:
- dependency-name: io.projectreactor.netty:reactor-netty-http
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic (#4592)

* Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic

Bumps [jettison](https://github.com/jettison-json/jettison) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/jettison-json/jettison/releases)
- [Commits](jettison-json/jettison@jettison-1.5.0...jettison-1.5.1)

---
updated-dependencies:
- dependency-name: org.codehaus.jettison:jettison
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Fix invalid search location of JDK for arm64 (#4613)

Related commit: 0e9f74e

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

* Remove unused object and import from TransportClusterAllocationExplainAction (#4639)

Signed-off-by: Kunal Kotwani <[email protected]>

Signed-off-by: Kunal Kotwani <[email protected]>

* Relax visibility of the HTTP_CHANNEL_KEY and HTTP_SERVER_CHANNEL_KEY to make it possible for the plugins to access associated Netty4HttpChannel / Netty4HttpServerChannel instance (#4638)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Alex Burck <[email protected]>
Signed-off-by: Marc Handalian <[email protected]>
Signed-off-by: Kunal Kotwani <[email protected]>
Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Tianli Feng <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Lukáš Vlček <[email protected]>
Signed-off-by: Suraj Singh <[email protected]>
Signed-off-by: Kartik Ganesh <[email protected]>
Signed-off-by: Rabi Panda <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Ajay Kumar Movva <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Rishikesh1159 <[email protected]>
Signed-off-by: Owais Kazi <[email protected]>
Signed-off-by: Xue Zhou <[email protected]>
Signed-off-by: Mike Wilkinson <[email protected]>
Signed-off-by: Andrija Pantovic <[email protected]>
Signed-off-by: Andrija Pantovic <[email protected]>
Signed-off-by: Ankit Kala <[email protected]>
Signed-off-by: Ketan Verma <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Hauck <[email protected]>
Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: Anshu Agarwal <[email protected]>
Signed-off-by: Navneet Verma <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Michael Froh <[email protected]>
Signed-off-by: Heemin Kim <[email protected]>
Signed-off-by: Dhwanil Patel <[email protected]>
Co-authored-by: Alex Burck <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Kunal Kotwani <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: Tianli Feng <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>
Co-authored-by: Lukáš Vlček <[email protected]>
Co-authored-by: Suraj Singh <[email protected]>
Co-authored-by: Kartik Ganesh <[email protected]>
Co-authored-by: Rabi Panda <[email protected]>
Co-authored-by: Bharathwaj G <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Movva Ajaykumar <[email protected]>
Co-authored-by: Craig Perkins <[email protected]>
Co-authored-by: Rishikesh Pasham <[email protected]>
Co-authored-by: Owais Kazi <[email protected]>
Co-authored-by: Xue Zhou <[email protected]>
Co-authored-by: mwilkinson-imo <[email protected]>
Co-authored-by: Mike Wilkinson <[email protected]>
Co-authored-by: Andrija Pantovic <[email protected]>
Co-authored-by: Ankit Kala <[email protected]>
Co-authored-by: Ketan Verma <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>
Co-authored-by: Hauck <[email protected]>
Co-authored-by: Ashish <[email protected]>
Co-authored-by: Anshu Agarwal <[email protected]>
Co-authored-by: Navneet Verma <[email protected]>
Co-authored-by: Rishab Nahata <[email protected]>
Co-authored-by: msfroh <[email protected]>
Co-authored-by: Heemin Kim <[email protected]>
  • Loading branch information
Show file tree
Hide file tree
Showing 341 changed files with 10,561 additions and 1,130 deletions.
2 changes: 2 additions & 0 deletions .ci/bwcVersions
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,5 @@ BWC_VERSION:
- "2.2.1"
- "2.2.2"
- "2.3.0"
- "2.3.1"
- "2.4.0"
2 changes: 2 additions & 0 deletions .github/workflows/changelog_verifier.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,5 @@ jobs:
ref: ${{ github.event.pull_request.head.sha }}

- uses: dangoslen/changelog-enforcer@v3
with:
skipLabels: "autocut"
8 changes: 7 additions & 1 deletion .github/workflows/version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
tags:
- '*.*.*'

jobs:
jobs:
build:
runs-on: ubuntu-latest
steps:
Expand Down Expand Up @@ -61,6 +61,8 @@ jobs:
commit-message: Incremented version to ${{ env.NEXT_VERSION }}
signoff: true
delete-branch: true
labels: |
autocut
title: '[AUTO] Incremented version to ${{ env.NEXT_VERSION }}.'
body: |
I've noticed that a new tag ${{ env.TAG }} was pushed, and incremented the version from ${{ env.CURRENT_VERSION }} to ${{ env.NEXT_VERSION }}.
Expand All @@ -86,6 +88,8 @@ jobs:
commit-message: Added bwc version ${{ env.NEXT_VERSION }}
signoff: true
delete-branch: true
labels: |
autocut
title: '[AUTO] [${{ env.BASE_X }}] Added bwc version ${{ env.NEXT_VERSION }}.'
body: |
I've noticed that a new tag ${{ env.TAG }} was pushed, and added a bwc version ${{ env.NEXT_VERSION }}.
Expand All @@ -111,6 +115,8 @@ jobs:
commit-message: Added bwc version ${{ env.NEXT_VERSION }}
signoff: true
delete-branch: true
labels: |
autocut
title: '[AUTO] [main] Added bwc version ${{ env.NEXT_VERSION }}.'
body: |
I've noticed that a new tag ${{ env.TAG }} was pushed, and added a bwc version ${{ env.NEXT_VERSION }}.
60 changes: 57 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,62 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)

## [Unreleased]
### Added
- Add support for s390x architecture ([#4001](https://github.com/opensearch-project/OpenSearch/pull/4001))
- Github workflow for changelog verification ([#4085](https://github.com/opensearch-project/OpenSearch/pull/4085))
- Point in time rest layer changes for create and delete PIT API ([#4064](https://github.com/opensearch-project/OpenSearch/pull/4064))
- Point in time rest layer changes for list PIT and PIT segments API ([#4388](https://github.com/opensearch-project/OpenSearch/pull/4388))
- Added @dreamer-89 as an Opensearch maintainer ([#4342](https://github.com/opensearch-project/OpenSearch/pull/4342))
- Added release notes for 1.3.5 ([#4343](https://github.com/opensearch-project/OpenSearch/pull/4343))
- Added release notes for 2.2.1 ([#4344](https://github.com/opensearch-project/OpenSearch/pull/4344))
- Label configuration for dependabot PRs ([#4348](https://github.com/opensearch-project/OpenSearch/pull/4348))
- Support for HTTP/2 (server-side) ([#3847](https://github.com/opensearch-project/OpenSearch/pull/3847))
- BWC version 2.2.2 ([#4383](https://github.com/opensearch-project/OpenSearch/pull/4383))
- Support for labels on version bump PRs, skip label support for changelog verifier ([#4391](https://github.com/opensearch-project/OpenSearch/pull/4391))
- Update previous release bwc version to 2.4.0 ([#4455](https://github.com/opensearch-project/OpenSearch/pull/4455))
- 2.3.0 release notes ([#4457](https://github.com/opensearch-project/OpenSearch/pull/4457))
- Added missing javadocs for `:distribution:tools` modules ([#4483](https://github.com/opensearch-project/OpenSearch/pull/4483))
- Add BWC version 2.3.1 ([#4513](https://github.com/opensearch-project/OpenSearch/pull/4513))
- [Segment Replication] Add snapshot and restore tests for segment replication feature ([#3993](https://github.com/opensearch-project/OpenSearch/pull/3993))
- Added missing javadocs for `:example-plugins` modules ([#4540](https://github.com/opensearch-project/OpenSearch/pull/4540))
- Add missing Javadoc tag descriptions in unit tests ([#4629](https://github.com/opensearch-project/OpenSearch/pull/4629))
### Dependencies
- Bumps `log4j-core` from 2.18.0 to 2.19.0
- Bumps `reactor-netty-http` from 1.0.18 to 1.0.23
- Bumps `jettison` from 1.5.0 to 1.5.1


### Dependencies
- Bumps `com.diffplug.spotless` from 6.9.1 to 6.10.0
- Bumps `xmlbeans` from 5.1.0 to 5.1.1
- Bumps `xmlbeans` from 5.1.0 to 5.1.1 ([#4354](https://github.com/opensearch-project/OpenSearch/pull/4354))
- Bumps `azure-core-http-netty` from 1.12.0 to 1.12.4 ([#4160](https://github.com/opensearch-project/OpenSearch/pull/4160))
- Bumps `azure-core` from 1.27.0 to 1.31.0 ([#4160](https://github.com/opensearch-project/OpenSearch/pull/4160))
- Bumps `azure-storage-common` from 12.16.0 to 12.18.0 ([#4160](https://github.com/opensearch-project/OpenSearch/pull/4160))
- Bumps `org.gradle.test-retry` from 1.4.0 to 1.4.1 ([#4411](https://github.com/opensearch-project/OpenSearch/pull/4411))
- Bumps `reactor-netty-core` from 1.0.19 to 1.0.22 ([#4447](https://github.com/opensearch-project/OpenSearch/pull/4447))
- Bumps `reactive-streams` from 1.0.3 to 1.0.4 ([#4488](https://github.com/opensearch-project/OpenSearch/pull/4488))
- Bumps `com.diffplug.spotless` from 6.10.0 to 6.11.0 ([#4547](https://github.com/opensearch-project/OpenSearch/pull/4547))
- Bumps `reactor-core` from 3.4.18 to 3.4.23 ([#4548](https://github.com/opensearch-project/OpenSearch/pull/4548))
- Bumps `jempbox` from 1.8.16 to 1.8.17 ([#4550](https://github.com/opensearch-project/OpenSearch/pull/4550))

### Changed
- Dependency updates (httpcore, mockito, slf4j, httpasyncclient, commons-codec) ([#4308](https://github.com/opensearch-project/OpenSearch/pull/4308))
- Use RemoteSegmentStoreDirectory instead of RemoteDirectory ([#4240](https://github.com/opensearch-project/OpenSearch/pull/4240))
- Plugin ZIP publication groupId value is configurable ([#4156](https://github.com/opensearch-project/OpenSearch/pull/4156))
- Update to Netty 4.1.80.Final ([#4359](https://github.com/opensearch-project/OpenSearch/pull/4359))
- Weighted round-robin scheduling policy for shard coordination traffic ([#4241](https://github.com/opensearch-project/OpenSearch/pull/4241))
- Add index specific setting for remote repository ([#4253](https://github.com/opensearch-project/OpenSearch/pull/4253))
- [Segment Replication] Update replicas to commit SegmentInfos instead of relying on SIS files from primary shards. ([#4402](https://github.com/opensearch-project/OpenSearch/pull/4402))
- [CCR] Add getHistoryOperationsFromTranslog method to fetch the history snapshot from translogs ([#3948](https://github.com/opensearch-project/OpenSearch/pull/3948))
- [Remote Store] Change behaviour in replica recovery for remote translog enabled indices ([#4318](https://github.com/opensearch-project/OpenSearch/pull/4318))
- PUT api for weighted shard routing ([#4272](https://github.com/opensearch-project/OpenSearch/pull/4272))
- Unmute test RelocationIT.testRelocationWhileIndexingRandom ([#4580](https://github.com/opensearch-project/OpenSearch/pull/4580))
- Add DecommissionService and helper to execute awareness attribute decommissioning ([#4084](https://github.com/opensearch-project/OpenSearch/pull/4084))
- Further simplification of the ZIP publication implementation ([#4360](https://github.com/opensearch-project/OpenSearch/pull/4360))
- Relax visibility of the HTTP_CHANNEL_KEY and HTTP_SERVER_CHANNEL_KEY to make it possible for the plugins to access associated Netty4HttpChannel / Netty4HttpServerChannel instance ([#4638](https://github.com/opensearch-project/OpenSearch/pull/4638))

### Deprecated

### Removed
- Remove deprecated code to add node name into log pattern of log4j property file ([#4568](https://github.com/opensearch-project/OpenSearch/pull/4568))
- Unused object and import within TransportClusterAllocationExplainAction ([#4639](https://github.com/opensearch-project/OpenSearch/pull/4639))

### Fixed
- `opensearch-service.bat start` and `opensearch-service.bat manager` failing to run ([#4289](https://github.com/opensearch-project/OpenSearch/pull/4289))
Expand All @@ -35,8 +69,23 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Add timeout on Mockito.verify to reduce flakyness in testReplicationOnDone test([#4314](https://github.com/opensearch-project/OpenSearch/pull/4314))
- Commit workflow for dependabot changelog helper ([#4331](https://github.com/opensearch-project/OpenSearch/pull/4331))
- Fixed cancellation of segment replication events ([#4225](https://github.com/opensearch-project/OpenSearch/pull/4225))
- [Segment Replication] Bump segment infos counter before commit during replica promotion ([#4365](https://github.com/opensearch-project/OpenSearch/pull/4365))
- Bugs for dependabot changelog verifier workflow ([#4364](https://github.com/opensearch-project/OpenSearch/pull/4364))
- Fix flaky random test `NRTReplicationEngineTests.testUpdateSegments` ([#4352](https://github.com/opensearch-project/OpenSearch/pull/4352))
- [Segment Replication] Extend FileChunkWriter to allow cancel on transport client ([#4386](https://github.com/opensearch-project/OpenSearch/pull/4386))
- [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica ([#4363](https://github.com/opensearch-project/OpenSearch/pull/4363))
- Fix NoSuchFileExceptions with segment replication when computing primary metadata snapshots ([#4366](https://github.com/opensearch-project/OpenSearch/pull/4366))
- [Segment Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test ([#4414](https://github.com/opensearch-project/OpenSearch/pull/4414))
- Fixed the `_cat/shards/10_basic.yml` test cases fix.
- [Segment Replication] Fix timeout issue by calculating time needed to process getSegmentFiles ([#4426](https://github.com/opensearch-project/OpenSearch/pull/4426))
- [Bug]: gradle check failing with java heap OutOfMemoryError ([#4328](https://github.com/opensearch-project/OpenSearch/))
- `opensearch.bat` fails to execute when install path includes spaces ([#4362](https://github.com/opensearch-project/OpenSearch/pull/4362))
- Getting security exception due to access denied 'java.lang.RuntimePermission' 'accessDeclaredMembers' when trying to get snapshot with S3 IRSA ([#4469](https://github.com/opensearch-project/OpenSearch/pull/4469))
- Fixed flaky test `ResourceAwareTasksTests.testTaskIdPersistsInThreadContext` ([#4484](https://github.com/opensearch-project/OpenSearch/pull/4484))
- Fixed the ignore_malformed setting to also ignore objects ([#4494](https://github.com/opensearch-project/OpenSearch/pull/4494))
- [Segment Replication] Ignore lock file when testing cleanupAndPreserveLatestCommitPoint ([#4544](https://github.com/opensearch-project/OpenSearch/pull/4544))
- Updated jackson to 2.13.4 and snakeyml to 1.32 ([#4556](https://github.com/opensearch-project/OpenSearch/pull/4556))
- [Bug]: Fixed invalid location of JDK dependency for arm64 architecture([#4613](https://github.com/opensearch-project/OpenSearch/pull/4613))

### Security
- CVE-2022-25857 org.yaml:snakeyaml DOS vulnerability ([#4341](https://github.com/opensearch-project/OpenSearch/pull/4341))
Expand All @@ -45,6 +94,11 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Added
- Github workflow for changelog verification ([#4085](https://github.com/opensearch-project/OpenSearch/pull/4085))
- Label configuration for dependabot PRs ([#4348](https://github.com/opensearch-project/OpenSearch/pull/4348))
- Added RestLayer Changes for PIT stats ([#4217](https://github.com/opensearch-project/OpenSearch/pull/4217))
- Added GeoBounds aggregation on GeoShape field type.([#4266](https://github.com/opensearch-project/OpenSearch/pull/4266))
- Addition of Doc values on the GeoShape Field
- Addition of GeoShape ValueSource level code interfaces for accessing the DocValues.
- Addition of Missing Value feature in the GeoShape Aggregations.

### Changed

Expand Down
14 changes: 12 additions & 2 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ plugins {
id 'lifecycle-base'
id 'opensearch.docker-support'
id 'opensearch.global-build-info'
id "com.diffplug.spotless" version "6.10.0" apply false
id "org.gradle.test-retry" version "1.4.0" apply false
id "com.diffplug.spotless" version "6.11.0" apply false
id "org.gradle.test-retry" version "1.4.1" apply false
id "test-report-aggregation"
id 'jacoco-report-aggregation'
}
Expand Down Expand Up @@ -264,6 +264,12 @@ tasks.register("branchConsistency") {
allprojects {
// configure compiler options
tasks.withType(JavaCompile).configureEach { JavaCompile compile ->
options.fork = true

configure(options.forkOptions) {
memoryMaximumSize = project.property('options.forkOptions.memoryMaximumSize')
}

// See please https://bugs.openjdk.java.net/browse/JDK-8209058
if (BuildParams.runtimeJavaVersion > JavaVersion.VERSION_11) {
compile.options.compilerArgs << '-Werror'
Expand Down Expand Up @@ -389,6 +395,10 @@ allprojects {
// the dependency is added.
gradle.projectsEvaluated {
allprojects {
project.tasks.withType(JavaForkOptions) {
maxHeapSize project.property('options.forkOptions.memoryMaximumSize')
}

if (project.path == ':test:framework') {
// :test:framework:test cannot run before and after :server:test
return
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@
public enum Architecture {

X64,
ARM64;
ARM64,
S390X;

public static Architecture current() {
final String architecture = System.getProperty("os.arch", "");
Expand All @@ -45,6 +46,8 @@ public static Architecture current() {
return X64;
case "aarch64":
return ARM64;
case "s390x":
return S390X;
default:
throw new IllegalArgumentException("can not determine architecture from [" + architecture + "]");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,9 @@ private String dependencyNotation(OpenSearchDistribution distribution) {
case X64:
classifier = ":" + distribution.getPlatform() + "-x64";
break;
case S390X:
classifier = ":" + distribution.getPlatform() + "-s390x";
break;
default:
throw new IllegalArgumentException("Unsupported architecture: " + distribution.getArchitecture());
}
Expand Down
4 changes: 2 additions & 2 deletions buildSrc/src/main/java/org/opensearch/gradle/Jdk.java
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@

public class Jdk implements Buildable, Iterable<File> {

private static final List<String> ALLOWED_ARCHITECTURES = Collections.unmodifiableList(Arrays.asList("aarch64", "x64"));
private static final List<String> ALLOWED_ARCHITECTURES = Collections.unmodifiableList(Arrays.asList("aarch64", "x64", "s390x"));
private static final List<String> ALLOWED_VENDORS = Collections.unmodifiableList(Arrays.asList("adoptium", "adoptopenjdk", "openjdk"));
private static final List<String> ALLOWED_PLATFORMS = Collections.unmodifiableList(
Arrays.asList("darwin", "freebsd", "linux", "mac", "windows")
Expand Down Expand Up @@ -128,7 +128,7 @@ public void setArchitecture(final String architecture) {
"unknown architecture [" + jdkArchitecture + "] for jdk [" + name + "], must be one of " + ALLOWED_ARCHITECTURES
);
}
this.architecture.set(architecture);
this.architecture.set(jdkArchitecture);
}

public String getBaseVersion() {
Expand Down
Loading

0 comments on commit 75783cb

Please sign in to comment.