Add PUT api to update shard routing weights #4272

anshu1106 · 2022-08-22T11:49:56Z

Description

This pull request adds PUT api to add/update weights for weighted routing policy.

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2022-08-22T11:58:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1979/
CommitID: 026d3ab

sachinpkale · 2022-08-25T08:22:26Z

precommit is failing. You can run ./gradlew precommit and check why it is failing. Mostly it is spotless check that fails.
build is failing. Please fix.
In PR description, update the checklist accordingly. Also, remove the placeholder text like [Describe what this change achieves]

.../org/opensearch/action/admin/cluster/shards/routing/wrr/put/ClusterPutWRRWeightsRequest.java

server/src/main/java/org/opensearch/client/ClusterAdminClient.java

server/src/test/java/org/opensearch/cluster/routing/OperationRoutingTests.java

...org/opensearch/action/admin/cluster/shards/routing/wrr/put/TransportPutWRRWeightsAction.java

server/src/main/java/org/opensearch/cluster/routing/IndexShardRoutingTable.java

server/src/main/java/org/opensearch/cluster/routing/WeightedRoundRobin.java

server/src/main/java/org/opensearch/cluster/metadata/WeightedRoundRobinMetadata.java

github-actions · 2022-08-29T12:58:05Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2186/
CommitID: 2f98d20a4fa2a881c4f91c184904602c118e1828

… routing Signed-off-by: Anshu Agarwal <[email protected]>

… shardRoutings Signed-off-by: Anshu Agarwal <[email protected]>

Remove Cluster Settings for WRR Add Code refactoring and Java docs Signed-off-by: Anshu Agarwal <[email protected]>

Signed-off-by: Anshu Agarwal <[email protected]>

Incorporate review comments and more UTs Signed-off-by: Anshu Agarwal <[email protected]>

Signed-off-by: Anshu Agarwal <[email protected]>

github-actions · 2022-09-01T12:21:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/2420/
CommitID: f07a9ad

codecov-commenter · 2022-09-01T12:23:06Z

Codecov Report

Merging #4272 (0a10275) into main (bb47419) will increase coverage by 0.01%.
The diff coverage is 61.90%.

@@             Coverage Diff              @@
##               main    #4272      +/-   ##
============================================
+ Coverage     70.55%   70.56%   +0.01%     
- Complexity    57432    57544     +112     
============================================
  Files          4635     4642       +7     
  Lines        276645   276792     +147     
  Branches      40489    40502      +13     
============================================
+ Hits         195180   195315     +135     
- Misses        65094    65121      +27     
+ Partials      16371    16356      -15

Impacted Files	Coverage Δ
...eighted/put/ClusterPutWeightedRoutingResponse.java	`0.00% <0.00%> (ø)`
.../org/opensearch/client/support/AbstractClient.java	`34.16% <0.00%> (-0.10%)`	⬇️
...eighted/put/TransportPutWeightedRoutingAction.java	`34.78% <34.78%> (ø)`
...weighted/put/ClusterPutWeightedRoutingRequest.java	`66.12% <66.12%> (ø)`
...search/cluster/routing/WeightedRoutingService.java	`70.96% <70.96%> (ø)`
...n/cluster/RestClusterPutWeightedRoutingAction.java	`76.92% <76.92%> (ø)`
.../main/java/org/opensearch/action/ActionModule.java	`98.18% <100.00%> (+0.01%)`	⬆️
.../weighted/put/ClusterPutWeightedRoutingAction.java	`100.00% <100.00%> (ø)`
...d/put/ClusterPutWeightedRoutingRequestBuilder.java	`100.00% <100.00%> (ø)`
.../src/main/java/org/opensearch/client/Requests.java	`46.00% <100.00%> (+1.10%)`	⬆️
... and 500 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: Anshu Agarwal <[email protected]>

github-actions · 2022-09-02T10:19:51Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2531/
CommitID: e7e592f

github-actions · 2022-09-02T10:37:09Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2532/
CommitID: e12d247

github-actions · 2022-09-02T10:49:56Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/2533/
CommitID: 3dd1d1a

Bukhtawar · 2022-09-26T09:26:24Z

server/src/main/java/org/opensearch/cluster/routing/WeightedRoutingService.java

+
+            @Override
+            public void onFailure(String source, Exception e) {
+                logger.warn(() -> new ParameterizedMessage("failed to update cluster state for weighted routing " + "weights [{}]", e));


nit: remove concatenation in the logging statement

Bukhtawar · 2022-09-26T09:26:43Z

server/src/main/java/org/opensearch/cluster/routing/WeightedRoutingService.java

+
+            @Override
+            public void clusterStateProcessed(String source, ClusterState oldState, ClusterState newState) {
+                logger.info("cluster weighted routing weights metadata change is processed by all the nodes");


nit: make this level debug

Bukhtawar · 2022-09-26T09:33:29Z

server/src/main/java/org/opensearch/cluster/routing/WeightedRoutingService.java

+/**
+ * * Service responsible for updating cluster state metadata with weighted routing weights
+ */
+public class WeightedRoutingService extends AbstractLifecycleComponent {


This strictly speaking isn't a LifecycleComponent, also where are we injecting this service?

Made this not to extend AbstractLifecycleComponent. This service is injected in transport action for put and delete weighted routing to update weights in cluster metadata

Bukhtawar · 2022-09-26T09:49:33Z

.../main/java/org/opensearch/rest/action/admin/cluster/RestClusterPutWeightedRoutingAction.java

+        if (request.hasParam("attribute")) {
+            attributeName = request.param("attribute");
+        }


I guess we are already adding an input validation, is the null check redundant? Gives an impression we could have null values at this point?

Bukhtawar · 2022-09-26T10:00:38Z

...arch/action/admin/cluster/shards/routing/weighted/put/TransportAddWeightedRoutingAction.java

+        if (!getAwarenessAttributes().contains(attributeName) || !attributeName.equalsIgnoreCase("zone")) {
+            ActionRequestValidationException validationException = null;
+
+            validationException = addValidationError(
+                String.format(Locale.ROOT, "invalid awareness attribute %s requested for updating weighted routing", attributeName),
+                validationException
+            );
+            throw validationException;
+        }
+    }


I don't think we can hardcode "zone". It should be a valid configured attribute name. Also to let you know there can be multiple such values supported

Checking just for attribute name in comma separated list of attributes

Bukhtawar · 2022-09-26T10:02:40Z

...arch/action/admin/cluster/shards/routing/weighted/put/TransportAddWeightedRoutingAction.java

+    @Override
+    protected ClusterBlockException checkBlock(ClusterPutWeightedRoutingRequest request, ClusterState state) {
+        return null;
+    }


Why override to null?
This should be

return state.blocks().globalBlockedException(ClusterBlockLevel.METADATA_WRITE);

Bukhtawar · 2022-09-26T10:03:39Z

server/src/main/java/org/opensearch/client/Requests.java

+     *
+     * @return update weight request
+     */
+    public static ClusterPutWeightedRoutingRequest putWeightedRoutingRequest() {


Should attribute be a passed here?

Bukhtawar · 2022-09-26T10:04:31Z

...arch/action/admin/cluster/shards/routing/weighted/put/ClusterPutWeightedRoutingResponse.java

+public class ClusterPutWeightedRoutingResponse extends AcknowledgedResponse implements ToXContentObject {
+    public ClusterPutWeightedRoutingResponse(boolean acknowledged) {
+        super(acknowledged);
+    }
+
+    public ClusterPutWeightedRoutingResponse(StreamInput in) throws IOException {
+        super(in);
+    }
+}


Whats the point extending?

Not needed to implement ToXContentObject since AcknowledgedResponse takes care of serialization/deserialization

Bukhtawar · 2022-09-26T10:05:49Z

...earch/action/admin/cluster/shards/routing/weighted/put/ClusterPutWeightedRoutingRequest.java

+            validationException = addValidationError(("Weight is not a number"), validationException);
+        }
+        if (countValueWithZeroWeights > 1) {
+            validationException = addValidationError(("More than one value has weight set as 0 "), validationException);


lets also add the count explicitly

Signed-off-by: Anshu Agarwal <[email protected]>

github-actions · 2022-09-26T12:24:53Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3398/
CommitID: 7763067

Signed-off-by: Anshu Agarwal <[email protected]>

github-actions · 2022-09-26T12:35:57Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3399/
CommitID: 3a0b0ff

github-actions · 2022-09-26T12:58:42Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3400/
CommitID: 703d918

github-actions · 2022-09-27T06:58:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3455/
CommitID: 329c6f2

anshu1106 · 2022-09-27T07:08:21Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/3455/

CommitID: 329c6f2

Seeing failure in PIT tests - org.opensearch.client.PitIT.testDeleteAllAndListAllPits

Re-triggered build

Signed-off-by: Anshu Agarwal <[email protected]>

github-actions · 2022-09-27T07:40:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3458/
CommitID: f3cf04d

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]> Signed-off-by: Vishal Sarda <[email protected]>

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

@dreamer-89

* [BUG] Running "opensearch-service.bat start" and "opensearch-service.bat manager" (#4289) * [BUG] Update opensearch-service-x64.exe parameters to //ES for Execute Service. Update opensearch-service-mgr.exe parameters to //ES for Edit Service. Add code comments for the Apache Commons Daemon. Signed-off-by: Alex Burck <[email protected]> * update changelog with pull request link Signed-off-by: Alex Burck <[email protected]> Signed-off-by: Alex Burck <[email protected]> * Removing dead code in RecoveryTarget. (#4278) * Removing dead code in RecoveryTarget. This code in RecoveryTarget is not invoked, all of these methods are implemented by the parent ReplicationTarget with the same behavior. Signed-off-by: Marc Handalian <[email protected]> * PR Comments. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Update the head ref to changelog verifier (#4296) * Update changelog contribution guide Signed-off-by: Kunal Kotwani <[email protected]> * Fix reference to pull request Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Add 2.x version to CHANGELOG (#4297) Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Do not fail replica shard due to primary closure (#4133) This commit prevents a replica shard from being failed in the case that a replication action to a replica is canceled due to the primary shard being closed. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]> * Some dependency updates (#4308) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses (#4307) * Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses Signed-off-by: Tianli Feng <[email protected]> * Add changelog Signed-off-by: Tianli Feng <[email protected]> Signed-off-by: Tianli Feng <[email protected]> Co-authored-by: Andrew Ross <[email protected]> * [BUG] Create logs directory before running OpenSearch on Windows (#4305) * [BUG] Create logs directory before running OpenSearch on Windows Signed-off-by: Alex Burck <[email protected]> * update changlog pr link Signed-off-by: Alex Burck <[email protected]> Signed-off-by: Alex Burck <[email protected]> * Use RemoteSegmentStoreDirectory instead of RemoteDirectory (#4240) * Use RemoteSegmentStoreDirectory instead of RemoteDirectory Signed-off-by: Sachin Kale <[email protected]> * ZIP publication groupId value is configurable (#4156) When publishing Zip POM the groupId value was hard-coded to `org.opensearch.plugin` value which worked fine for existing core plugins but is not convenient for other plugins (such as community plugins maintained in independent repositories). This PR changes the sources of the ZIP publishing groupId value. Specifically, there are two ways to set the value: 1) It is automatically inherited from the Gradle "project.group" 2) It can be manually specified in the ZIP publication POM object This PR also brings a major rework of tests in PublishTests class. Individual testing scenarios are driven by "real" gradle building scripts (utilizing `java-gradle-plugin` gradle plugin). Closes #3692 Signed-off-by: Lukáš Vlček <[email protected]> Signed-off-by: Lukáš Vlček <[email protected]> * [Segment Replication] Add timeout on Mockito.verify to reduce flakyness in testReplicationOnDone test (#4314) * [Segment Replication] testReplicationOnDone Add timeout to allow time for verify call Signed-off-by: Suraj Singh <[email protected]> * Update changelog Signed-off-by: Suraj Singh <[email protected]> * Add change log entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Adding @dreamer-89 to Opensearch maintainers. (#4342) Signed-off-by: Kartik Ganesh <[email protected]> Signed-off-by: Kartik Ganesh <[email protected]> * [CVE] Update snakeyaml dependency (#4341) The package `org.yaml:snakeyaml` before version 1.31 are vulnerable to Denial of Service (DoS) due missing to nested depth limitation for collections. Details at https://nvd.nist.gov/vuln/detail/CVE-2022-25857 Signed-off-by: Rabi Panda <[email protected]> * Fixed commit workflow for dependabot PR helper (#4331) * Fixed label for dependabot PR helper Signed-off-by: Kunal Kotwani <[email protected]> * Update autocommit workflow for dependabot changelog Signed-off-by: Kunal Kotwani <[email protected]> * Add version config for dependabot changelog helper Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Add release notes for patch release 1.3.5 (#4343) Signed-off-by: Rabi Panda <[email protected]> * Add release notes for patch release 2.2.1 (#4344) Signed-off-by: Rabi Panda <[email protected]> * Add label configuration for dependabot PRs (#4348) * Support for HTTP/2 (server-side) (#3847) * Support for HTTP/2 (server-side) Signed-off-by: Andriy Redko <[email protected]> * Addressing code review comments Signed-off-by: Andriy Redko <[email protected]> * Added HTTP/1.1 channel configuration Signed-off-by: Andriy Redko <[email protected]> * Addressing code review comments Signed-off-by: Andriy Redko <[email protected]> * Update pul request URL in CHANGELOG.md Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Fix token usage for changelog helper (#4351) Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Revert "Fix token usage for changelog helper (#4351)" (#4361) This reverts commit c28221e. Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Segment Replication - Implement segment replication event cancellation. (#4225) * Segment Replication. Fix Cancellation of replication events. This PR updates segment replication paths to correctly cancel replication events on the primary and replica. In the source service, any ongoing event for a primary that is sending to a replica that shuts down or is promoted as a new primary are cancelled. In the target service, any ongoing event for a replica that is promoted as a new primary or is fetching from a primary that shuts down. It wires up SegmentReplicationSourceService as an IndexEventListener so that it can respond to events and cancel any ongoing transfer state. This change also includes some test cleanup for segment replication to rely on actual components over mocks. Signed-off-by: Marc Handalian <[email protected]> Fix to not start/stop SegmentReplicationSourceService as a lifecycle component with feature flag off. Signed-off-by: Marc Handalian <[email protected]> Update logic to properly mark SegmentReplicationTarget as cancelled when cancel initiated by primary. Signed-off-by: Marc Handalian <[email protected]> Minor updates from self review. Signed-off-by: Marc Handalian <[email protected]> * Add missing changelog entry. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Bug fixes for dependabot changelog verifier (#4364) * Fix token usage for changelog helper Signed-off-by: Kunal Kotwani <[email protected]> * Add conditional check for dependabot steps Signed-off-by: Kunal Kotwani <[email protected]> * Add dependency section Signed-off-by: Kunal Kotwani <[email protected]> * Bug fixes for dependabot changelog verifier Signed-off-by: Kunal Kotwani <[email protected]> * Update the changelog Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Add changes for Create PIT and Delete PIT rest layer and rest high level client (#4064) * Create and delete PIT search rest layer changes Signed-off-by: Bharathwaj G <[email protected]> * Bump com.diffplug.spotless from 6.9.1 to 6.10.0 (#4319) * Update to Netty 4.1.80.Final (#4359) Signed-off-by: Andriy Redko <[email protected]> * Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment (#4354) * Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment Bumps xmlbeans from 5.1.0 to 5.1.1. --- updated-dependencies: - dependency-name: org.apache.xmlbeans:xmlbeans dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Fix randomized test failure NRTReplicationEngineTests.testUpdateSegments (#4352) Overload `generateHistoryOnReplica` to be able to generate only a specific `Engine.Operation.TYPE` operations as required by the `testUpdateSegments` test Signed-off-by: Rabi Panda <[email protected]> Signed-off-by: Rabi Panda <[email protected]> * [AUTO] [main] Added bwc version 2.2.2. (#4383) * Added bwc version 2.2.2 * Add changelog Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Kunal Kotwani <[email protected]> * [Segment Replication] Bump segment infos counter before commit during replica promotion (#4365) * [Segment Replication] Bump segment infos counter before commit during replica promotion Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica (#4363) * [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> * Address review comments Signed-off-by: Suraj Singh <[email protected]> * Address review comments 2 Signed-off-by: Suraj Singh <[email protected]> * Test failures Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Adding support for labels on version bump PRs, skip label support for… (#4391) * Adding support for labels on version bump PRs, skip label support for changelog verifier Signed-off-by: Kunal Kotwani <[email protected]> * Add changelog Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * [Segment Replication] Extend FileChunkWriter to allow cancel on transport client (#4386) * [Segment Replication] Extend FileChunkWriter to allow cancel on retryable transport client Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> * Address review comments Signed-off-by: Suraj Singh <[email protected]> * Integration test Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Added RestLayer Changes for PIT stats (#4217) Signed-off-by: Ajay Kumar Movva <[email protected]> * Modified cat shards test for pit stats (#4408) Signed-off-by: Ajay Kumar Movva <[email protected]> * [Remote Store] Add index specific setting for remote repository (#4253) * Add index specific setting for remote repository * Fix for failover incremental uploads Signed-off-by: Sachin Kale <[email protected]> * [Semgnet Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test (#4414) * [Semgnet Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> * Update changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards. (#4366) * Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards. This change fixes the errors that occur when computing metadata snapshots on primary shards from the latest in-memory SegmentInfos. The error occurs when a segments_N file that is referenced by the in-memory infos is deleted as part of a concurrent commit. The segments themselves are incref'd by IndexWriter.incRefDeleter but the commit file (Segments_N) is not. This change resolves this by ignoring the segments_N file when computing metadata for CopyState and only sending incref'd segment files to replicas. Signed-off-by: Marc Handalian <[email protected]> * Fix spotless. Signed-off-by: Marc Handalian <[email protected]> * Update StoreTests.testCleanupAndPreserveLatestCommitPoint to assert additional segments are deleted. Signed-off-by: Marc Handalian <[email protected]> * Rename snapshot to metadataMap in CheckpointInfoResponse. Signed-off-by: Marc Handalian <[email protected]> * Refactor segmentReplicationDiff method to compute off two maps instead of MetadataSnapshots. Signed-off-by: Marc Handalian <[email protected]> * Fix spotless. Signed-off-by: Marc Handalian <[email protected]> * Revert catchall in SegmentReplicationSourceService. Signed-off-by: Marc Handalian <[email protected]> * Revert log lvl change. Signed-off-by: Marc Handalian <[email protected]> * Fix SegmentReplicationTargetTests Signed-off-by: Marc Handalian <[email protected]> * Cleanup unused logger. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Co-authored-by: Suraj Singh <[email protected]> * Bump org.gradle.test-retry from 1.4.0 to 1.4.1 (#4411) * Bump org.gradle.test-retry from 1.4.0 to 1.4.1 Bumps org.gradle.test-retry from 1.4.0 to 1.4.1. --- updated-dependencies: - dependency-name: org.gradle.test-retry dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Revert to Netty 4.1.79.Final (#4428) Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Craig Perkins <[email protected]> * [Segment Replication] Fix timeout issue by calculating time needed to process getSegmentFiles. (#4426) * Fix timeout issue by calculating time needed to process getSegmentFiles. Signed-off-by: Rishikesh1159 <[email protected]> * Formatting sizeOfSegmentFiles for time calculation. Signed-off-by: Rishikesh1159 <[email protected]> * Addressing comments and applying spotless check. Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> * [Bug]: gradle check failing with java heap OutOfMemoryError (#4328) * [Bug]: gradle check failing with java heap OutOfMemoryError Signed-off-by: Andriy Redko <[email protected]> * Fork JavaCompile task Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Bump azure-core-http-netty from 1.12.0 to 1.12.4 in /plugins/repository-azure (#4160) * Bump azure-core-http-netty in /plugins/repository-azure Bumps [azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.12.0 to 1.12.4. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-core_1.12.0...azure-core-http-netty_1.12.4) --- updated-dependencies: - dependency-name: com.azure:azure-core-http-netty dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Added missing forbidden classes Signed-off-by: Owais Kazi <[email protected]> * update version of azure-storage-common and azure-core-http-netty Signed-off-by: Xue Zhou <[email protected]> * update version of azure-storage-common and azure-core-http-netty Signed-off-by: Xue Zhou <[email protected]> * adding changelog information Signed-off-by: Xue Zhou <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: Xue Zhou <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Xue Zhou <[email protected]> * Bugfix: Allow opensearch.bat file and opensearch-env.bat files to run when install path includes a space. (#4362) * Bugfix: Prevent escaping of in `else-if` statement by setting variable without double quotes. Signed-off-by: Mike Wilkinson <[email protected]> * Add changelog entry for fix Signed-off-by: Mike Wilkinson <[email protected]> * Escape double quotes for environment variables set by `opensearch-env.bat`. Explicitly apply quotes where those environment variables are invoked. Signed-off-by: Mike Wilkinson <[email protected]> Signed-off-by: Mike Wilkinson <[email protected]> Co-authored-by: Mike Wilkinson <[email protected]> * [Segment Replication] - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. (#4402) * Segment Replication - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. This change updates replicas to commit SegmentInfos before the shard is closed, on receiving a new commit point from a primary, and when a new primary is detected. This change also makes the public commitSegmentInfos on NRTEngine obsolete, refactoring IndexShard to simply call reset on the engine. Signed-off-by: Marc Handalian <[email protected]> * Remove noise & extra log statement. Signed-off-by: Marc Handalian <[email protected]> * PR feedback. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Bump reactor-netty-core from 1.0.19 to 1.0.22 in /plugins/repository-azure (#4447) * Bump reactor-netty-core in /plugins/repository-azure Bumps [reactor-netty-core](https://github.com/reactor/reactor-netty) from 1.0.19 to 1.0.22. - [Release notes](https://github.com/reactor/reactor-netty/releases) - [Commits](reactor/reactor-netty@v1.0.19...v1.0.22) --- updated-dependencies: - dependency-name: io.projectreactor.netty:reactor-netty-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add bwcVersion 2.4.0 (#4455) * Add bwcVersion 2.4.0 Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * add support for s390x architecture (#4001) * add s390x support in systemcallfilter #4000 Signed-off-by: Andrija Pantovic <[email protected]> * add gradle cfg for s390x Signed-off-by: Andrija Pantovic <[email protected]> * change assertion for architecture Signed-off-by: Andrija Pantovic <[email protected]> * change assertion for architecture Signed-off-by: Andrija Pantovic <[email protected]> * update changelog Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> * 2.3.0 release notes (#4457) (#4464) * 2.3.0 release notes Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Add getHistoryOperationsFromTranslog method to fetch the history snapshot from translogs (#3948) * Add getHistoryOperationsFromTranslog method to fetch the hostory snapshot from translogs Signed-off-by: Ankit Kala <[email protected]> * Fixed flaky test: ResourceAwareTasksTests.testTaskIdPersistsInThreadContext (#4484) Signed-off-by: Ketan Verma <[email protected]> * Getting security exception due to access denied 'java.lang.RuntimePermission' 'accessDeclaredMembers' when trying to get snapshot with S3 IRSA (#4469) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Co-authored-by: Suraj Singh <[email protected]> * [Javadoc] add missing javadocs for :distribution:tools modules (#4483) * Add javadocs for distribution:tools:java-version-checker Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for distribution:tools:keystore-cli Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for distribution:tools:plugin-cli Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for distribution:tools:launchers Signed-off-by: Daniel Widdis <[email protected]> * Change log Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Bump reactive-streams from 1.0.3 to 1.0.4 in /plugins/repository-azure (#4488) * Bump reactive-streams from 1.0.3 to 1.0.4 in /plugins/repository-azure Bumps [reactive-streams](https://github.com/reactive-streams/reactive-streams) from 1.0.3 to 1.0.4. - [Release notes](https://github.com/reactive-streams/reactive-streams/releases) - [Changelog](https://github.com/reactive-streams/reactive-streams-jvm/blob/master/RELEASE-NOTES.md) - [Commits](reactive-streams/reactive-streams-jvm@v1.0.3...v1.0.4) --- updated-dependencies: - dependency-name: org.reactivestreams:reactive-streams dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Ignore all malformed objects when ignore_malformed is true (#4494) Fixes a bug to not fail the entire document when "ignore_malformed" is set to true. Allowing the valid fields to be indexed and ignore only the malformed fields. Signed-off-by: Hauck <[email protected]> * [AUTO] [main] Added bwc version 2.3.1. (#4513) * Added bwc version 2.3.1 * Add changelog of adding bwc version 2.3.1 Signed-off-by: Tianli Feng <[email protected]> * [Segment Replication] Add snapshot and restore tests for segment replication feature (#3993) * [Segment Replication] Add snapshots tests with segment replication enabled Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry, address review comments, add failover test Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Address review comments 2 Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * [Remote Store] Change behaviour in replica recovery for remote translog enabled indices (#4318) Signed-off-by: Ashish Singh <[email protected]> * Weighted round-robin scheduling policy for shard coordination traffic… (#4241) * Weighted round-robin scheduling policy for shard coordination traffic routing Signed-off-by: Anshu Agarwal <[email protected]> * [Javadoc] add missing javadocs for :example-plugin modules (#4540) * Add javadocs for example-plugins:custom-settings module Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for example-plugins:custom-significance-heuristic Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for example-plugins:custom-suggester Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:painless-allowlist Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:rescore Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:rest-handler Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:script-expert-scoring Signed-off-by: Daniel Widdis <[email protected]> * Remove exclusions for module which aren't giving javadoc errors Signed-off-by: Daniel Widdis <[email protected]> * CI testing apparently needs to instantiate a no-op class Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Add GeoBounds aggregation on GeoShape field type.(#3980) (#4266) Enables geo_bounds aggregation to work with geo_shape field types. This enhancement includes: * Addition of Doc values on the GeoShape Field. * Addition of GeoShape ValueSource level code interfaces for accessing the DocValues. * Addition of Missing Value feature in the GeoShape Aggregations. Signed-off-by: Navneet Verma <[email protected]> * Updates jackson and snakeyml dependencies (#4556) * Upgrade jackson and snakeyml dependencies Signed-off-by: Owais Kazi <[email protected]> * Updated changelog Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: Owais Kazi <[email protected]> * Bump com.diffplug.spotless from 6.10.0 to 6.11.0 (#4547) * Bump com.diffplug.spotless from 6.10.0 to 6.11.0 --- updated-dependencies: - dependency-name: com.diffplug.spotless dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure (#4548) * Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure Bumps [reactor-core](https://github.com/reactor/reactor-core) from 3.4.18 to 3.4.23. - [Release notes](https://github.com/reactor/reactor-core/releases) - [Commits](reactor/reactor-core@v3.4.18...v3.4.23) --- updated-dependencies: - dependency-name: io.projectreactor:reactor-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment (#4550) * Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment Bumps jempbox from 1.8.16 to 1.8.17. --- updated-dependencies: - dependency-name: org.apache.pdfbox:jempbox dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Bump log4j-core from 2.18.0 to 2.19.0 in /buildSrc/src/testKit/thirdPartyAudit (#4549) * Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit Bumps log4j-core from 2.18.0 to 2.19.0. --- updated-dependencies: - dependency-name: org.apache.logging.log4j:log4j-core dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Ignore lock file when testing cleanupAndPreserveLatestCommitPoint (#4544) Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Remove deprecated code to add node name into log pattern of log4j property file (#4568) Signed-off-by: Tianli Feng <[email protected]> * Unmute RelocationIT.testRelocationWhileIndexingRandom. (#4580) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Add DecommissionService and helper to execute awareness attribute decommissioning (#4084) * Add Executor to decommission node attribute * Decommission service implementation with cluster metadata * Master abdication changes to decommission local awareness leader * Update join validator changes to validate decommissioned node join request Signed-off-by: Rishab Nahata <[email protected]> * Added rest layer changes for List all PITs and PIT segments (#4388) * Changes for list all and pit segments Signed-off-by: Bharathwaj G <[email protected]> * Add PUT api to update shard routing weights (#4272) * Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]> * Further simplification of the ZIP publication implementation (#4360) A follow-up of PR #4156 that brings in further simplification of the ZIP publication implementation and new tests. It is possible to remove some of the initialization code because the work is already handled by other library under the hood (most likely by NebulaPublishPlugin). For instance the condition to test the `groupId` presence: `if (groupId == null)` was pointless. It was never `true`. If the `project.group` is not defined the publication task fails with an exception, if there is a custom `groupId` value setup in publication config then it overrides the `project.group` as well. Tests are provided to cover these use cases. As for the new tests they cover the following cases: - verify that the task "publishToMavenLocal" gets expected results. The inspiration for this comes from opensearch-project/opensearch-plugin-template-java#35 - verify that applying only the 'opensearch.pluginzip' is enough, because both the NebulaPublish and MavenPublishPlugin plugins are added explicitly and tasks are chained correctly - verify that if the plugin is applied but no publication is defined then a message is printed for the user Signed-off-by: Lukáš Vlček <[email protected]> Signed-off-by: Lukáš Vlček <[email protected]> * Add missing Javadoc tag descriptions in unit tests (#4629) * Add missing Javadoc tag descriptions in unit tests Using JDK14 (downloaded from https://jdk.java.net/archive/), I tried running ./gradlew check, and received errors like: server/src/test/java/org/opensearch/indices/replication/SegmentReplicationTargetTests.java:440: warning: no description for @throws The build then failed with "warnings found and -Werror specified". I tried again with JDK17 and the build passed. According to DEVELOPER_GUIDE.md, we should be able to build with JDK14, I added the missing Javadoc tag descriptions. Signed-off-by: Michael Froh <[email protected]> * Update CHANGELOG.md Signed-off-by: Michael Froh <[email protected]> Signed-off-by: Michael Froh <[email protected]> * Bump reactor-netty-http from 1.0.18 to 1.0.23 in /plugins/repository-azure (#4594) * Bump reactor-netty-http in /plugins/repository-azure Bumps [reactor-netty-http](https://github.com/reactor/reactor-netty) from 1.0.18 to 1.0.23. - [Release notes](https://github.com/reactor/reactor-netty/releases) - [Commits](reactor/reactor-netty@v1.0.18...v1.0.23) --- updated-dependencies: - dependency-name: io.projectreactor.netty:reactor-netty-http dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic (#4592) * Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic Bumps [jettison](https://github.com/jettison-json/jettison) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/jettison-json/jettison/releases) - [Commits](jettison-json/jettison@jettison-1.5.0...jettison-1.5.1) --- updated-dependencies: - dependency-name: org.codehaus.jettison:jettison dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Fix invalid search location of JDK for arm64 (#4613) Related commit: 0e9f74e Signed-off-by: Heemin Kim <[email protected]> Signed-off-by: Heemin Kim <[email protected]> * Remove unused object and import from TransportClusterAllocationExplainAction (#4639) Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Relax visibility of the HTTP_CHANNEL_KEY and HTTP_SERVER_CHANNEL_KEY to make it possible for the plugins to access associated Netty4HttpChannel / Netty4HttpServerChannel instance (#4638) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Alex Burck <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Tianli Feng <[email protected]> Signed-off-by: Sachin Kale <[email protected]> Signed-off-by: Lukáš Vlček <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Kartik Ganesh <[email protected]> Signed-off-by: Rabi Panda <[email protected]> Signed-off-by: Bharathwaj G <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Ajay Kumar Movva <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: Xue Zhou <[email protected]> Signed-off-by: Mike Wilkinson <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Ankit Kala <[email protected]> Signed-off-by: Ketan Verma <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Hauck <[email protected]> Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Anshu Agarwal <[email protected]> Signed-off-by: Navneet Verma <[email protected]> Signed-off-by: Rishab Nahata <[email protected]> Signed-off-by: Michael Froh <[email protected]> Signed-off-by: Heemin Kim <[email protected]> Signed-off-by: Dhwanil Patel <[email protected]> Co-authored-by: Alex Burck <[email protected]> Co-authored-by: Marc Handalian <[email protected]> Co-authored-by: Kunal Kotwani <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: Tianli Feng <[email protected]> Co-authored-by: Sachin Kale <[email protected]> Co-authored-by: Lukáš Vlček <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Co-authored-by: Kartik Ganesh <[email protected]> Co-authored-by: Rabi Panda <[email protected]> Co-authored-by: Bharathwaj G <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Movva Ajaykumar <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Rishikesh Pasham <[email protected]> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Xue Zhou <[email protected]> Co-authored-by: mwilkinson-imo <[email protected]> Co-authored-by: Mike Wilkinson <[email protected]> Co-authored-by: Andrija Pantovic <[email protected]> Co-authored-by: Ankit Kala <[email protected]> Co-authored-by: Ketan Verma <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> Co-authored-by: Hauck <[email protected]> Co-authored-by: Ashish <[email protected]> Co-authored-by: Anshu Agarwal <[email protected]> Co-authored-by: Navneet Verma <[email protected]> Co-authored-by: Rishab Nahata <[email protected]> Co-authored-by: msfroh <[email protected]> Co-authored-by: Heemin Kim <[email protected]>

* [BUG] Running "opensearch-service.bat start" and "opensearch-service.bat manager" (#4289) * [BUG] Update opensearch-service-x64.exe parameters to //ES for Execute Service. Update opensearch-service-mgr.exe parameters to //ES for Edit Service. Add code comments for the Apache Commons Daemon. Signed-off-by: Alex Burck <[email protected]> * update changelog with pull request link Signed-off-by: Alex Burck <[email protected]> Signed-off-by: Alex Burck <[email protected]> * Removing dead code in RecoveryTarget. (#4278) * Removing dead code in RecoveryTarget. This code in RecoveryTarget is not invoked, all of these methods are implemented by the parent ReplicationTarget with the same behavior. Signed-off-by: Marc Handalian <[email protected]> * PR Comments. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Update the head ref to changelog verifier (#4296) * Update changelog contribution guide Signed-off-by: Kunal Kotwani <[email protected]> * Fix reference to pull request Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Add 2.x version to CHANGELOG (#4297) Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Do not fail replica shard due to primary closure (#4133) This commit prevents a replica shard from being failed in the case that a replication action to a replica is canceled due to the primary shard being closed. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]> * Some dependency updates (#4308) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses (#4307) * Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses Signed-off-by: Tianli Feng <[email protected]> * Add changelog Signed-off-by: Tianli Feng <[email protected]> Signed-off-by: Tianli Feng <[email protected]> Co-authored-by: Andrew Ross <[email protected]> * [BUG] Create logs directory before running OpenSearch on Windows (#4305) * [BUG] Create logs directory before running OpenSearch on Windows Signed-off-by: Alex Burck <[email protected]> * update changlog pr link Signed-off-by: Alex Burck <[email protected]> Signed-off-by: Alex Burck <[email protected]> * Use RemoteSegmentStoreDirectory instead of RemoteDirectory (#4240) * Use RemoteSegmentStoreDirectory instead of RemoteDirectory Signed-off-by: Sachin Kale <[email protected]> * ZIP publication groupId value is configurable (#4156) When publishing Zip POM the groupId value was hard-coded to `org.opensearch.plugin` value which worked fine for existing core plugins but is not convenient for other plugins (such as community plugins maintained in independent repositories). This PR changes the sources of the ZIP publishing groupId value. Specifically, there are two ways to set the value: 1) It is automatically inherited from the Gradle "project.group" 2) It can be manually specified in the ZIP publication POM object This PR also brings a major rework of tests in PublishTests class. Individual testing scenarios are driven by "real" gradle building scripts (utilizing `java-gradle-plugin` gradle plugin). Closes #3692 Signed-off-by: Lukáš Vlček <[email protected]> Signed-off-by: Lukáš Vlček <[email protected]> * [Segment Replication] Add timeout on Mockito.verify to reduce flakyness in testReplicationOnDone test (#4314) * [Segment Replication] testReplicationOnDone Add timeout to allow time for verify call Signed-off-by: Suraj Singh <[email protected]> * Update changelog Signed-off-by: Suraj Singh <[email protected]> * Add change log entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Adding @dreamer-89 to Opensearch maintainers. (#4342) Signed-off-by: Kartik Ganesh <[email protected]> Signed-off-by: Kartik Ganesh <[email protected]> * [CVE] Update snakeyaml dependency (#4341) The package `org.yaml:snakeyaml` before version 1.31 are vulnerable to Denial of Service (DoS) due missing to nested depth limitation for collections. Details at https://nvd.nist.gov/vuln/detail/CVE-2022-25857 Signed-off-by: Rabi Panda <[email protected]> * Fixed commit workflow for dependabot PR helper (#4331) * Fixed label for dependabot PR helper Signed-off-by: Kunal Kotwani <[email protected]> * Update autocommit workflow for dependabot changelog Signed-off-by: Kunal Kotwani <[email protected]> * Add version config for dependabot changelog helper Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Add release notes for patch release 1.3.5 (#4343) Signed-off-by: Rabi Panda <[email protected]> * Add release notes for patch release 2.2.1 (#4344) Signed-off-by: Rabi Panda <[email protected]> * Add label configuration for dependabot PRs (#4348) * Support for HTTP/2 (server-side) (#3847) * Support for HTTP/2 (server-side) Signed-off-by: Andriy Redko <[email protected]> * Addressing code review comments Signed-off-by: Andriy Redko <[email protected]> * Added HTTP/1.1 channel configuration Signed-off-by: Andriy Redko <[email protected]> * Addressing code review comments Signed-off-by: Andriy Redko <[email protected]> * Update pul request URL in CHANGELOG.md Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Fix token usage for changelog helper (#4351) Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Revert "Fix token usage for changelog helper (#4351)" (#4361) This reverts commit c28221e0176ad3ce782c18a2b23ea2e59ed0e0a8. Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Segment Replication - Implement segment replication event cancellation. (#4225) * Segment Replication. Fix Cancellation of replication events. This PR updates segment replication paths to correctly cancel replication events on the primary and replica. In the source service, any ongoing event for a primary that is sending to a replica that shuts down or is promoted as a new primary are cancelled. In the target service, any ongoing event for a replica that is promoted as a new primary or is fetching from a primary that shuts down. It wires up SegmentReplicationSourceService as an IndexEventListener so that it can respond to events and cancel any ongoing transfer state. This change also includes some test cleanup for segment replication to rely on actual components over mocks. Signed-off-by: Marc Handalian <[email protected]> Fix to not start/stop SegmentReplicationSourceService as a lifecycle component with feature flag off. Signed-off-by: Marc Handalian <[email protected]> Update logic to properly mark SegmentReplicationTarget as cancelled when cancel initiated by primary. Signed-off-by: Marc Handalian <[email protected]> Minor updates from self review. Signed-off-by: Marc Handalian <[email protected]> * Add missing changelog entry. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Bug fixes for dependabot changelog verifier (#4364) * Fix token usage for changelog helper Signed-off-by: Kunal Kotwani <[email protected]> * Add conditional check for dependabot steps Signed-off-by: Kunal Kotwani <[email protected]> * Add dependency section Signed-off-by: Kunal Kotwani <[email protected]> * Bug fixes for dependabot changelog verifier Signed-off-by: Kunal Kotwani <[email protected]> * Update the changelog Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Add changes for Create PIT and Delete PIT rest layer and rest high level client (#4064) * Create and delete PIT search rest layer changes Signed-off-by: Bharathwaj G <[email protected]> * Bump com.diffplug.spotless from 6.9.1 to 6.10.0 (#4319) * Update to Netty 4.1.80.Final (#4359) Signed-off-by: Andriy Redko <[email protected]> * Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment (#4354) * Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment Bumps xmlbeans from 5.1.0 to 5.1.1. --- updated-dependencies: - dependency-name: org.apache.xmlbeans:xmlbeans dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Fix randomized test failure NRTReplicationEngineTests.testUpdateSegments (#4352) Overload `generateHistoryOnReplica` to be able to generate only a specific `Engine.Operation.TYPE` operations as required by the `testUpdateSegments` test Signed-off-by: Rabi Panda <[email protected]> Signed-off-by: Rabi Panda <[email protected]> * [AUTO] [main] Added bwc version 2.2.2. (#4383) * Added bwc version 2.2.2 * Add changelog Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Kunal Kotwani <[email protected]> * [Segment Replication] Bump segment infos counter before commit during replica promotion (#4365) * [Segment Replication] Bump segment infos counter before commit during replica promotion Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica (#4363) * [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> * Address review comments Signed-off-by: Suraj Singh <[email protected]> * Address review comments 2 Signed-off-by: Suraj Singh <[email protected]> * Test failures Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Adding support for labels on version bump PRs, skip label support for… (#4391) * Adding support for labels on version bump PRs, skip label support for changelog verifier Signed-off-by: Kunal Kotwani <[email protected]> * Add changelog Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * [Segment Replication] Extend FileChunkWriter to allow cancel on transport client (#4386) * [Segment Replication] Extend FileChunkWriter to allow cancel on retryable transport client Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> * Address review comments Signed-off-by: Suraj Singh <[email protected]> * Integration test Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Added RestLayer Changes for PIT stats (#4217) Signed-off-by: Ajay Kumar Movva <[email protected]> * Modified cat shards test for pit stats (#4408) Signed-off-by: Ajay Kumar Movva <[email protected]> * [Remote Store] Add index specific setting for remote repository (#4253) * Add index specific setting for remote repository * Fix for failover incremental uploads Signed-off-by: Sachin Kale <[email protected]> * [Semgnet Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test (#4414) * [Semgnet Replication] Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> * Update changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards. (#4366) * Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards. This change fixes the errors that occur when computing metadata snapshots on primary shards from the latest in-memory SegmentInfos. The error occurs when a segments_N file that is referenced by the in-memory infos is deleted as part of a concurrent commit. The segments themselves are incref'd by IndexWriter.incRefDeleter but the commit file (Segments_N) is not. This change resolves this by ignoring the segments_N file when computing metadata for CopyState and only sending incref'd segment files to replicas. Signed-off-by: Marc Handalian <[email protected]> * Fix spotless. Signed-off-by: Marc Handalian <[email protected]> * Update StoreTests.testCleanupAndPreserveLatestCommitPoint to assert additional segments are deleted. Signed-off-by: Marc Handalian <[email protected]> * Rename snapshot to metadataMap in CheckpointInfoResponse. Signed-off-by: Marc Handalian <[email protected]> * Refactor segmentReplicationDiff method to compute off two maps instead of MetadataSnapshots. Signed-off-by: Marc Handalian <[email protected]> * Fix spotless. Signed-off-by: Marc Handalian <[email protected]> * Revert catchall in SegmentReplicationSourceService. Signed-off-by: Marc Handalian <[email protected]> * Revert log lvl change. Signed-off-by: Marc Handalian <[email protected]> * Fix SegmentReplicationTargetTests Signed-off-by: Marc Handalian <[email protected]> * Cleanup unused logger. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Co-authored-by: Suraj Singh <[email protected]> * Bump org.gradle.test-retry from 1.4.0 to 1.4.1 (#4411) * Bump org.gradle.test-retry from 1.4.0 to 1.4.1 Bumps org.gradle.test-retry from 1.4.0 to 1.4.1. --- updated-dependencies: - dependency-name: org.gradle.test-retry dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Revert to Netty 4.1.79.Final (#4428) Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Craig Perkins <[email protected]> * [Segment Replication] Fix timeout issue by calculating time needed to process getSegmentFiles. (#4426) * Fix timeout issue by calculating time needed to process getSegmentFiles. Signed-off-by: Rishikesh1159 <[email protected]> * Formatting sizeOfSegmentFiles for time calculation. Signed-off-by: Rishikesh1159 <[email protected]> * Addressing comments and applying spotless check. Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> * [Bug]: gradle check failing with java heap OutOfMemoryError (#4328) * [Bug]: gradle check failing with java heap OutOfMemoryError Signed-off-by: Andriy Redko <[email protected]> * Fork JavaCompile task Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Bump azure-core-http-netty from 1.12.0 to 1.12.4 in /plugins/repository-azure (#4160) * Bump azure-core-http-netty in /plugins/repository-azure Bumps [azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.12.0 to 1.12.4. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](https://github.com/Azure/azure-sdk-for-java/compare/azure-core_1.12.0...azure-core-http-netty_1.12.4) --- updated-dependencies: - dependency-name: com.azure:azure-core-http-netty dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Added missing forbidden classes Signed-off-by: Owais Kazi <[email protected]> * update version of azure-storage-common and azure-core-http-netty Signed-off-by: Xue Zhou <[email protected]> * update version of azure-storage-common and azure-core-http-netty Signed-off-by: Xue Zhou <[email protected]> * adding changelog information Signed-off-by: Xue Zhou <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: Xue Zhou <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Xue Zhou <[email protected]> * Bugfix: Allow opensearch.bat file and opensearch-env.bat files to run when install path includes a space. (#4362) * Bugfix: Prevent escaping of in `else-if` statement by setting variable without double quotes. Signed-off-by: Mike Wilkinson <[email protected]> * Add changelog entry for fix Signed-off-by: Mike Wilkinson <[email protected]> * Escape double quotes for environment variables set by `opensearch-env.bat`. Explicitly apply quotes where those environment variables are invoked. Signed-off-by: Mike Wilkinson <[email protected]> Signed-off-by: Mike Wilkinson <[email protected]> Co-authored-by: Mike Wilkinson <[email protected]> * [Segment Replication] - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. (#4402) * Segment Replication - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. This change updates replicas to commit SegmentInfos before the shard is closed, on receiving a new commit point from a primary, and when a new primary is detected. This change also makes the public commitSegmentInfos on NRTEngine obsolete, refactoring IndexShard to simply call reset on the engine. Signed-off-by: Marc Handalian <[email protected]> * Remove noise & extra log statement. Signed-off-by: Marc Handalian <[email protected]> * PR feedback. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Bump reactor-netty-core from 1.0.19 to 1.0.22 in /plugins/repository-azure (#4447) * Bump reactor-netty-core in /plugins/repository-azure Bumps [reactor-netty-core](https://github.com/reactor/reactor-netty) from 1.0.19 to 1.0.22. - [Release notes](https://github.com/reactor/reactor-netty/releases) - [Commits](https://github.com/reactor/reactor-netty/compare/v1.0.19...v1.0.22) --- updated-dependencies: - dependency-name: io.projectreactor.netty:reactor-netty-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add bwcVersion 2.4.0 (#4455) * Add bwcVersion 2.4.0 Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * add support for s390x architecture (#4001) * add s390x support in systemcallfilter https://github.com/opensearch-project/OpenSearch/issues/4000 Signed-off-by: Andrija Pantovic <[email protected]> * add gradle cfg for s390x Signed-off-by: Andrija Pantovic <[email protected]> * change assertion for architecture Signed-off-by: Andrija Pantovic <[email protected]> * change assertion for architecture Signed-off-by: Andrija Pantovic <[email protected]> * update changelog Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> * 2.3.0 release notes (#4457) (#4464) * 2.3.0 release notes Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Add getHistoryOperationsFromTranslog method to fetch the history snapshot from translogs (#3948) * Add getHistoryOperationsFromTranslog method to fetch the hostory snapshot from translogs Signed-off-by: Ankit Kala <[email protected]> * Fixed flaky test: ResourceAwareTasksTests.testTaskIdPersistsInThreadContext (#4484) Signed-off-by: Ketan Verma <[email protected]> * Getting security exception due to access denied 'java.lang.RuntimePermission' 'accessDeclaredMembers' when trying to get snapshot with S3 IRSA (#4469) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Co-authored-by: Suraj Singh <[email protected]> * [Javadoc] add missing javadocs for :distribution:tools modules (#4483) * Add javadocs for distribution:tools:java-version-checker Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for distribution:tools:keystore-cli Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for distribution:tools:plugin-cli Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for distribution:tools:launchers Signed-off-by: Daniel Widdis <[email protected]> * Change log Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Bump reactive-streams from 1.0.3 to 1.0.4 in /plugins/repository-azure (#4488) * Bump reactive-streams from 1.0.3 to 1.0.4 in /plugins/repository-azure Bumps [reactive-streams](https://github.com/reactive-streams/reactive-streams) from 1.0.3 to 1.0.4. - [Release notes](https://github.com/reactive-streams/reactive-streams/releases) - [Changelog](https://github.com/reactive-streams/reactive-streams-jvm/blob/master/RELEASE-NOTES.md) - [Commits](https://github.com/reactive-streams/reactive-streams/compare/v1.0.3...v1.0.4) --- updated-dependencies: - dependency-name: org.reactivestreams:reactive-streams dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Ignore all malformed objects when ignore_malformed is true (#4494) Fixes a bug to not fail the entire document when "ignore_malformed" is set to true. Allowing the valid fields to be indexed and ignore only the malformed fields. Signed-off-by: Hauck <[email protected]> * [AUTO] [main] Added bwc version 2.3.1. (#4513) * Added bwc version 2.3.1 * Add changelog of adding bwc version 2.3.1 Signed-off-by: Tianli Feng <[email protected]> * [Segment Replication] Add snapshot and restore tests for segment replication feature (#3993) * [Segment Replication] Add snapshots tests with segment replication enabled Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry, address review comments, add failover test Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Address review comments 2 Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * [Remote Store] Change behaviour in replica recovery for remote translog enabled indices (#4318) Signed-off-by: Ashish Singh <[email protected]> * Weighted round-robin scheduling policy for shard coordination traffic… (#4241) * Weighted round-robin scheduling policy for shard coordination traffic routing Signed-off-by: Anshu Agarwal <[email protected]> * [Javadoc] add missing javadocs for :example-plugin modules (#4540) * Add javadocs for example-plugins:custom-settings module Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for example-plugins:custom-significance-heuristic Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs for example-plugins:custom-suggester Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:painless-allowlist Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:rescore Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:rest-handler Signed-off-by: Daniel Widdis <[email protected]> * Add javadocs to example-plugins:script-expert-scoring Signed-off-by: Daniel Widdis <[email protected]> * Remove exclusions for module which aren't giving javadoc errors Signed-off-by: Daniel Widdis <[email protected]> * CI testing apparently needs to instantiate a no-op class Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Add GeoBounds aggregation on GeoShape field type.(#3980) (#4266) Enables geo_bounds aggregation to work with geo_shape field types. This enhancement includes: * Addition of Doc values on the GeoShape Field. * Addition of GeoShape ValueSource level code interfaces for accessing the DocValues. * Addition of Missing Value feature in the GeoShape Aggregations. Signed-off-by: Navneet Verma <[email protected]> * Updates jackson and snakeyml dependencies (#4556) * Upgrade jackson and snakeyml dependencies Signed-off-by: Owais Kazi <[email protected]> * Updated changelog Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: Owais Kazi <[email protected]> * Bump com.diffplug.spotless from 6.10.0 to 6.11.0 (#4547) * Bump com.diffplug.spotless from 6.10.0 to 6.11.0 --- updated-dependencies: - dependency-name: com.diffplug.spotless dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure (#4548) * Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure Bumps [reactor-core](https://github.com/reactor/reactor-core) from 3.4.18 to 3.4.23. - [Release notes](https://github.com/reactor/reactor-core/releases) - [Commits](https://github.com/reactor/reactor-core/compare/v3.4.18...v3.4.23) --- updated-dependencies: - dependency-name: io.projectreactor:reactor-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment (#4550) * Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment Bumps jempbox from 1.8.16 to 1.8.17. --- updated-dependencies: - dependency-name: org.apache.pdfbox:jempbox dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Bump log4j-core from 2.18.0 to 2.19.0 in /buildSrc/src/testKit/thirdPartyAudit (#4549) * Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit Bumps log4j-core from 2.18.0 to 2.19.0. --- updated-dependencies: - dependency-name: org.apache.logging.log4j:log4j-core dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Ignore lock file when testing cleanupAndPreserveLatestCommitPoint (#4544) Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Remove deprecated code to add node name into log pattern of log4j property file (#4568) Signed-off-by: Tianli Feng <[email protected]> * Unmute RelocationIT.testRelocationWhileIndexingRandom. (#4580) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Add DecommissionService and helper to execute awareness attribute decommissioning (#4084) * Add Executor to decommission node attribute * Decommission service implementation with cluster metadata * Master abdication changes to decommission local awareness leader * Update join validator changes to validate decommissioned node join request Signed-off-by: Rishab Nahata <[email protected]> * Added rest layer changes for List all PITs and PIT segments (#4388) * Changes for list all and pit segments Signed-off-by: Bharathwaj G <[email protected]> * Add PUT api to update shard routing weights (#4272) * Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]> * Further simplification of the ZIP publication implementation (#4360) A follow-up of PR #4156 that brings in further simplification of the ZIP publication implementation and new tests. It is possible to remove some of the initialization code because the work is already handled by other library under the hood (most likely by NebulaPublishPlugin). For instance the condition to test the `groupId` presence: `if (groupId == null)` was pointless. It was never `true`. If the `project.group` is not defined the publication task fails with an exception, if there is a custom `groupId` value setup in publication config then it overrides the `project.group` as well. Tests are provided to cover these use cases. As for the new tests they cover the following cases: - verify that the task "publishToMavenLocal" gets expected results. The inspiration for this comes from https://github.com/opensearch-project/opensearch-plugin-template-java/pull/35 - verify that applying only the 'opensearch.pluginzip' is enough, because both the NebulaPublish and MavenPublishPlugin plugins are added explicitly and tasks are chained correctly - verify that if the plugin is applied but no publication is defined then a message is printed for the user Signed-off-by: Lukáš Vlček <[email protected]> Signed-off-by: Lukáš Vlček <[email protected]> * Add missing Javadoc tag descriptions in unit tests (#4629) * Add missing Javadoc tag descriptions in unit tests Using JDK14 (downloaded from https://jdk.java.net/archive/), I tried running ./gradlew check, and received errors like: server/src/test/java/org/opensearch/indices/replication/SegmentReplicationTargetTests.java:440: warning: no description for @throws The build then failed with "warnings found and -Werror specified". I tried again with JDK17 and the build passed. According to DEVELOPER_GUIDE.md, we should be able to build with JDK14, I added the missing Javadoc tag descriptions. Signed-off-by: Michael Froh <[email protected]> * Update CHANGELOG.md Signed-off-by: Michael Froh <[email protected]> Signed-off-by: Michael Froh <[email protected]> * Bump reactor-netty-http from 1.0.18 to 1.0.23 in /plugins/repository-azure (#4594) * Bump reactor-netty-http in /plugins/repository-azure Bumps [reactor-netty-http](https://github.com/reactor/reactor-netty) from 1.0.18 to 1.0.23. - [Release notes](https://github.com/reactor/reactor-netty/releases) - [Commits](https://github.com/reactor/reactor-netty/compare/v1.0.18...v1.0.23) --- updated-dependencies: - dependency-name: io.projectreactor.netty:reactor-netty-http dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic (#4592) * Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic Bumps [jettison](https://github.com/jettison-json/jettison) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/jettison-json/jettison/releases) - [Commits](https://github.com/jettison-json/jettison/compare/jettison-1.5.0...jettison-1.5.1) --- updated-dependencies: - dependency-name: org.codehaus.jettison:jettison dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Fix invalid search location of JDK for arm64 (#4613) Related commit: 0e9f74e35f1255cb9ec45be3d8960aad195a9f6e Signed-off-by: Heemin Kim <[email protected]> Signed-off-by: Heemin Kim <[email protected]> * Remove unused object and import from TransportClusterAllocationExplainAction (#4639) Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> * Relax visibility of the HTTP_CHANNEL_KEY and HTTP_SERVER_CHANNEL_KEY to make it possible for the plugins to access associated Netty4HttpChannel / Netty4HttpServerChannel instance (#4638) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Bumping hadoop-hdfs version to 3.3.4 (#4644) * Updating hadoop-hdfs version Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> Signed-off-by: Sarat Vemulapalli <[email protected]> * Bugfix/496 rollover alias with filters (#4499) * 496: Added filters, indexRouting and searchRouting when doing rollover on alias Signed-off-by: Stevan Buzejic <[email protected]> * 496: Removed unused map property in metada alias rollover tests Signed-off-by: Stevan Buzejic <[email protected]> Signed-off-by: Stevan Buzejic <[email protected]> Co-authored-by: Stevan Buzejic <[email protected]> * Load the deprecated master role in a dedicated method instead of in setAdditionalRoles() (#4582) Signed-off-by: Tianli Feng <[email protected]> * Add APIs (GET/PUT) to decommission awareness attribute (#4261) * Add APIs (GET/PUT) to decommission awareness attribute Signed-off-by: Rishab Nahata <[email protected]> * Adding check to make sure checkpoint is not processed when a shard's shard routing is primary. (#4630) Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> Co-authored-by: Suraj Singh <[email protected]> * Bump azure-storage-common from 12.18.0 to 12.18.1 in /plugins/repository-azure (#4664) * Bump azure-storage-common in /plugins/repository-azure Bumps [azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.18.0 to 12.18.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](https://github.com/Azure/azure-sdk-for-java/compare/azure-storage-blob_12.18.0...azure-storage-common_12.18.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Bump forbiddenapis from 3.3 to 3.4 in /buildSrc/src/testKit/thirdPartyAudit (#4662) * Bump forbiddenapis in /buildSrc/src/testKit/thirdPartyAudit Bumps [forbiddenapis](https://github.com/policeman-tools/forbidden-apis) from 3.3 to 3.4. - [Release notes](https://github.com/policeman-tools/forbidden-apis/releases) - [Commits](https://github.com/policeman-tools/forbidden-apis/compare/3.3...3.4) --- updated-dependencies: - dependency-name: de.thetaphi:forbiddenapis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Fixed the SnapshotsInProgress error during index deletion (#4570) Signed-off-by: Xue Zhou <[email protected]> * Update to Apache Lucene 9.4.0 (#4661) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Fixed day of year defaulting for round up parser (#4627) Signed-off-by: Vishal Sarda <[email protected]> * Fix PIT flaky tests (#4632) * Fixing PIT flaky tests Signed-off-by: Bharathwaj G <[email protected]> * Add getter for path field in NestedQueryBuilder (#4636) * Pre-commit checks now check for missing javadoc, and the Gradle check Jenkins build now waits for pre-commit checks to pass before attempting (#4660) * Ignore all malformed objects when ignore_malformed is true (#4494) Fixes a bug to not fail the entire document when "ignore_malformed" is set to true. Allowing the valid fields to be indexed and ignore only the malformed fields. Signed-off-by: Hauck <[email protected]> Signed-off-by: Braddy Yeoh <[email protected]> * Adding javadoc check to the precommi\t GitHub Action Signed-off-by: Braddy Yeoh <[email protected]> * gradle check now only runs after successful gradle precommit check Signed-off-by: Braddy Yeoh <[email protected]> * Added to CHANGELOG Signed-off-by: Braddy Yeoh [email protected] Signed-off-by: Braddy Yeoh <[email protected]> Signed-off-by: Hauck <[email protected]> Signed-off-by: Braddy Yeoh <[email protected]> Signed-off-by: Braddy Yeoh [email protected] Co-authored-by: Hauck <[email protected]> Co-authored-by: Braddy Yeoh <[email protected]> * Backport Apache Lucene version change for 2.4.0 (#4677) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Attempt to fix Github workflow for Gradle Check job (#4679) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Adding precommit support for Windows (#4676) * First draft for Windows precommit Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding precommit support for MacOS (darwin) (#4682) * First draft for Windows precommit Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding precommit for macos Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding Changelog Signed-off-by: Sarat Vemulapalli <[email protected]> Signed-off-by: Sarat Vemulapalli <[email protected]> * Fix flaky DecommissionControllerTests.testTimesOut (#4683) This test fails pretty reliably if I run it on repeat. I believe the problem is that the test assumes the function will take longer than 2ms, which is likely not a valid assumption in all cases. Fortunately, I can pass in a zero duration which is guaranteed to timeout even if the system clock does not advance at all. Also moved the assertions out of the callback into the main test method, otherwise the assertion error messages would get buried and the test report would just show a timeout error. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]> * Include Windows OS in Bootstrap initializeNatives() check for definitelyRunningAsRoot() (#4656) * Bump jna from 5.11.0 to 5.12.1 Signed-off-by: Daniel Widdis <[email protected]> * Include Windows OS in check for definitelyRunningAsRoot() Signed-off-by: Daniel Widdis <[email protected]> * Combine try/catch blocks Signed-off-by: Daniel Widdis <[email protected]> * Someone sniped my PR number between predicting and pushing Signed-off-by: Daniel Widdis <[email protected]> * Update SHAs Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> * Add release notes for 1.3.6 (#4681) Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> * Service Layer changes for Recommission API (#4320) * Recommission API service level changes Signed-off-by: pranikum <[email protected]> * Add GET api to get shard routing weights (#4275) * Weighted round-robin scheduling policy for shard coordination traffic routing Signed-off-by: Anshu Agarwal <[email protected]> * Fix new race condition in DecommissionControllerTests (#4688) My previous fix introduced a new race condition by making the assertions before waiting on the latch. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]> * Fixing SearchStats (de)serialization (#4697) Fixes bwc failure caused by commit 6993ac9834d0f7e8e77e6cade09de2245d2a20f7. Signed-off-by: Andriy Redko <[email protected]> * Fixed misunderstanding message 'No OpenSearchException found' when detailed_error disabled (#4669) * Fixed misunderstanding message 'No OpenSearchException found' when detailed_error disabled Signed-off-by: Xue Zhou <[email protected]> * Update GeoGrid base class access modifier to support extensibility (#4572) * Update access modifier to support extensibility Change access modifier from default to protected. This will help to build new geo based aggregation outside OpenSearch, by keeping GeoGrid Classes as base class. Signed-off-by: Vijayan Balasubramanian <[email protected]> * Updated CHANGELOG Added PR details to CHANGELOG.md Signed-off-by: Vijayan Balasubramanian <[email protected]> * Rename InternalGeoGridBucket to BaseGeoGridBucket Update class names, references and comments. Signed-off-by: Vijayan Balasubramanian <[email protected]> * Rename InternalGeoGrid to BaseGeoGrid Signed-off-by: Vijayan Balasubramanian <[email protected]> * Make GridBucket classes package-private Signed-off-by: Vijayan Balasubramanian <[email protected]> * Remove Internal prefix from Geo Grid classes Signed-off-by: Vijayan Balasubramanian <[email protected]> * Update constructor and class access modifier Signed-off-by: Vijayan Balasubramanian <[email protected]> * Update access modifier based on usage Made classes package private if it is not used outside the package. Signed-off-by: Vijayan Balasubramanian <[email protected]> Signed-off-by: Vijayan Balasubramanian <[email protected]> * [Remove] LegacyESVersion.V_7_0_* and V_7_1_* constants (#2768) * [Remove] LegacyESVersion.V_7_0_* and V_7_1_* constants Removes all usages of LegacyESVersion.V_7_0_ and LegacyESVersion.V_7_1 version constants. Signed-off-by: Nicholas Walter Knize <[email protected]> * Rebase from main Signed-off-by: Nicholas Walter Knize <[email protected]> * fix serialization issue with build flavor removal Signed-off-by: Nicholas Walter Knize <[email protected]> * remove stale bwc test Signed-off-by: Nicholas Walter Knize <[email protected]> * rebase and update Signed-off-by: Nicholas Walter Knize <[email protected]> * cleanup Signed-off-by: Nicholas Walter Knize <[email protected]> * fix failing mapper test Signed-off-by: Nicholas Walter Knize <[email protected]> Signed-off-by: Nicholas Walter Knize <[email protected]> * Fixing Gradle warnings associated with publishPluginZipPublicationToXxx tasks (#4696) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Bump gson from 2.9.0 to 2.9.1 in /test/fixtures/hdfs-fixture (#4066) * Bump gson from 2.9.0 to 2.9.1 in /test/fixtures/hdfs-fixture Bumps [gson](https://github.com/google/gson) from 2.9.0 to 2.9.1. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](https://github.com/google/gson/compare/gson-parent-2.9.0...gson-parent-2.9.1) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Always auto release the flood stage block (#4703) * Always auto release the flood stage block Removes support for using a system property to disable the automatic release of the write block applied when a node exceeds the flood-stage watermark. Signed-off-by: Nicholas Walter Knize <[email protected]> * update IAE message Signed-off-by: Nicholas Walter Knize <[email protected]> Signed-off-by: Nicholas Walter Knize <[email protected]> * Controlling discovery for decommissioned nodes (#4590) * Controlling discovery for decommissioned nodes Signed-off-by: Rishab Nahata <[email protected]> * [Remove] LegacyESVersion.V_7_2_* and V_7_3_* constants (#4702) Removes all usages of LegacyESVersion.V_7_2_ and LegacyESVersion.V_7_3 version constants along with related ancient APIs. Signed-off-by: Nicholas Walter Knize <[email protected]> * [Remove] LegacyESVersion.V_7_4_* and V_7_5_* constants (#4704) Removes all usages of LegacyESVersion.V_7_4_ and LegacyESVersion.V_7_5 version constants along with ancient API logic. Signed-off-by: Nicholas Walter Knize <[email protected]> Signed-off-by: Alex Burck <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Tianli Feng <[email protected]> Signed-off-by: Sachin Kale <[email protected]> Signed-off-by: Lukáš Vlček <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Kartik Ganesh <[email protected]> Signed-off-by: Rabi Panda <[email protected]> Signed-off-by: Bharathwaj G <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Ajay Kumar Movva <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: Xue Zhou <[email protected]> Signed-off-by: Mike Wilkinson <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Andrija Pantovic <[email protected]> Signed-off-by: Ankit Kala <[email protected]> Signed-off-by: Ketan Verma <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Hauck <[email protected]> Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Anshu Agarwal <[email protected]> Signed-off-by: Navneet Verma <[email protected]> Signed-off-by: Rishab Nahata <[email protected]> Signed-off-by: Michael Froh <[email protected]> Signed-off-by: Heemin Kim <[email protected]> Signed-off-by: Sarat Vemulapalli <[email protected]> Signed-off-by: Stevan Buzejic <[email protected]> Signed-off-by: Vishal Sarda <[email protected]> Signed-off-by: Braddy Yeoh <[email protected]> Signed-off-by: Braddy Yeoh [email protected] Signed-off-by: pranikum <[email protected]> Signed-off-by: Vijayan Balasubramanian <[email protected]> Signed-off-by: Nicholas Walter Knize <[email protected]> Signed-off-by: Dhwanil Patel <[email protected]> Co-authored-by: Alex Burck <[email protected]> Co-authored-by: Marc Handalian <[email protected]> Co-authored-by: Kunal Kotwani <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: Tianli Feng <[email protected]> Co-authored-by: Sachin Kale <[email protected]> Co-authored-by: Lukáš Vlček <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Co-authored-by: Kartik Ganesh <[email protected]> Co-authored-by: Rabi Panda <[email protected]> Co-authored-by: Bharathwaj G <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Movva Ajaykumar <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Rishikesh Pasham <[email protected]> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Xue Zhou <[email protected]> Co-authored-by: mwilkinson-imo <[email protected]> Co-authored-by: Mike Wilkinson <[email protected]> Co-authored-by: Andrija Pantovic <[email protected]> Co-authored-by: Ankit Kala <[email protected]> Co-authored-by: Ketan Verma <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> Co-authored-by: Hauck <[email protected]> Co-authored-by: Ashish <[email protected]> Co-authored-by: Anshu Agarwal <[email protected]> Co-authored-by: Navneet Verma <[email protected]> Co-authored-by: Rishab Nahata <[email protected]> Co-authored-by: msfroh <[email protected]> Co-authored-by: Heemin Kim <[email protected]> Co-authored-by: Sarat Vemulapalli <[email protected]> Co-authored-by: Stevan Buzejic <[email protected]> Co-authored-by: Stevan Buzejic <[email protected]> Co-authored-by: Xue Zhou <[email protected]> Co-authored-by: Vishal Sarda <[email protected]> Co-authored-by: Pavlo Smahin <[email protected]> Co-authored-by: Braddy Yeoh <[email protected]> Co-authored-by: Braddy Yeoh <[email protected]> Co-authored-by: pranikum <[email protected]> Co-authored-by: Vijayan Balasubramanian <[email protected]> Co-authored-by: Nick Knize <[email protected]>

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

* Weighted round-robin scheduling policy for shard coordination traffic… (#4241) * Add PUT api to update shard routing weights (#4272) * Add GET api to get shard routing weights (#4275) * Fix weighted routing metadata deserialization error during node restart (#4691) * Delete API for weighted round robin search routing (#4400) * Mark apis experimental Signed-off-by: Anshu Agarwal <[email protected]>

* Weighted round-robin scheduling policy for shard coordination traffic… (#4241) * Add PUT api to update shard routing weights (#4272) * Add GET api to get shard routing weights (#4275) * Fix weighted routing metadata deserialization error during node restart (#4691) * Delete API for weighted round robin search routing (#4917 * Mark apis experimental Signed-off-by: Anshu Agarwal <[email protected]>

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 requested review from a team and reta as code owners August 22, 2022 11:49

Gaganjuneja reviewed Aug 26, 2022

View reviewed changes

anshu1106 force-pushed the feature/wrr-put-api branch from 026d3ab to 2f98d20 Compare August 29, 2022 12:50

Anshu Agarwal and others added 12 commits September 1, 2022 13:02

Weighted round-robin scheduling policy for shard coordination traffic…

162cbeb

… routing Signed-off-by: Anshu Agarwal <[email protected]>

Add caching layer for wrr shard routing and moved wrr routing call to…

8513c4f

… shardRoutings Signed-off-by: Anshu Agarwal <[email protected]>

Integrate ARS with weighted round robin,

4cdbee3

Remove Cluster Settings for WRR Add Code refactoring and Java docs Signed-off-by: Anshu Agarwal <[email protected]>

Remove ARS and add tests for zone with undefined weight

7a49e5c

Signed-off-by: Anshu Agarwal <[email protected]>

Add changelog for the commit

03c4d23

Signed-off-by: Anshu Agarwal <[email protected]>

Merge branch 'main' into feature/wrr-shard-routing-core

7c64537

Fix java doc, add test of WeightedRoundRobinRouting metadata

154a3b9

Signed-off-by: Anshu Agarwal <[email protected]>

Fix minor change related to shuffling wrr shard routings

3afb4e3

Signed-off-by: Anshu Agarwal <[email protected]>

Add PUT api for updating weighted round robin shard routing weights

d9e2b08

Signed-off-by: Anshu Agarwal <[email protected]>

Add more validations for request body for put wrr weights

c3b847a

Incorporate review comments and more UTs Signed-off-by: Anshu Agarwal <[email protected]>

Update thread pool executor and do code refactoring

9cc2cd5

Signed-off-by: Anshu Agarwal <[email protected]>

Fix missing java doc build failure

f07a9ad

Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 force-pushed the feature/wrr-put-api branch from 2f98d20 to f07a9ad Compare September 1, 2022 11:56

anshu1106 requested a review from Gaganjuneja September 1, 2022 11:57

Throw ActionRequestValidationException on invalid awareness attribute

e12d247

Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 force-pushed the feature/wrr-put-api branch from e7e592f to e12d247 Compare September 2, 2022 10:16

Merge branch 'main' into feature/wrr-shard-routing-core

30e17bc

pranikum mentioned this pull request Sep 5, 2022

Graceful node decommission wrr #4405

Closed

6 tasks

Bukhtawar reviewed Sep 26, 2022

View reviewed changes

Address review comments

7763067

Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 force-pushed the feature/wrr-put-api branch from c8836b7 to 7763067 Compare September 26, 2022 11:59

Merge branch 'main' into feature/wrr-put-api

3a0b0ff

Address review comments

703d918

Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 requested a review from Bukhtawar September 26, 2022 13:05

Add explicit count in request validation error message

f3cf04d

Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 force-pushed the feature/wrr-put-api branch from 329c6f2 to f3cf04d Compare September 27, 2022 07:11

Bukhtawar approved these changes Sep 27, 2022

View reviewed changes

Bukhtawar merged commit 9b2202a into opensearch-project:main Sep 27, 2022

Vishalks pushed a commit to Vishalks/OpenSearch that referenced this pull request Sep 28, 2022

Add PUT api to update shard routing weights (opensearch-project#4272)

542c8ff

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 mentioned this pull request Oct 6, 2022

[META] Weighted Routing Issue List #4526

Open

20 tasks

sashashura pushed a commit to sashashura/OpenSearch that referenced this pull request Oct 12, 2022

Add PUT api to update shard routing weights (opensearch-project#4272)

a2e18d3

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Nov 1, 2022

Add PUT api to update shard routing weights (opensearch-project#4272)

a6e3c0e

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 mentioned this pull request Nov 1, 2022

[Backport] Weighted routing 2.x #5008

Merged

6 tasks

anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Nov 3, 2022

Add PUT api to update shard routing weights (opensearch-project#4272)

e0ee00c

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

anshu1106 mentioned this pull request Nov 3, 2022

[Backport 2.4]Weighted request routing #5057

Merged

6 tasks

ashking94 pushed a commit to ashking94/OpenSearch that referenced this pull request Nov 7, 2022

Add PUT api to update shard routing weights (opensearch-project#4272)

f7278e8

* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PUT api to update shard routing weights #4272

Add PUT api to update shard routing weights #4272

anshu1106 commented Aug 22, 2022 •

edited

Loading

github-actions bot commented Aug 22, 2022

sachinpkale commented Aug 25, 2022

github-actions bot commented Aug 29, 2022

github-actions bot commented Sep 1, 2022

codecov-commenter commented Sep 1, 2022 •

edited

Loading

github-actions bot commented Sep 2, 2022

github-actions bot commented Sep 2, 2022

github-actions bot commented Sep 2, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022

Bukhtawar Sep 26, 2022

anshu1106 Sep 26, 2022 •

edited

Loading

github-actions bot commented Sep 26, 2022

github-actions bot commented Sep 26, 2022

github-actions bot commented Sep 26, 2022

github-actions bot commented Sep 27, 2022

anshu1106 commented Sep 27, 2022 •

edited

Loading

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Add PUT api to update shard routing weights #4272

Add PUT api to update shard routing weights #4272

Conversation

anshu1106 commented Aug 22, 2022 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Aug 22, 2022

Gradle Check (Jenkins) Run Completed with:

sachinpkale commented Aug 25, 2022

github-actions bot commented Aug 29, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 1, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Sep 1, 2022 • edited Loading

Codecov Report

github-actions bot commented Sep 2, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 2, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 2, 2022

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anshu1106 Sep 26, 2022 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Sep 26, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 26, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 26, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

anshu1106 commented Sep 27, 2022 • edited Loading

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

anshu1106 commented Aug 22, 2022 •

edited

Loading

codecov-commenter commented Sep 1, 2022 •

edited

Loading

anshu1106 Sep 26, 2022 •

edited

Loading

anshu1106 commented Sep 27, 2022 •

edited

Loading