New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify diff calculation for remote routing table #15100

Merged

Bukhtawar merged 8 commits into opensearch-project:main from Bukhtawar:main-simplify-rrt

Sep 2, 2024

Collaborator

Bukhtawar commented Aug 4, 2024 •

edited

Loading

Description

Simplifies diff calculation for remote routing table and makes it compatible with Diff. This change make it simpler to switch between complete diff and incremental diffs.

Some changes in the PR might not be backward compatible with existing Public APIs and would be addressed as a follow up PR

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.


          Add Ashish Singh as maintainer

46f2d41

Signed-off-by: Bukhtawar Khan <[email protected]>

Bukhtawar requested review from anasalkouz, andrross, ashking94, CEHENKLE, dblock, dbwiddis, gbbafna, kotwanikunal, mch2, msfroh, nknize, owaiskazi19, reta, Rishikesh1159, sachinpkale, saratvemulapalli, shwetathareja, sohami and VachaShah as code owners

August 4, 2024 01:50

Bukhtawar marked this pull request as draft

August 4, 2024 01:50

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 3fadc9f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for dd25e86: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 4625fc9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 4cc04c6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for d284c32: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for b1781a0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 607317a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 4607c5d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Aug 4, 2024

❌ Gradle check result for 9b5c271: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 40c9372: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

shailendra0811 force-pushed the main-simplify-rrt branch from 40c9372 to 11d6e8e Compare

September 2, 2024 08:26

Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 11d6e8e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Fix build.

50c98d0

Signed-off-by: Shailendra Singh <[email protected]>

shailendra0811 force-pushed the main-simplify-rrt branch from 11d6e8e to 50c98d0 Compare

September 2, 2024 09:30

Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 50c98d0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 33535ed: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Merge branch 'main' into main-simplify-rrt.

ea927cc

Signed-off-by: Shailendra Singh <[email protected]>

shailendra0811 force-pushed the main-simplify-rrt branch from 33535ed to ea927cc Compare

September 2, 2024 11:45

Contributor

github-actions bot commented Sep 2, 2024

✅ Gradle check result for ea927cc: SUCCESS

shwetathareja approved these changes

View reviewed changes

Bukhtawar merged commit 7247266 into opensearch-project:main

33 of 34 checks passed

opensearch-trigger-bot bot pushed a commit that referenced this pull request


          Simplify diff calculation for remote routing table (#15100)

51e135c

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Shailendra Singh <[email protected]>
(cherry picked from commit 7247266)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

opensearch-trigger-bot bot mentioned this pull request

[Backport 2.x] Simplify diff calculation for remote routing table #15582

Merged

himshikha reviewed

View reviewed changes

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java

Comment on lines +538 to +541

+                              && routingTableDiff.provideDiff() != null
+                              && (!routingTableDiff.provideDiff().getDiffs().isEmpty()
+                                  || !routingTableDiff.provideDiff().getDeletes().isEmpty()
+                                  || !routingTableDiff.provideDiff().getUpserts().isEmpty())) ? 1 : 0);

Contributor

himshikha Sep 2, 2024

nit: looks like repeated logic, can we extract this out, or maybe add a method like hasDiff in the routingTableDiff object?

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java

-                      updatedIndexRouting.addAll(
-                          remoteRoutingTableService.getUpdatedIndexRoutingTableMetadata(diff.getIndicesRoutingUpdated(), manifest.getIndicesRouting())
-                      );
+                      if (manifest.getCodecVersion() == CODEC_V2 || manifest.getCodecVersion() == CODEC_V3) {

Contributor

himshikha Sep 2, 2024

Should we just check Codec < V4?

Bukhtawar pushed a commit that referenced this pull request


          Simplify diff calculation for remote routing table (#15100) (#15582)

50f411e

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <[email protected]>

This was referenced Sep 6, 2024

[AUTOCUT] Gradle Check Flaky Test Report for MasterServiceTests #15809

Open

[AUTOCUT] Gradle Check Flaky Test Report for SnapshotStatusApisIT #15815

Closed

[AUTOCUT] Gradle Check Flaky Test Report for RemoteStoreUploadIndexPathIT #15827

Open

[AUTOCUT] Gradle Check Flaky Test Report for RepositoriesIT #15841

Open

akolarkunnu pushed a commit to akolarkunnu/OpenSearch that referenced this pull request


          Simplify diff calculation for remote routing table (opensearch-projec…

9d91758

…t#15100)

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Shailendra Singh <[email protected]>

This was referenced Sep 11, 2024

[AUTOCUT] Gradle Check Flaky Test Report for MixedClusterClientYamlTestSuiteIT #14294

Open

[AUTOCUT] Gradle Check Flaky Test Report for IndexingIT #14302

Open

[AUTOCUT] Gradle Check Flaky Test Report for RemoteRestoreSnapshotIT #14324

Open

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Open

[AUTOCUT] Gradle Check Flaky Test Report for ClientYamlTestSuiteIT #14319

Open

prudhvigodithi mentioned this pull request

[AUTOCUT] Gradle Check Flaky Test Report for MixedClusterClientYamlTestSuiteIT prudhvigodithi/opensearch-build#82

Open

dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request


          Simplify diff calculation for remote routing table (opensearch-projec…

e3236d2

…t#15100)

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Shailendra Singh <[email protected]>

dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request


          Simplify diff calculation for remote routing table (opensearch-projec…

b95bd5f

…t#15100)

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Shailendra Singh <[email protected]>

dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request


          Simplify diff calculation for remote routing table (opensearch-projec…

…t#15100)

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Shailendra Singh <[email protected]>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

himshikha himshikha left review comments

shailendra0811 shailendra0811 left review comments

shwetathareja shwetathareja approved these changes

anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

andrross Awaiting requested review from andrross andrross is a code owner

ashking94 Awaiting requested review from ashking94 ashking94 is a code owner

CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

dblock Awaiting requested review from dblock dblock is a code owner

dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

mch2 Awaiting requested review from mch2 mch2 is a code owner

msfroh Awaiting requested review from msfroh msfroh is a code owner

nknize Awaiting requested review from nknize nknize is a code owner

owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

reta Awaiting requested review from reta reta is a code owner

Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

sohami Awaiting requested review from sohami sohami is a code owner

VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

jainankitk Awaiting requested review from jainankitk jainankitk is a code owner

linuxpi Awaiting requested review from linuxpi linuxpi is a code owner

gaobinlong Awaiting requested review from gaobinlong

jed326 Awaiting requested review from jed326

peternied Awaiting requested review from peternied

Labels

backport 2.x skip-changelog