Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to support replication plugin on OpenSearch v2.2.0 #469

Merged

Conversation

saikaranam-amazon
Copy link
Member

@saikaranam-amazon saikaranam-amazon commented Aug 3, 2022

Description

Changes to support replication plugin on OpenSearch v2.2.0

Issues Resolved

#462

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@saikaranam-amazon
Copy link
Member Author

Security tests are failing as release artifacts are not available

Can't get https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.0/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-security-2.2.0.0.zip to /home/runner/work/cross-cluster-replication/cross-cluster-replication/build/dependencies/security/opensearch-security-2.2.0.0.zip

@codecov-commenter
Copy link

codecov-commenter commented Aug 3, 2022

Codecov Report

Merging #469 (91060f5) into main (db5029a) will increase coverage by 0.42%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main     #469      +/-   ##
============================================
+ Coverage     74.69%   75.12%   +0.42%     
- Complexity     1005     1008       +3     
============================================
  Files           141      141              
  Lines          4565     4579      +14     
  Branches        504      506       +2     
============================================
+ Hits           3410     3440      +30     
+ Misses          838      824      -14     
+ Partials        317      315       -2     
Impacted Files Coverage Δ
...lication/metadata/TransportUpdateMetadataAction.kt 76.36% <100.00%> (ø)
...ation/task/shard/ShardReplicationChangesTracker.kt 60.60% <0.00%> (-3.04%) ⬇️
...tion/action/index/TransportReplicateIndexAction.kt 76.66% <0.00%> (+2.59%) ⬆️
...earch/replication/metadata/UpdateIndexBlockTask.kt 77.41% <0.00%> (+3.22%) ⬆️
.../org/opensearch/replication/util/ValidationUtil.kt 62.74% <0.00%> (+6.93%) ⬆️
...cation/action/changes/TransportGetChangesAction.kt 88.88% <0.00%> (+25.39%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@saikaranam-amazon saikaranam-amazon marked this pull request as ready for review August 3, 2022 07:42
@saikaranam-amazon saikaranam-amazon requested a review from a team August 3, 2022 07:42
@saikaranam-amazon
Copy link
Member Author

Security tests are failing as release artifacts are not available

Can't get https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.0/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-security-2.2.0.0.zip to /home/runner/work/cross-cluster-replication/cross-cluster-replication/build/dependencies/security/opensearch-security-2.2.0.0.zip

@prudhvigodithi Are we not publishing pre-release artifacts under the above repo, once they are part of the manifest?

@saikaranam-amazon
Copy link
Member Author

saikaranam-amazon commented Aug 3, 2022

Security plugin build is failing due to compilation issue

/home/runner/work/cross-cluster-replication/cross-cluster-replication/security/src/main/java/org/opensearch/security/configuration/DlsFlsFilterLeafReader.java:920: error: <anonymous org.opensearch.security.configuration.DlsFlsFilterLeafReader$4> is not abstract and does not override abstract method docValueCount() in SortedSetDocValues
                return new SortedSetDocValues() {
                                                ^
/home/runner/work/cross-cluster-replication/cross-cluster-replication/security/src/main/java/org/opensearch/security/configuration/DlsFlsFilterLeafReader.java:968: error: docValueCount() in <anonymous org.opensearch.security.configuration.DlsFlsFilterLeafReader$4> cannot override docValueCount() in SortedSetDocValues
> Task :compileJava
                    public long docValueCount() {
                                ^
  return type long is not compatible with int
/home/runner/work/cross-cluster-replication/cross-cluster-replication/security/src/main/java/org/opensearch/security/configuration/DlsFlsFilterLeafReader.java:967: error: method does not override or implement a method from a supertype
                    @Override
                    ^

Security plugin fix is tracked under: opensearch-project/security#1988

@prudhvigodithi
Copy link
Member

prudhvigodithi commented Aug 3, 2022

Hey @saikaranam-amazon I see with 2.2.0 the distribution build did not pass with security plugin still, hence you are not able to access the zip.
https://build.ci.opensearch.org/job/distribution-build-opensearch/5855/
@peterzhuamazon @bbarani

Copy link
Member

@ankitkala ankitkala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also open the PR to update the security plugin path to latest and link it here for reference.

@saikaranam-amazon saikaranam-amazon merged commit db1b226 into opensearch-project:main Aug 4, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 4, 2022
saikaranam-amazon added a commit that referenced this pull request Aug 4, 2022
Signed-off-by: Sai Kumar <[email protected]>
(cherry picked from commit db1b226)

Co-authored-by: Sai Kumar <[email protected]>
kaituo added a commit to kaituo/anomaly-detection-1 that referenced this pull request Sep 8, 2023
Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR opensearch-project#469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR opensearch-project#681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
kaituo added a commit to kaituo/anomaly-detection-1 that referenced this pull request Sep 8, 2023
Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR opensearch-project#469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR opensearch-project#681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
jackiehanyang pushed a commit to opensearch-project/anomaly-detection that referenced this pull request Sep 8, 2023
…sue (#1029)

Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR #469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR #681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
opensearch-trigger-bot bot pushed a commit to opensearch-project/anomaly-detection that referenced this pull request Sep 8, 2023
…sue (#1029)

Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR #469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR #681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
(cherry picked from commit 3c1a830)
jackiehanyang pushed a commit to opensearch-project/anomaly-detection that referenced this pull request Sep 8, 2023
…sue (#1029) (#1030)

Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR #469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR #681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
(cherry picked from commit 3c1a830)

Co-authored-by: Kaituo Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants