Upgrade dependencies #4047

VachaShah · 2022-07-29T20:32:45Z

Signed-off-by: Vacha Shah [email protected]

Description

Upgrading dependencies for:

hadoop3
aws

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Vacha Shah <[email protected]>

github-actions · 2022-07-29T20:46:02Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1121/
CommitID: 35a9906

buildSrc/build.gradle

reta · 2022-07-29T20:53:01Z

...er/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java

@@ -248,7 +248,7 @@ private void assertSegmentStats(int numberOfReplicas) throws IOException {

    /**
     * Waits until the replica is caught up to the latest primary segments gen.
-     * @throws Exception
+     * @throws Exception if assertion fails


VachaShah · 2022-07-29T20:56:45Z

Gradle check failure:

FAILURE: Build failed with an exception.

* What went wrong:
Could not receive a message from the daemon.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org/
Gradle Check Failed!

Signed-off-by: Vacha Shah <[email protected]>

dreamer-89 · 2022-07-30T00:57:49Z

plugins/repository-hdfs/licenses/jetty-server-LICENSE.txt

@@ -0,0 +1,202 @@
+


@VachaShah @reta : Are there any guidelines for reviewing license content.

I got the license from https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-server/9.4.48.v20220622 Licenses section.

github-actions · 2022-07-30T01:19:17Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/1126/
CommitID: e474455

codecov-commenter · 2022-07-30T01:20:06Z

Codecov Report

Merging #4047 (bf7e1e8) into main (85bfde1) will decrease coverage by 0.03%.
The diff coverage is 78.33%.

@@             Coverage Diff              @@
##               main    #4047      +/-   ##
============================================
- Coverage     70.65%   70.62%   -0.04%     
- Complexity    56904    56922      +18     
============================================
  Files          4579     4582       +3     
  Lines        273780   273913     +133     
  Branches      40143    40158      +15     
============================================
+ Hits         193445   193446       +1     
- Misses        64119    64219     +100     
- Partials      16216    16248      +32

Impacted Files	Coverage Δ
...org/opensearch/cluster/service/ClusterService.java	`80.55% <0.00%> (ø)`
...rg/opensearch/common/settings/ClusterSettings.java	`91.89% <ø> (ø)`
.../opensearch/common/util/concurrent/BaseFuture.java	`64.40% <0.00%> (ø)`
...ns/bucket/adjacency/AdjacencyMatrixAggregator.java	`19.78% <0.00%> (ø)`
...a/org/opensearch/test/OpenSearchIntegTestCase.java	`56.83% <0.00%> (-1.06%)`	⬇️
...c/main/java/org/opensearch/test/RandomObjects.java	`90.90% <0.00%> (-1.95%)`	⬇️
...luster/metadata/MetadataUpdateSettingsService.java	`69.94% <37.50%> (-0.92%)`	⬇️
...g/opensearch/cluster/coordination/Coordinator.java	`78.73% <50.00%> (-0.73%)`	⬇️
.../org/opensearch/cluster/service/MasterService.java	`83.19% <62.50%> (+0.95%)`	⬆️
...g/opensearch/index/mapper/DocCountFieldMapper.java	`79.41% <79.41%> (ø)`
... and 529 more

Help us with your feedback. Take ten seconds to tell us how you rate us.

VachaShah · 2022-08-01T18:22:04Z

@reta Can you review this again please?

reta · 2022-08-01T18:39:08Z

test/fixtures/hdfs-fixture/build.gradle

@@ -44,4 +44,6 @@ dependencies {
  api "com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}"
  api 'net.minidev:json-smart:2.4.8'
  api "org.mockito:mockito-core:${versions.mockito}"
+  api "com.google.protobuf:protobuf-java:3.21.2"
+  api "org.eclipse.jetty.websocket:websocket-client:9.4.48.v20220622"


I believe this dependency could be excluded, there is no WS usage anywhere (afaik), right?

reta · 2022-08-01T18:40:00Z

plugins/repository-hdfs/build.gradle

@@ -83,6 +83,7 @@ dependencies {
  api 'net.minidev:json-smart:2.4.8'
  api 'org.apache.zookeeper:zookeeper:3.8.0'
  api "io.netty:netty-all:${versions.netty}"
+  api "org.eclipse.jetty:jetty-server:9.4.48.v20220622"


I think we could exclude this one as well, is it really needed?

Signed-off-by: Vacha Shah <[email protected]>

github-actions · 2022-08-01T20:56:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/1207/
CommitID: bf7e1e8

* Upgrading dependencies for hadoop and aws-java-sdk Signed-off-by: Vacha Shah <[email protected]> * Fixing precommit Signed-off-by: Vacha Shah <[email protected]> * Upgrading transitive dependencies Signed-off-by: Vacha Shah <[email protected]> * Excluding transitive dependencies Signed-off-by: Vacha Shah <[email protected]>

opensearch-trigger-bot · 2022-08-08T17:58:36Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4047-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 88f5537b10a77cd1a7aa32990df54e581f1ea369
# Push it to GitHub
git push --set-upstream origin backport/backport-4047-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4047-to-2.x.

* Upgrading dependencies for hadoop and aws-java-sdk Signed-off-by: Vacha Shah <[email protected]> * Fixing precommit Signed-off-by: Vacha Shah <[email protected]> * Upgrading transitive dependencies Signed-off-by: Vacha Shah <[email protected]> * Excluding transitive dependencies Signed-off-by: Vacha Shah <[email protected]> (cherry picked from commit 88f5537)

* Upgrading dependencies for hadoop and aws-java-sdk Signed-off-by: Vacha Shah <[email protected]> * Fixing precommit Signed-off-by: Vacha Shah <[email protected]> * Upgrading transitive dependencies Signed-off-by: Vacha Shah <[email protected]> * Excluding transitive dependencies Signed-off-by: Vacha Shah <[email protected]> (cherry picked from commit 88f5537) (cherry picked from commit 7d92565)

* Upgrade dependencies (#4047) * Upgrading dependencies for hadoop and aws-java-sdk Signed-off-by: Vacha Shah <[email protected]> * Fixing precommit Signed-off-by: Vacha Shah <[email protected]> * Upgrading transitive dependencies Signed-off-by: Vacha Shah <[email protected]> * Excluding transitive dependencies Signed-off-by: Vacha Shah <[email protected]> (cherry picked from commit 88f5537) * Removing seg-rep IT class from the scope of this backport The original change included a javadoc update to the IT class, but the test class was never backported from #3743 to 2.x. Since the scope of this change is to backport dependency upgrades only, this class has been removed. Signed-off-by: Kartik Ganesh <[email protected]> Co-authored-by: Vacha Shah <[email protected]>

* Upgrade dependencies (#4047) * Upgrading dependencies for hadoop and aws-java-sdk Signed-off-by: Vacha Shah <[email protected]> * Fixing precommit Signed-off-by: Vacha Shah <[email protected]> * Upgrading transitive dependencies Signed-off-by: Vacha Shah <[email protected]> * Excluding transitive dependencies Signed-off-by: Vacha Shah <[email protected]> (cherry picked from commit 88f5537) (cherry picked from commit 7d92565) * Removing seg-rep IT class from the scope of this backport The original change included a javadoc update to the IT class, but the test class was never backported from #3743 to 2.x. Since the scope of this change is to backport dependency upgrades only, this class has been removed. Signed-off-by: Kartik Ganesh <[email protected]> (cherry picked from commit d6ead92) Co-authored-by: Vacha Shah <[email protected]>

opensearch-trigger-bot · 2023-01-26T17:41:00Z

The backport to 1.3 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-1.3 1.3
# Navigate to the new working tree
pushd ../.worktrees/backport-1.3
# Create a new branch
git switch --create backport/backport-4047-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 88f5537b10a77cd1a7aa32990df54e581f1ea369
# Push it to GitHub
git push --set-upstream origin backport/backport-4047-to-1.3
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-1.3

Then, create a pull request where the base branch is 1.3 and the compare/head branch is backport/backport-4047-to-1.3.

VachaShah added 2 commits July 29, 2022 19:29

Upgrading dependencies for hadoop and aws-java-sdk

c0ee103

Signed-off-by: Vacha Shah <[email protected]>

Fixing precommit

35a9906

Signed-off-by: Vacha Shah <[email protected]>

VachaShah requested review from a team and reta as code owners July 29, 2022 20:32

reta reviewed Jul 29, 2022

View reviewed changes

buildSrc/build.gradle Show resolved Hide resolved

reta reviewed Jul 29, 2022

View reviewed changes

reta approved these changes Jul 29, 2022

View reviewed changes

Upgrading transitive dependencies

e474455

Signed-off-by: Vacha Shah <[email protected]>

dreamer-89 reviewed Jul 30, 2022

View reviewed changes

VachaShah requested a review from reta July 30, 2022 02:59

reta reviewed Aug 1, 2022

View reviewed changes

Excluding transitive dependencies

bf7e1e8

Signed-off-by: Vacha Shah <[email protected]>

reta approved these changes Aug 1, 2022

View reviewed changes

VachaShah merged commit 88f5537 into opensearch-project:main Aug 1, 2022

saratvemulapalli added the backport 2.x Backport to 2.x branch label Aug 8, 2022

saratvemulapalli mentioned this pull request Aug 8, 2022

[BUG] Update aws-java-sdk-s3 to >1.12.261 #4161

Closed

kartg mentioned this pull request Aug 8, 2022

[Backport 2.x] Upgrade dependencies #4165

Merged

5 tasks

kartg mentioned this pull request Aug 8, 2022

[Backport 2.2] Upgrade dependencies #4166

Merged

5 tasks

ryanbogan added the backport 1.3 Backport to 1.3 branch label Jan 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade dependencies #4047

Upgrade dependencies #4047

VachaShah commented Jul 29, 2022

github-actions bot commented Jul 29, 2022

reta Jul 29, 2022

VachaShah commented Jul 29, 2022

dreamer-89 Jul 30, 2022

VachaShah Jul 30, 2022

github-actions bot commented Jul 30, 2022

codecov-commenter commented Jul 30, 2022 •

edited

Loading

VachaShah commented Aug 1, 2022

reta Aug 1, 2022

reta Aug 1, 2022

github-actions bot commented Aug 1, 2022

opensearch-trigger-bot bot commented Aug 8, 2022

opensearch-trigger-bot bot commented Jan 26, 2023

Upgrade dependencies #4047

Upgrade dependencies #4047

Conversation

VachaShah commented Jul 29, 2022

Description

Issues Resolved

Check List

github-actions bot commented Jul 29, 2022

Gradle Check (Jenkins) Run Completed with:

reta Jul 29, 2022

Choose a reason for hiding this comment

VachaShah commented Jul 29, 2022

dreamer-89 Jul 30, 2022

Choose a reason for hiding this comment

VachaShah Jul 30, 2022

Choose a reason for hiding this comment

github-actions bot commented Jul 30, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Jul 30, 2022 • edited Loading

Codecov Report

VachaShah commented Aug 1, 2022

reta Aug 1, 2022

Choose a reason for hiding this comment

reta Aug 1, 2022

Choose a reason for hiding this comment

github-actions bot commented Aug 1, 2022

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 8, 2022

opensearch-trigger-bot bot commented Jan 26, 2023

codecov-commenter commented Jul 30, 2022 •

edited

Loading