Skip to content

Commit

Permalink
remove aws-sdk from ranger-extension (#16011)
Browse files Browse the repository at this point in the history
Fixes # size blowup regression introduced in #15443

This PR removes the transitive dependency of ranger-plugins-audit to reduce the size of the compiled artifacts

* add aws-logs-sdk to ensure that all the transitive dependencies are satisfied
* replace aws-bundle-sdk with aws-logs-sdk
* add additional guidance on ranger update, add dependency ignore to satisfy dependency analyzer
* add aws-sdk-logs to list of ignored dependencies to satisfy the maven plugin
* align aws-sdk versions
  • Loading branch information
janjwerner-confluent authored Mar 8, 2024
1 parent 6076649 commit a7b2747
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 8 deletions.
23 changes: 17 additions & 6 deletions extensions-core/druid-ranger-security/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,6 @@
<artifactId>woodstox-core</artifactId>
<version>6.4.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bundle</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

Expand Down Expand Up @@ -178,6 +173,11 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
</exclusion>
<!-- removing the complete bundle sdk to be replaced with only logs-sdk -->
<exclusion>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bundle</artifactId>
</exclusion>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>*</artifactId>
Expand All @@ -193,7 +193,14 @@
</exclusions>
<scope>compile</scope>
</dependency>

<!-- This is a transitive dependency of ranger-plugins-audit
added here to replace bloated bundle sdk, remove this if /when
apache ranger replaces bundle-skd with logs-sdk -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-logs</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
Expand Down Expand Up @@ -254,6 +261,10 @@
<!-- These are needed for scope: compile -->
<dependency>org.apache.ranger:ranger-plugins-audit</dependency>
</usedDependencies>
<!-- this is due to replacement of aws-bundle-sdk with aws-logs-sdk -->
<ignoredDependencies>
<ignoredDependency>com.amazonaws:aws-java-sdk-logs</ignoredDependency>
</ignoredDependencies>
</configuration>
</plugin>
</plugins>
Expand Down
14 changes: 12 additions & 2 deletions licenses.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4650,13 +4650,23 @@ libraries:

---

name: com.amazonaws aws-java-sdk-bundle
name: com.amazonaws aws-java-sdk-logs
license_category: binary
version: 1.12.638
module: druid-ranger-security
license_name: Apache License version 2.0
libraries:
- com.amazonaws: aws-java-sdk-bundle
- com.amazonaws: aws-java-sdk-logs

---

name: com.amazonaws jmespath-java
license_category: binary
version: 1.12.638
module: druid-ranger-security
license_name: Apache License version 2.0
libraries:
- com.amazonaws: jmespath-java

---

Expand Down
2 changes: 2 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@
<aether.version>0.9.0.M2</aether.version>
<apache.curator.version>5.5.0</apache.curator.version>
<apache.kafka.version>3.6.1</apache.kafka.version>
<!-- when updating apache ranger, verify the usage of aws-bundle-sdk vs aws-logs-sdk
and update as needed in extensions-core/druid-ranger-security/pm.xml -->
<apache.ranger.version>2.4.0</apache.ranger.version>
<gson.version>2.10.1</gson.version>
<scala.library.version>2.13.11</scala.library.version>
Expand Down

0 comments on commit a7b2747

Please sign in to comment.