Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#351 Update metadata to be compatible with Java 17 #424

Merged
merged 1 commit into from
Oct 18, 2024
Merged

#351 Update metadata to be compatible with Java 17 #424

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions DRAFT_RELEASE_NOTES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,8 @@ the `jackson-mapper-asl` dependency artifact in the root pom.xml file to enable

## Conditional Steps

### Data Access Dependency Update
Data Access no longer needs to explicitly set the `kafka-clients` dependency. It can be removed along with its
exclusion from the `quarkus-smallrye-reactive-messaging-kafka` dependency.
### Kafka Clients Dependency Update
Data Access and the Java testing module (`<YOUR_PROJECT>-tests/<YOUR_PROJECT>-tests-java/pom.xml`) no longer need to explicitly set the `kafka-clients` dependency. It can be removed along with its exclusion from the `quarkus-smallrye-reactive-messaging-kafka` dependency.

### For projects that have customized the Spark Operator Service Account permissions
The service account for the pipeline invocation service is now separated from spark operator and configured solely for the service.
Expand Down
15 changes: 10 additions & 5 deletions build-parent/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,9 @@
<version.jakarta.activation.api>2.1.2</version.jakarta.activation.api>
<version.javax.servlet>3.1.0</version.javax.servlet>

<!-- Spark Default Dependencies. See `spark-*` profiles below for alternative sets -->
<!-- Spark Default Dependencies. See `spark-*` profiles below for alternative sets -->
<version.spark>3.5.2</version.spark>
<version.scala>2.12.13</version.scala>
<version.scala>2.12.20</version.scala>
<version.scala.minor>2.12</version.scala.minor>
<version.delta>3.2.1</version.delta>
<version.elasticsearch>8.9.0</version.elasticsearch>
Expand Down Expand Up @@ -867,9 +867,14 @@
<TESTCONTAINERS_RYUK_DISABLED>${TESTCONTAINERS_RYUK_DISABLED}</TESTCONTAINERS_RYUK_DISABLED>
<!--<TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX>YOUR MIRROR OR INTENRAL REPO GOES HERE - e.g., nexus.foobar.com/</TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX>-->
</environmentVariables>
<!-- Spark needs to use an internal API that has been made inaccessible by default. This
exposes it -->
<argLine>--add-exports java.base/sun.nio.ch=ALL-UNNAMED</argLine>
<!-- Spark needs to use an internal API that has been made inaccessible by default, this is resolved by add-exports.
Apache Hive currently does not support java 17, so the add-opens workaround is used to allow the illegal reflection
call until Hive updates their compatible Java versions.
https://stackoverflow.com/questions/41265266/how-to-solve-inaccessibleobjectexception-unable-to-make-member-accessible-m?rq=2-->
<argLine>
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
--add-opens java.base/java.net=ALL-UNNAMED
</argLine>
</configuration>
</plugin>
<plugin>
Expand Down
6 changes: 6 additions & 0 deletions extensions/extensions-docker/aissemble-metadata/src/main/resources/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,9 @@ COPY --chown=default target/quarkus-app/lib/ $JAVA_APP_DIR/lib/
COPY --chown=default target/quarkus-app/*.jar $JAVA_APP_DIR/
COPY --chown=default target/quarkus-app/app/ $JAVA_APP_DIR/app/
COPY --chown=default target/quarkus-app/quarkus/ $JAVA_APP_DIR/quarkus/

# Spark needs to use an internal API that has been made inaccessible by default, this is resolved by add-exports.
# Apache Hive currently does not support java 17, so the add-opens workaround is used to allow the illegal reflection
# call until Hive updates their compatible Java versions.
# https://stackoverflow.com/questions/41265266/how-to-solve-inaccessibleobjectexception-unable-to-make-member-accessible-m?rq=2
ENV JAVA_OPTS_APPEND="${JAVA_OPTS_APPEND} --add-exports java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED"
2 changes: 1 addition & 1 deletion extensions/extensions-docker/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
<module>aissemble-fastapi</module>
<module>aissemble-hive-service</module>
<module>aissemble-jenkins</module>
<!--<module>aissemble-metadata</module>-->
<module>aissemble-metadata</module>
<module>aissemble-nvidia</module>
<module>aissemble-policy-decision-point</module>
<module>aissemble-spark</module>
Expand Down
70 changes: 38 additions & 32 deletions extensions/extensions-metadata-service/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,11 +112,7 @@
<artifactId>extensions-data-delivery-spark</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.12</artifactId>
<version>${version.spark}</version>
</dependency>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I: Removed as this dependency is defined in the extensions-data-delivery-spark module along with the other spark-* dependencies.


<!-- Downgrade Scala 2.13 (from Quarkus) to Scala 2.12 -->
<dependency>
<groupId>org.scala-lang</groupId>
Expand All @@ -141,44 +137,61 @@
<version>${version.javax.servlet}</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${version.mysql-connector}</version>
<groupId>io.smallrye.reactive</groupId>
<artifactId>smallrye-reactive-messaging-api</artifactId>
<version>${version.smallrye.reactive.messaging}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-reactive-messaging</artifactId>
<groupId>org.eclipse.microprofile.metrics</groupId>
<artifactId>microprofile-metrics-api</artifactId>
<version>5.1.1</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kafka-client</artifactId>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-reactive-messaging-kafka</artifactId>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy</artifactId>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-parameter-names</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>jakarta.enterprise</groupId>
<artifactId>jakarta.enterprise.cdi-api</artifactId>
<version>${version.jakarta.cdi}</version>
</dependency>
<dependency>
<groupId>jakarta.inject</groupId>
<artifactId>jakarta.inject-api</artifactId>
<version>${version.jakarta.inject}</version>
</dependency>
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
<version>${version.jakarta.wr.rs}</version>
</dependency>

<!-- Quarkus Dependencies -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-openapi</artifactId>
<exclusions>
<exclusion>
<groupId>org.jboss.slf4j</groupId>
<artifactId>slf4j-jboss-logmanager</artifactId>
</exclusion>
</exclusions>
<artifactId>quarkus-kafka-client</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy-jackson</artifactId>
</dependency>
<dependency>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-reactive-messaging-kafka</artifactId>
</dependency>

<!-- Testing Dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
Expand All @@ -187,7 +200,6 @@
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
</dependency>

<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
Expand All @@ -197,12 +209,6 @@
<artifactId>smallrye-reactive-messaging-in-memory</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.masterthought</groupId>
<artifactId>cucumber-reporting</artifactId>
<version>${version.cucumber.reporting.plugin}</version>
<scope>test</scope>
</dependency>
</dependencies>

</project>
2 changes: 1 addition & 1 deletion extensions/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
<module>extensions-docker</module>
<module>extensions-alerting</module>
<module>extensions-messaging</module>
<!--<module>extensions-metadata-service</module>-->
<module>extensions-metadata-service</module>
<module>extensions-helm</module>
<module>extensions-pipeline-invocation-service</module>
</modules>
Expand Down
6 changes: 0 additions & 6 deletions ...ources/archetype-resources/__rootArtifactId__-tests/__rootArtifactId__-tests-java/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,12 +120,6 @@
<version>${version.resteasy}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.7.2</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
Expand Down
62 changes: 25 additions & 37 deletions foundation/foundation-metadata/foundation-metadata-producer/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,65 +11,53 @@
<artifactId>foundation-metadata-producer</artifactId>
<name>aiSSEMBLE::Foundation::Metadata::Producer</name>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.boozallen.aissemble</groupId>
<artifactId>bom-component</artifactId>
<version>${project.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.boozallen.aissemble</groupId>
<artifactId>aissemble-quarkus-bom</artifactId>
<version>${project.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>io.smallrye.reactive</groupId>
<artifactId>smallrye-reactive-messaging-api</artifactId>
<version>${version.smallrye.reactive.messaging}</version>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>foundation-metadata-core</artifactId>
<version>${project.version}</version>
</dependency>

<!-- Logging Dependencies (from build-parent pom) -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>${version.jackson}</version>
</dependency>

<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kafka-client</artifactId>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-parameter-names</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>com.boozallen.aissemble</groupId>
<artifactId>foundation-messaging-java</artifactId>
<version>${project.version}</version>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>3.8.0</version>
</dependency>
<dependency>
<groupId>io.smallrye.reactive</groupId>
<artifactId>smallrye-reactive-messaging-api</artifactId>
<version>${version.smallrye.reactive.messaging}</version>
</dependency>
<dependency>
<groupId>jakarta.enterprise</groupId>
<artifactId>jakarta.enterprise.cdi-api</artifactId>
<version>${version.jakarta.cdi}</version>
</dependency>
<dependency>
<groupId>jakarta.inject</groupId>
<artifactId>jakarta.inject-api</artifactId>
<version>${version.jakarta.inject}</version>
</dependency>
</dependencies>

<build>
Expand Down