Feature: Upgrade IT infrastructure for JDK 17

[cwoods-cpointe]

Description

Support for Java 11 has ended so we want to ensure aiSSEMBLE is compatible with Java 17. This ticket will focus on the integration test infrastructure.

DOD

• All aiSSEMBLE and downstream testing modules build on Java 17
• Downstream apps can create IT modules that build using Java 17
• Create any Baton migrations necessary
• Downstream apps can run the integration tests
  • Note integration tests are not stable at the moment (See references)

Test Strategy/Script

test 1

• Create a 1.10.0 downstream

mvn archetype:generate -B -DarchetypeGroupId=com.boozallen.aissemble \
                          -DarchetypeArtifactId=foundation-archetype \
                          -DarchetypeVersion=1.10.0-SNAPSHOT \
                          -DartifactId=test-project\
                          -DgroupId=org.test \
                          -DprojectName='Test' \
                          -DprojectGitUrl=test.org/test-project \
&& cd test-project

• Add spark pipelines into MDA
  • resources.zip
• Keep building the project with mvn clean install and resolve all manual actions
• Build the application with the integration-test profile
• Verify:
  • All pods start up healthy
  • The spark history and thrift service pods are named spark-history and thrift-server respectively
  • The test pod waits for spark-operator and thrift-server before attempting to start
  • The test pod logs show the Cucumber scenario was ran
• After the test pods come down, Deploy the project with tilt up
• Verify all pods come up
• Verify data access can connect to the thrift-server by running. Will fail to find the data but we are just testing the connection

URL - localhost:8081/graphql
Body - query test {Person(table: "person") {name}}

Test 2 - Migration

• Create a 1.9.2 downstream project

mvn archetype:generate -B -DarchetypeGroupId=com.boozallen.aissemble \
                          -DarchetypeArtifactId=foundation-archetype \
                          -DarchetypeVersion=1.9.2 \
                          -DartifactId=test-project-192\
                          -DgroupId=org.test \
                          -DprojectName='Test' \
                          -DprojectGitUrl=test.org/test-project \
&& cd test-project-192

• Repeatedly build with mvn clean install and resolve manual actions until there are no more to resolve
• Verify the test-project-192-tests/test-project-192-tests-docker/**/Dockerfile contains the java 11 image
• Upgrade aiSSEMBLE to 1.10.0-SNAPSHOT
• Build the test-project-192-tests module
• Verify the test-project-192-tests/test-project-192-tests-docker/**/Dockerfile contains the java 17 image

References/Additional Context

IT modules failing - #191 (comment)

• remove zookeeper from -tests-helm Tiltfile
• remove boozallen/ prefix for the pdp docker build call ref in the -tests-helm Tiltfile
• remove ${DOCKER_BASELINE_REPO_ID} from ${DOCKER_BASELINE_REPO_ID}/openjdk:11-slim in the tests dockerfile
• set the docker image name properly in -tests/-tests-helm/src/main/resources/test-chart/values.yaml (currently is incorrectly set to example-tests-helm but should be example-tests-docker
• resolve breaking pipeline-test pod error that occurs after the above fixes are incorporated (the jar that is copied into the *-tests-docker Dockerfile is not accessible, likely can be fixed with chmod)
• Jar file executed but finds no main manifest
• (nice to have) improve *-tests-helm maven build log output to better denote integration test hanging due to a test pod error / failed deployment
• (nice to have and not exclusive to integration-test) investigate INFO --- [et.reactor-0] localstack.request.aws : AWS s3.HeadObject => 404 (NoSuchKey) repeated logging with s3-local Tilt output

[cwoods-cpointe]

Downstream projects are able to start the integration tests, deploy the IT apps, kick off the cucumber tests.
But the test fails because a known issue with the IT infrastructure. Will spit this ticket here as all the Java17 work is completed and the follow on will tackle running pipelines in the ITs. Just need to create one Baton migration