Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Update ci.yaml to setup Java 21 CI tests #2299

Merged
merged 26 commits into from
Dec 26, 2023
Merged

Conversation

ddixit14
Copy link
Contributor

@ddixit14 ddixit14 commented Dec 11, 2023

This PR aims to setup Java 21 build for the repository.

@ddixit14 ddixit14 requested a review from a team as a code owner December 11, 2023 20:22
@product-auto-label product-auto-label bot added the size: xs Pull request size is extra small. label Dec 11, 2023
@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels Dec 11, 2023
@ddixit14
Copy link
Contributor Author

build (21) failed at Integration tests step in commit 7a314dc .
image

@product-auto-label product-auto-label bot added size: xs Pull request size is extra small. and removed size: s Pull request size is small. labels Dec 11, 2023
@mpeddada1
Copy link
Contributor

@ddixit14 Please ignore the graalvm-presubmit-sdk-platform-a check for now. Currently making some changes in #2298

@ddixit14
Copy link
Contributor Author

ddixit14 commented Dec 12, 2023

After enabling the tests verbosity, I find that the tests have actually failed (not a timeout or too big size failure).
How to see what happened in the bazel runner? can I somehow get access to the bazel logs? @suztomo maybe just print out the log file?

2023-12-12T18:35:49.3675599Z (18:35:49) INFO: Build completed, 13 tests FAILED, 1259 total actions
2023-12-12T18:35:49.3797785Z //test/integration:apigeeconnect                                         PASSED in 0.3s
2023-12-12T18:35:49.3849170Z   WARNING: //test/integration:apigeeconnect: Test execution time (0.3s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3851209Z //test/integration:asset                                                 PASSED in 0.4s
2023-12-12T18:35:49.3853374Z   WARNING: //test/integration:asset: Test execution time (0.4s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3855360Z //test/integration:bigtable                                              PASSED in 0.7s
2023-12-12T18:35:49.3857296Z   WARNING: //test/integration:bigtable: Test execution time (0.7s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3859167Z //test/integration:compute                                               PASSED in 0.5s
2023-12-12T18:35:49.3861069Z   WARNING: //test/integration:compute: Test execution time (0.5s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3863026Z //test/integration:credentials                                           PASSED in 0.3s
2023-12-12T18:35:49.3864970Z   WARNING: //test/integration:credentials: Test execution time (0.3s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3867116Z //test/integration:iam                                                   PASSED in 0.3s
2023-12-12T18:35:49.3869376Z   WARNING: //test/integration:iam: Test execution time (0.3s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3871434Z //test/integration:kms                                                   PASSED in 0.9s
2023-12-12T18:35:49.3873878Z   WARNING: //test/integration:kms: Test execution time (0.9s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3876308Z //test/integration:library                                               PASSED in 0.9s
2023-12-12T18:35:49.3878641Z   WARNING: //test/integration:library: Test execution time (0.9s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3880848Z //test/integration:logging                                               PASSED in 0.7s
2023-12-12T18:35:49.3883162Z   WARNING: //test/integration:logging: Test execution time (0.7s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3885015Z //test/integration:pubsub                                                PASSED in 0.9s
2023-12-12T18:35:49.3886879Z   WARNING: //test/integration:pubsub: Test execution time (0.9s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3888734Z //test/integration:redis                                                 PASSED in 0.4s
2023-12-12T18:35:49.3890536Z   WARNING: //test/integration:redis: Test execution time (0.4s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3895351Z //test/integration:storage                                               PASSED in 0.4s
2023-12-12T18:35:49.3897527Z   WARNING: //test/integration:storage: Test execution time (0.4s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
2023-12-12T18:35:49.3899849Z //test/integration:com.google.cloud.asset.v1.AssetServiceClientTest      FAILED in 1.3s
2023-12-12T18:35:49.3935991Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.asset.v1.AssetServiceClientTest/test.log
2023-12-12T18:35:49.3943743Z //test/integration:com.google.cloud.bigtable.data.v2.BaseBigtableDataClientTest FAILED in 0.9s
2023-12-12T18:35:49.3950761Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.bigtable.data.v2.BaseBigtableDataClientTest/test.log
2023-12-12T18:35:49.3958113Z //test/integration:com.google.cloud.example.library.v1.LibraryServiceClientTest FAILED in 0.8s
2023-12-12T18:35:49.3965087Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.example.library.v1.LibraryServiceClientTest/test.log
2023-12-12T18:35:49.3972479Z //test/integration:com.google.cloud.iam.credentials.v1.IAMCredentialsClientTest FAILED in 0.7s
2023-12-12T18:35:49.3979567Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.iam.credentials.v1.IAMCredentialsClientTest/test.log
2023-12-12T18:35:49.3985599Z //test/integration:com.google.cloud.kms.v1.KeyManagementServiceClientTest FAILED in 0.6s
2023-12-12T18:35:49.3988895Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.kms.v1.KeyManagementServiceClientTest/test.log
2023-12-12T18:35:49.3991772Z //test/integration:com.google.cloud.logging.v2.ConfigClientTest          FAILED in 0.5s
2023-12-12T18:35:49.3996104Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.logging.v2.ConfigClientTest/test.log
2023-12-12T18:35:49.3999064Z //test/integration:com.google.cloud.logging.v2.LoggingClientTest         FAILED in 0.4s
2023-12-12T18:35:49.4002028Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.logging.v2.LoggingClientTest/test.log
2023-12-12T18:35:49.4004734Z //test/integration:com.google.cloud.logging.v2.MetricsClientTest         FAILED in 0.7s
2023-12-12T18:35:49.4009856Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.logging.v2.MetricsClientTest/test.log
2023-12-12T18:35:49.4012783Z //test/integration:com.google.cloud.pubsub.v1.SubscriptionAdminClientTest FAILED in 0.9s
2023-12-12T18:35:49.4016144Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.pubsub.v1.SubscriptionAdminClientTest/test.log
2023-12-12T18:35:49.4018958Z //test/integration:com.google.cloud.pubsub.v1.TopicAdminClientTest       FAILED in 0.6s
2023-12-12T18:35:49.4021696Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.pubsub.v1.TopicAdminClientTest/test.log
2023-12-12T18:35:49.4024107Z //test/integration:com.google.cloud.redis.v1beta1.CloudRedisClientTest   FAILED in 0.7s
2023-12-12T18:35:49.4026867Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.cloud.redis.v1beta1.CloudRedisClientTest/test.log
2023-12-12T18:35:49.4029344Z //test/integration:com.google.iam.v1.IAMPolicyClientTest                 FAILED in 0.8s
2023-12-12T18:35:49.4031906Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.iam.v1.IAMPolicyClientTest/test.log
2023-12-12T18:35:49.4034524Z //test/integration:com.google.storage.v2.StorageClientTest               FAILED in 0.8s
2023-12-12T18:35:49.4037103Z   /home/runner/.cache/bazel/_bazel_runner/cf5fb8c2c66b8a944ea6af7c9b248db5/execroot/gapic_generator_java/bazel-out/k8-fastbuild/testlogs/test/integration/com.google.storage.v2.StorageClientTest/test.log
2023-12-12T18:35:49.4038866Z 
2023-12-12T18:35:49.4039257Z Executed 25 out of 25 tests: 12 tests pass and 13 fail locally.
2023-12-12T18:35:49.5084632Z ##[error]Process completed with exit code 3.
2023-12-12T18:35:49.5171980Z Post job cleanup.
2023-12-12T18:35:49.6602232Z Post job cleanup.
2023-12-12T18:35:49.7429103Z [command]/usr/bin/git version
2023-12-12T18:35:49.7473824Z git version 2.43.0

@ddixit14
Copy link
Contributor Author

ddixit14 commented Dec 13, 2023

reproduced the error locally, this is the test.log output for one of the tests -

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //test/integration:com.google.cloud.asset.v1.AssetServiceClientTest
-----------------------------------------------------------------------------
JUnit4 Test Runner
java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release
	at java.base/java.lang.System.setSecurityManager(System.java:429)
	at com.google.testing.junit.runner.junit4.JUnit4Runner.installSecurityManager(JUnit4Runner.java:256)
	at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:113)
	at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:148)
	at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:75)

BazelTestRunner exiting with a return value of 1
JVM shutdown hooks (if any) will run now.
The JVM will exit once they complete.

-- JVM shutdown starting at 2023-12-12 21:04:22 --

Looks like (Google's) com.google.testing.junit.runner.junit4.JUnit4Runner does not seem to work in Java 21.

@ddixit14 ddixit14 changed the title chore: Update ci.yaml to setup Java 21 unit testing chore: Update ci.yaml to setup Java 21 CI tests Dec 13, 2023
@ddixit14
Copy link
Contributor Author

During local building, after updating to Bazel 6.4.0, got an error during gradle execution (forgot to record error). Current version does not support Java 21. Hence upgrading it to 8.4.

@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels Dec 18, 2023
@ddixit14
Copy link
Contributor Author

Got the error -

Starting a Gradle Daemon (subsequent builds will be faster)

FAILURE: Build failed with an exception.

* Where:
Build file '/private/tmp/java-storage/google-cloud-storage-v2-java/grpc-google-cloud-storage-v2-java/build.gradle' line: 21

* What went wrong:
A problem occurred evaluating project ':grpc-google-cloud-storage-v2-java'.
> Could not set unknown property 'classifier' for task ':grpc-google-cloud-storage-v2-java:javadocJar' of type org.gradle.api.tasks.bundling.Jar.

Possible root cause - Gradle 8 introduced the archiveClassifier property as a replacement for classifier. It serves the same purpose of customizing the name and location of the JAR file. Looks like we need to modify our jar generation tasks configuration to use archiveClassifier instead of classifier.

@ddixit14 ddixit14 requested review from blakeli0 and suztomo December 18, 2023 16:23
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.3-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.4-bin.zip
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this because Gradle 8.4 is the minimum version that supports Java 21?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -28,7 +28,7 @@ jobs:
mvn install -B -ntp -DskipTests -Dclirr.skip -Dcheckstyle.skip
- name: Integration Tests
run: |
bazelisk --batch test //test/integration/...
bazelisk --batch test //test/integration/... --test_verbose_timeout_warnings
Copy link
Member

@suztomo suztomo Dec 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you still need this? If yes, would you add source code comment about the significance for Java 21?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this. removed it.

@@ -28,7 +28,7 @@ jobs:
mvn install -B -ntp -DskipTests -Dclirr.skip -Dcheckstyle.skip
- name: Integration Tests
run: |
bazelisk --batch test //test/integration/...
bazelisk --batch test //test/integration/...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove unnecessary space ad the end. Same for Line 126.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git diff origin/main is your friend.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All done.

Copy link

Quality Gate Passed Quality Gate passed for 'java_showcase_integration_tests'

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@ddixit14 ddixit14 requested review from suztomo and blakeli0 December 19, 2023 15:59
@suztomo
Copy link
Member

suztomo commented Dec 20, 2023

In the release note of next release of this repository, would you (manually) add a comment about the Gradle version change?

@@ -17,12 +17,12 @@ dependencies {
}

task javadocJar(type: Jar) {
classifier = 'javadoc'
archiveClassifier = 'javadoc'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the minimum Gradle version that supports archiveClassifier? This file is used by self-service client libraries, if a customer is using an very old version of Gradle, I guess it would not work anymore for them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried 8.1 but didn't work. As suggested, I have separated out build (21) job and excluded the self service clients steps in it. Does that work?

@ddixit14 ddixit14 requested a review from blakeli0 December 26, 2023 14:35
@@ -48,7 +48,28 @@ jobs:
pushd /tmp/java-compute/google-cloud-compute-small-v1-java
./gradlew clean build publishToMavenLocal sourcesJar allJars
popd

build-java-21:
name: "build(21) except self-service clients"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you add source code comment why build (21) is special?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

Copy link

Quality Gate Passed Quality Gate passed for 'gapic-generator-java-root'

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

Copy link

Quality Gate Passed Quality Gate passed for 'gapic-generator-java-root'

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

Copy link

Quality Gate Passed Quality Gate passed for 'java_showcase_integration_tests'

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@ddixit14 ddixit14 merged commit 3d885ba into main Dec 26, 2023
34 of 38 checks passed
@ddixit14 ddixit14 deleted the ddixit14-patch-2 branch December 26, 2023 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: s Pull request size is small.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants