Adjust LCMS substitutions for GraalVM JDK 17 #19534

gsmet · 2021-08-20T10:22:38Z

Fixes #19356

@zakkak would you have any way to test that? I did that completely in the dark...

zakkak · 2021-08-20T12:06:55Z

@zakkak would you have any way to test that? I did that completely in the dark...

We currently don't have any JDK 17 builds in our pipelines, but @Karm is working on it. Maybe he can help testing it.

AFAIK we also don't have any way to stress test LCMS support in JDK 11 either.

jaikiran · 2021-08-21T15:08:35Z

The JVMCI JDK17 releases/builds are made available at https://github.com/graalvm/labs-openjdk-17. I used the labsjdk-ce-17-jvmci-21.3-b01 to build the latest GraalVM community edition from the GraalVM repo. I then used it to test the getting-started quickstart of quarkus to build and use the native image for it.

An additional change was needed in this PR because it was failing during native image build with:

Error: Could not find target method: static void io.quarkus.runtime.graal.Target_sun_java2d_cmm_lcms_LCMS_JDK17.getProfileDataNative(long,byte[])
com.oracle.svm.core.util.UserError$UserException: Could not find target method: static void io.quarkus.runtime.graal.Target_sun_java2d_cmm_lcms_LCMS_JDK17.getProfileDataNative(long,byte[])

In the latest JDK17 version, this method now takes just one parameter. I've fixed that part and updated this PR. With this change, the basic test and building of the getting-started quickstart against a JDK17 built GraalVM and GraalVM 21.1.0 (Java 11) release both went fine.

Once the CI jobs of this PR come out fine, I think it probably is safe to merge this.

quarkus-bot · 2021-08-21T19:24:15Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `f2b7390`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 11
⌛	JVM Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
✔️	JVM Tests - JDK 16
⌛	Maven Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✖	MicroProfile TCKs Tests	`Verify`	Test failures	Logs	Raw logs
⌛	Native Tests - Data2	`Set up JDK 11`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Data4	`Set up JDK 11`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Data6	`Set up JDK 11`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

✖ io.quarkus.gradle.AddExtensionToModuleInMultiModuleKtsProjectTest.testBasicMultiModuleBuild line 35 - More details - Source on GitHub

java.lang.AssertionError: 

Expecting actual:
  "plugins {
    id("io.quarkus")
}

val quarkusPlatformGroupId: String by project
val quarkusPlatformArtifactId: String by project
val quarkusPlatformVersion: String by project

dependencies {
    implementation(project(":common"))
    implementation("io.quarkus:quarkus-resteasy")

    implementation(enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}"))
}"
to contain:
  "implementation("io.quarkus:quarkus-openshift")" 
	at io.quarkus.gradle.AddExtensionToModuleInMultiModuleKtsProjectTest.testBasicMultiModuleBuild(AddExtensionToModuleInMultiModuleKtsProjectTest.java:35)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.M...

⚙️ Maven Tests - JDK 11 Windows #

📦 integration-tests/maven

✖ io.quarkus.maven.it.BuildIT.testClassLoaderLinkageError line 79 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0
 but was: 1
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at io.quarkus.maven.it.BuildIT.build(BuildIT.java:175)
	at io.quarkus.maven.it.BuildIT.build(BuildIT.java:160)
	at io.quarkus.maven.it.BuildIT.testClassLoaderLinkageError(BuildIT.java:79)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invo...

⚙️ MicroProfile TCKs Tests #

📦 tcks/resteasy-reactive/target/testsuite/tests

✖ com.sun.ts.tests.jaxrs.spec.filter.exception.JAXRSClient0019.throwSecondExceptionFromMapperFirstFromPostMatchingFilterTest line 240 - More details - Source on GitHub

com.sun.ts.tests.jaxrs.common.JAXRSCommonClient$Fault: [JAXRSCommonClient] null failed!  Check output for cause of failure.
	at com.sun.ts.tests.jaxrs.spec.filter.exception.JAXRSClient0019.throwSecondExceptionFromMapperFirstFromPostMatchingFilterTest(JAXRSClient0019.java:240)
Caused by: com.sun.ts.tests.common.webclient.TestFailureException: 
[FATAL] Unexpected failure during test execution.
Client client = ClientFactory.newClient();
WebTarget target = client.target("http://localhost:8081/jaxrs_spec_filter_exception_web/resource/echo");
Invocation.Builder builder;
builder = target.request("*/*");
builder.header("PostMatchingFilterThrowsException","ThrowExceptionOnceAgain");
Invocation i;
i=builder.build("POST",Entity.entity(0,*/*);
i.invoke();
java.lang.NullPointerException
	at com.sun.ts.tests.jaxrs.spec.filter.exception.JAXRSClient0019.throwSecondExceptionFromMapperFirstFromPostMatchingFilterTest(JAXRSClient0019.java:240)
Caused by: javax.ws.rs.ProcessingException: java.lang.NullP...

Fixes quarkusio#19356

gsmet · 2021-08-23T13:29:37Z

Pushed a rebase as there were too many CI failures.

quarkus-bot · 2021-08-23T17:21:31Z

Failing Jobs - Building `257df1e`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
⌛	MicroProfile TCKs Tests	`Verify`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.DotEnvQuarkusDevModeConfigurationTest.main line 13 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:166)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:939)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:908)
	at io.quarkus.test.devmode.util.DevModeTestUtils.getHttpResponse(DevModeTestUtils.java:130)
	at io.quarkus.gradle.devmode.QuarkusDevGradleTestBase.getHttpResponse(QuarkusDevGradleTestBase.java:126)
	at io.quarkus.gradle.devmode.QuarkusDevGradleTestBase.getHttpRes...

gsmet · 2021-08-23T17:44:57Z

@jaikiran can I have your approval? :)

jaikiran

This looks good and more importantly doesn't regress against existing Java/Native image versions. Approved :)

Karm · 2021-08-24T23:45:26Z

@gsmet @zakkak I can confirm it's fixed ✔️

Running with JDK 17 based Mandrel 21.3-SNAPSHOT:

Quarkus 2.2.0.CR1

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on native-image 21.3.0-deve98ae39603 Mandrel Distribution (Java Version 17-internal+0-adhoc.tester.jdk17)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /home/karm/workspaceRH/mandrel-release/mandrel-java17-21.3.0-deve98ae39603/bin/native-image -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -jar code-with-quarkus-1.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:NativeLinkerOption=-no-pie -H:-UseServiceLoaderFeature -H:+StackTrace -H:-ParseOnce code-with-quarkus-1.0.0-SNAPSHOT-runner
[code-with-quarkus-1.0.0-SNAPSHOT-runner:211626]    classlist:   1,187.86 ms,  0.93 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:211626]        setup:     471.32 ms,  0.93 GB
Error: Static modifier mismatch: private long io.quarkus.runtime.graal.Target_sun_java2d_cmm_lcms_LCMS.loadProfileNative(byte[],java.lang.Object), private static native long sun.java2d.cmm.lcms.LCMS.loadProfileNative(byte[],java.lang.Object)
com.oracle.svm.core.util.UserError$UserException: Static modifier mismatch: private long io.quarkus.runtime.graal.Target_sun_java2d_cmm_lcms_LCMS.loadProfileNative(byte[],java.lang.Object), private static native long sun.java2d.cmm.lcms.LCMS.loadProfileNative(byte[],java.lang.Object)
	at com.oracle.svm.core.util.UserError.abort(UserError.java:68)
	at com.oracle.svm.core.util.UserError.guarantee(UserError.java:96)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.findOriginalMethod(AnnotationSubstitutionProcessor.java:725)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleMethodInAliasClass(AnnotationSubstitutionProcessor.java:368)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleAliasClass(AnnotationSubstitutionProcessor.java:340)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:310)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:266)
	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:944)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:873)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:531)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:492)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:400)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:566)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:122)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:596)
[code-with-quarkus-1.0.0-SNAPSHOT-runner:211626]      [total]:   1,695.69 ms,  0.93 GB
# Printing build artifacts to: /tmp/code-with-quarkus/target/code-with-quarkus-1.0.0-SNAPSHOT-native-image-source-jar/code-with-quarkus-1.0.0-SNAPSHOT-runner.build_artifacts.txt
Error: Image build request failed with exit status 1

Quarkus 999-SNAPSHOT `0110329`

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on native-image 21.3.0-deve98ae39603 Mandrel Distribution (Java Version 17-internal+0-adhoc.tester.jdk17)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /home/karm/workspaceRH/mandrel-release/mandrel-java17-21.3.0-deve98ae39603/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -jar code-with-quarkus-1.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:NativeLinkerOption=-no-pie -H:-UseServiceLoaderFeature -H:+StackTrace -H:-ParseOnce code-with-quarkus-1.0.0-SNAPSHOT-runner
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]    classlist:   1,121.82 ms,  0.96 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]        (cap):     466.06 ms,  0.94 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]        setup:   1,790.03 ms,  0.94 GB
The bundle named: messages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
01:32:42,101 INFO  [org.jbo.threads] JBoss Threads version 3.4.2.Final
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]     (clinit):     480.00 ms,  4.27 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]   (typeflow):   2,433.94 ms,  4.27 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]    (objects):  15,841.08 ms,  4.27 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]   (features):     788.91 ms,  4.27 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]     analysis:  20,550.71 ms,  4.27 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]     universe:   1,647.85 ms,  4.27 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]      (parse):   3,751.59 ms,  4.79 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]     (inline):   4,122.05 ms,  6.11 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]    (compile):  19,457.91 ms,  6.14 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]      compile:  29,120.34 ms,  6.14 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]        image:   2,718.64 ms,  6.14 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]        write:     427.25 ms,  6.14 GB
[code-with-quarkus-1.0.0-SNAPSHOT-runner:210174]      [total]:  57,626.01 ms,  6.14 GB
# Printing build artifacts to: /tmp/code-with-quarkus/target/code-with-quarkus-1.0.0-SNAPSHOT-native-image-source-jar/code-with-quarkus-1.0.0-SNAPSHOT-runner.build_artifacts.txt
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] objcopy --strip-debug code-with-quarkus-1.0.0-SNAPSHOT-runner
[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 60038ms

Although if you go beyond Hello World, there will be more substitutions adjustments needed: #19633

quarkus-bot bot added the area/core label Aug 20, 2021

gsmet mentioned this pull request Aug 20, 2021

sun.java2d.cmm.lcms.LCMS#setTagDataNative substitution is broken on JDK17 #19356

Closed

jaikiran force-pushed the lcms-jdk17 branch from 8ff5ac4 to f2b7390 Compare August 21, 2021 15:02

Adjust LCMS substitutions for GraalVM JDK 17

257df1e

Fixes quarkusio#19356

gsmet force-pushed the lcms-jdk17 branch from f2b7390 to 257df1e Compare August 23, 2021 13:29

zakkak added the area/native-image label Aug 23, 2021

jaikiran approved these changes Aug 24, 2021

View reviewed changes

gsmet merged commit 610a35a into quarkusio:main Aug 24, 2021

quarkus-bot bot added this to the 2.3 - main milestone Aug 24, 2021

This was referenced Aug 25, 2021

Use 21.2.0.java11 for Windows Native quarkus-qe/quarkus-test-framework#220

Merged

Update GraalVM version to 21.2 for Windows Native quarkus-qe/quarkus-test-suite#197

Merged

zakkak mentioned this pull request Aug 25, 2021

#19534 broke backwards compatibility with GraalVM/Mandrel 20.3 #19645

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adjust LCMS substitutions for GraalVM JDK 17 #19534

Adjust LCMS substitutions for GraalVM JDK 17 #19534

gsmet commented Aug 20, 2021

zakkak commented Aug 20, 2021 •

edited

Loading

jaikiran commented Aug 21, 2021 •

edited

Loading

quarkus-bot bot commented Aug 21, 2021 •

edited

Loading

Failing Jobs - Building `f2b7390`

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

⚙️ Maven Tests - JDK 11 Windows #

📦 integration-tests/maven

⚙️ MicroProfile TCKs Tests #

📦 tcks/resteasy-reactive/target/testsuite/tests

gsmet commented Aug 23, 2021

quarkus-bot bot commented Aug 23, 2021

gsmet commented Aug 23, 2021

jaikiran left a comment

Karm commented Aug 24, 2021

Adjust LCMS substitutions for GraalVM JDK 17 #19534

Adjust LCMS substitutions for GraalVM JDK 17 #19534

Conversation

gsmet commented Aug 20, 2021

zakkak commented Aug 20, 2021 • edited Loading

jaikiran commented Aug 21, 2021 • edited Loading

quarkus-bot bot commented Aug 21, 2021 • edited Loading

Failing Jobs - Building f2b7390

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

⚙️ Maven Tests - JDK 11 Windows #

📦 integration-tests/maven

⚙️ MicroProfile TCKs Tests #

📦 tcks/resteasy-reactive/target/testsuite/tests

gsmet commented Aug 23, 2021

quarkus-bot bot commented Aug 23, 2021

Failing Jobs - Building 257df1e

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

gsmet commented Aug 23, 2021

jaikiran left a comment

Choose a reason for hiding this comment

Karm commented Aug 24, 2021

Quarkus 2.2.0.CR1

Quarkus 999-SNAPSHOT 0110329

zakkak commented Aug 20, 2021 •

edited

Loading

jaikiran commented Aug 21, 2021 •

edited

Loading

quarkus-bot bot commented Aug 21, 2021 •

edited

Loading

Failing Jobs - Building `f2b7390`

Failing Jobs - Building `257df1e`

Quarkus 999-SNAPSHOT `0110329`