Use new SerializationFeature.register API with GraalVM/Mandrel >=21.3 #19520

zakkak · 2021-08-19T23:04:05Z

oracle/graal#3050 introduces a number of changes
around the internal APIs that are being used by Quarkus to register
classes for serialization thus breaking serialization support when using
21.3-dev

These changes are:

SerializationRegistry class has been moved from
com.oracle.svm.core.jdk.serialize to
com.oracle.svm.reflect.serialize
addReflections method moved from
com.oracle.svm.reflect.serialize.hosted.SerializationFeature to
com.oracle.svm.reflect.serialize.hosted.SerializationBuilder and
became private

oracle/graal#3050 also introduces
SerializationFeature.register(Class<?>... classes) which is available
as a public API (not internal) and can save us from having to keep up
with the internal APIs as new releases come out. This PR leverages the
new API to register classes for serialization when using GraalVM/Mandrel >=21.3.

Closes #19338

zakkak · 2021-08-19T23:05:34Z

CI run with 21.3-dev: https://github.com/graalvm/mandrel/actions/runs/1148784045 (please don't merge before it completes)

zakkak · 2021-08-20T09:35:13Z

The tests are looking good, but I am switching this to draft to make it use SerializationFeature.register(Class<?>... classes) with GraalVM >=21.3

galderz · 2021-08-20T09:39:08Z

Great work with the fix and thanks for spotting this early on @zakkak! 👏

zakkak · 2021-08-20T11:53:14Z

New CI run to test branch with 21.3: https://github.com/graalvm/mandrel/actions/runs/1150563667

gsmet · 2021-08-20T12:05:32Z

Yeah, thanks @zakkak ! I'll have a look early next week.

quarkus-bot · 2021-08-20T16:13:58Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `dfedfb2`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 16

Full information is available in the Build summary check run.

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.DotEnvQuarkusDevModeConfigurationTest.main line 13 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:166)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:939)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:908)
	at io.quarkus.test.devmode.util.DevModeTestUtils.getHttpResponse(DevModeTestUtils.java:130)
	at io.quarkus.gradle.devmode.QuarkusDevGradleTestBase.getHttpResponse(QuarkusDevGradleTestBase.java:126)
	at io.quarkus.gradle.devmode.QuarkusDevGradleTestBase.getHttpRes...

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/hibernate-reactive-panache

✖ io.quarkus.it.panache.reactive.PanacheFunctionalityTest.testPanacheFunctionality line 49 - More details - Source on GitHub

java.net.SocketTimeoutException: Read timed out
	at java.base/java.net.SocketInputStream.socketRead0(Native Method)
	at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:161)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:82)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:276)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeade...

gsmet

Spotted a small typo and added a question.

@zakkak I suppose we want this in 2.2?

gsmet · 2021-08-23T17:46:29Z

core/deployment/src/main/java/io/quarkus/deployment/steps/NativeImageAutoFeatureStep.java

-                ofMethod("com.oracle.svm.reflect.serialize.hosted.SerializationFeature", "addReflections", void.class,
-                        Class.class, Class.class),
-                clazz, objectClass);
+        ResultHandle addRedlectionsArgs2 = tc.newArray(Class.class, tc.load(2));


Suggested change

ResultHandle addRedlectionsArgs2 = tc.newArray(Class.class, tc.load(2));

ResultHandle addReflectionsArgs2 = tc.newArray(Class.class, tc.load(2));

gsmet · 2021-08-23T17:46:39Z

core/deployment/src/main/java/io/quarkus/deployment/steps/NativeImageAutoFeatureStep.java

+        ResultHandle addRedlectionsArgs2 = tc.newArray(Class.class, tc.load(2));
+        tc.writeArrayValue(addRedlectionsArgs2, 0, clazz);
+        tc.writeArrayValue(addRedlectionsArgs2, 1, objectClass);
+        tc.invokeVirtualMethod(invokeMethodDescriptor, addReflectionsLookupMethod, tc.loadNull(), addRedlectionsArgs2);


Suggested change

tc.invokeVirtualMethod(invokeMethodDescriptor, addReflectionsLookupMethod, tc.loadNull(), addRedlectionsArgs2);

tc.invokeVirtualMethod(invokeMethodDescriptor, addReflectionsLookupMethod, tc.loadNull(), addReflectionsArgs2);

gsmet · 2021-08-23T17:47:36Z

core/deployment/src/main/java/io/quarkus/deployment/steps/NativeImageAutoFeatureStep.java

+        ResultHandle addRedlectionsArgs2 = tc.newArray(Class.class, tc.load(2));
+        tc.writeArrayValue(addRedlectionsArgs2, 0, clazz);
+        tc.writeArrayValue(addRedlectionsArgs2, 1, objectClass);
+        tc.invokeVirtualMethod(invokeMethodDescriptor, addReflectionsLookupMethod, tc.loadNull(), addRedlectionsArgs2);


What is the tc.loadNull() supposed to be?

invoke's first argument is the this object to invoke the method on, so since this is a static method I am passing null as the first argument.

If the underlying method is static, then the specified obj argument is ignored. It may be null.
See https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/reflect/Method.html#invoke(java.lang.Object,java.lang.Object...)

oracle/graal#3050 introduces a number of changes around the internal APIs that are being used by Quarkus to register classes for serialization thus breaking serialization support when using 21.3-dev These changes are: 1. `SerializationRegistry` class has been moved from `com.oracle.svm.core.jdk.serialize` to `com.oracle.svm.reflect.serialize` 2. `addReflections` method moved from `com.oracle.svm.reflect.serialize.hosted.SerializationFeature` to `com.oracle.svm.reflect.serialize.hosted.SerializationBuilder` and became `private` oracle/graal#3050 also introduces `SerializationFeature.register(Class<?>... classes)` which is available as a public API (not internal) and can save us from having to keep up with the internal APIs as new releases come out. This PR leverages the new API to register classes for serialization when using GraalVM/Mandrel >=21.3. Closes quarkusio#19338

zakkak · 2021-08-23T18:58:39Z

Spotted a small typo and added a question.

Good catch! Fixed and answered respectively :)

@zakkak I suppose we want this in 2.2?

Yes we will eventually need it in 2.2.

gsmet · 2021-08-24T11:44:42Z

Thanks, nice work!

quarkus-bot bot added the area/core label Aug 19, 2021

zakkak added the area/native-image label Aug 19, 2021

zakkak marked this pull request as draft August 20, 2021 09:34

galderz approved these changes Aug 20, 2021

View reviewed changes

zakkak force-pushed the fix-serialization-support branch 2 times, most recently from e7f1750 to dfedfb2 Compare August 20, 2021 11:51

zakkak changed the title ~~SerializationRegistry and addReflections moved in GraalVM 21.3-dev~~ Use new SerializationFeature.register API with GraalVM/Mandrel >=21.3 Aug 20, 2021

zakkak marked this pull request as ready for review August 20, 2021 11:53

gsmet requested changes Aug 23, 2021

View reviewed changes

gsmet added the triage/backport? label Aug 23, 2021

zakkak force-pushed the fix-serialization-support branch from dfedfb2 to 4921b9f Compare August 23, 2021 18:57

gsmet approved these changes Aug 24, 2021

View reviewed changes

gsmet merged commit 77e8f88 into quarkusio:main Aug 24, 2021

quarkus-bot bot added this to the 2.3 - main milestone Aug 24, 2021

zakkak deleted the fix-serialization-support branch August 24, 2021 11:49

gsmet modified the milestones: 2.3 - main, 2.2.0.Final Aug 24, 2021

gsmet removed the triage/backport? label Aug 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use new SerializationFeature.register API with GraalVM/Mandrel >=21.3 #19520

Use new SerializationFeature.register API with GraalVM/Mandrel >=21.3 #19520

zakkak commented Aug 19, 2021 •

edited

Loading

zakkak commented Aug 19, 2021

zakkak commented Aug 20, 2021

galderz commented Aug 20, 2021

zakkak commented Aug 20, 2021

gsmet commented Aug 20, 2021

quarkus-bot bot commented Aug 20, 2021 •

edited

Loading

Failing Jobs - Building `dfedfb2`

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/hibernate-reactive-panache

gsmet left a comment

gsmet Aug 23, 2021

gsmet Aug 23, 2021

gsmet Aug 23, 2021

zakkak Aug 23, 2021 •

edited

Loading

zakkak commented Aug 23, 2021

gsmet commented Aug 24, 2021

	ResultHandle addRedlectionsArgs2 = tc.newArray(Class.class, tc.load(2));
	ResultHandle addReflectionsArgs2 = tc.newArray(Class.class, tc.load(2));

	tc.invokeVirtualMethod(invokeMethodDescriptor, addReflectionsLookupMethod, tc.loadNull(), addRedlectionsArgs2);
	tc.invokeVirtualMethod(invokeMethodDescriptor, addReflectionsLookupMethod, tc.loadNull(), addReflectionsArgs2);

Use new SerializationFeature.register API with GraalVM/Mandrel >=21.3 #19520

Use new SerializationFeature.register API with GraalVM/Mandrel >=21.3 #19520

Conversation

zakkak commented Aug 19, 2021 • edited Loading

zakkak commented Aug 19, 2021

zakkak commented Aug 20, 2021

galderz commented Aug 20, 2021

zakkak commented Aug 20, 2021

gsmet commented Aug 20, 2021

quarkus-bot bot commented Aug 20, 2021 • edited Loading

Failing Jobs - Building dfedfb2

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/hibernate-reactive-panache

gsmet left a comment

Choose a reason for hiding this comment

gsmet Aug 23, 2021

Choose a reason for hiding this comment

gsmet Aug 23, 2021

Choose a reason for hiding this comment

gsmet Aug 23, 2021

Choose a reason for hiding this comment

zakkak Aug 23, 2021 • edited Loading

Choose a reason for hiding this comment

zakkak commented Aug 23, 2021

gsmet commented Aug 24, 2021

zakkak commented Aug 19, 2021 •

edited

Loading

quarkus-bot bot commented Aug 20, 2021 •

edited

Loading

Failing Jobs - Building `dfedfb2`

zakkak Aug 23, 2021 •

edited

Loading