Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use new RuntimeResourceSupport public API with GraalVM >= 22.3 #27728

Merged
merged 4 commits into from
Sep 6, 2022
Merged

Use new RuntimeResourceSupport public API with GraalVM >= 22.3 #27728

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1573ce4
Drop support for "legacy" (pre 21.1) localization feature
zakkak Sep 5, 2022
f35f7de
Use io.quarkus.runtime.ReflectionUtil#lookupMethod
zakkak Sep 5, 2022
d94509a
Use new RuntimeResourceSupport public API with GraalVM >= 22.3
zakkak Sep 5, 2022
85da3b5
Wrap trueBlock and falseBlock invocations in try-with-resources blocks
zakkak Sep 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
254 changes: 141 additions & 113 deletions core/deployment/src/main/java/io/quarkus/deployment/steps/NativeImageFeatureStep.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import static io.quarkus.gizmo.MethodDescriptor.ofMethod;

import java.lang.reflect.AccessibleObject;
import java.lang.reflect.Constructor;
import java.lang.reflect.Executable;
import java.lang.reflect.Field;
Expand All @@ -20,6 +19,7 @@
import org.graalvm.nativeimage.hosted.Feature;
import org.graalvm.nativeimage.hosted.RuntimeClassInitialization;
import org.graalvm.nativeimage.hosted.RuntimeReflection;
import org.graalvm.nativeimage.impl.ConfigurationCondition;

import io.quarkus.deployment.annotations.BuildProducer;
import io.quarkus.deployment.annotations.BuildStep;
Expand Down Expand Up @@ -99,8 +99,8 @@ public class NativeImageFeatureStep {
static final String BEFORE_ANALYSIS_ACCESS = Feature.BeforeAnalysisAccess.class.getName();
static final String DURING_SETUP_ACCESS = Feature.DuringSetupAccess.class.getName();
static final String DYNAMIC_PROXY_REGISTRY = "com.oracle.svm.core.jdk.proxy.DynamicProxyRegistry";
static final String LEGACY_LOCALIZATION_FEATURE = "com.oracle.svm.core.jdk.LocalizationFeature";
static final String LOCALIZATION_FEATURE = "com.oracle.svm.core.jdk.localization.LocalizationFeature";
static final String RUNTIME_RESOURCE_SUPPORT = "org.graalvm.nativeimage.impl.RuntimeResourceSupport";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zakkak I thought the public API is org.graalvm.nativeimage.hosted.RuntimeResourceAccess?

Copy link
Contributor Author

@zakkak zakkak Sep 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To my understanding they are both public APIs that will be shipped in graal-sdk.

org.graalvm.nativeimage.hosted.RuntimeResourceAccess is part of the org.graalvm.nativeimage.hosted package and org.graalvm.nativeimage.impl.RuntimeResourceSupport is part of the org.graalvm.nativeimage package which are both included in graal-sdk.

The reason I went with RuntimeResourceSupport instead of RuntimeResourceAccess is that the former doesn't require to specify a module for the resources, and in Quarkus we are not using modules.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. When I see impl in an API class it raises a red flag to me. While it's true RuntimeResourceSupport doesn't require a module, we should get away with ClassLoader.getSystemClassLoader().getUnnamedModule() and use the RuntimeResourceAccess. In fact, RuntimeResourceAccess uses RuntimeResourceSupport which supports the idea that using the former will be a more robust solution. Also, you don't need to do the ImageSingleton handling yourself and it's in line with what we do in #27690. Your mileage may vary.

Copy link
Contributor Author

@zakkak zakkak Sep 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tough part is not about getting the unnamed module but getting the named module for JDK classes, e.g.

quarkus/core/deployment/src/main/java/io/quarkus/deployment/steps/ResourceBundleStep.java

Line 15 in 1f87a53

return new NativeImageResourceBundleBuildItem("sun.security.util.Resources");

Suggestions are welcome :)

Copy link
Contributor

@jerboaa jerboaa Sep 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sun.security.util.Resources is part of java.base so how about? int.class.getModule() The correct way is to use Foo.class.getModule() of course, but you might have a chicken/egg problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just an example, there are more such cases and while getting the module might not be that hard what makes it hard is the fact that we need to get the module in the generated native-image Feature. So to make use of the RuntimeResourceAccess we need to:

  1. Extend NativeImageResourceBundleBuildItem to contain information about the module
  2. Generate the necessary code, using gizmo, to programmatically get that module and pass it to RuntimeResourceAccess in the generated native-image Feature.

Since RuntimeResourceSupport appears to be public API I don't see the need to do the above which will make the corresponding code even more complex.

As an alternative I believe that if we really don't want to use RuntimeResourceSupport we should suggest the augmentation of RuntimeResourceAccess to support registering resources without the module name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently, I was wrong. org.graalvm.nativeimage.impl is indeed not supposed to be public API. But we already open it in

quarkus/core/deployment/src/main/java/io/quarkus/deployment/steps/NativeImageFeatureStep.java

Lines 130 to 132 in 92fba0a

// required in order to access org.graalvm.nativeimage.impl.RuntimeSerializationSupport and org.graalvm.nativeimage.impl.ConfigurationCondition
features.produce(
new JPMSExportBuildItem("org.graalvm.sdk", "org.graalvm.nativeimage.impl", GraalVM.Version.VERSION_22_1_0));
which led me to thinking it's accessible by default.

I will need to prepare a follow up fix for this. Thanks for questioning this and making have another look @jerboaa!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zakkak Thanks for coming back to this! I was meaning to come back to this discussion, but it dropped on the floor (for me). Yes, we should revisit this. IMHO, there might be latent bugs by not passing in Modules resources belong to.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIP PR taking care of the above: #28093

public static final MethodDescriptor WEAK_REFLECTION_REGISTRATION = MethodDescriptor.ofMethod(WeakReflection.class,
"register", void.class, Feature.BeforeAnalysisAccess.class, Class.class, boolean.class, boolean.class,
boolean.class);
Expand Down Expand Up @@ -288,55 +288,63 @@ public void write(String s, byte[] bytes) {

/* Resource includes and excludes */
if (!resourcePatterns.isEmpty()) {
// Needed to access com.oracle.svm.core.configure.ResourcesRegistry.*
// Needed to access com.oracle.svm.core.configure.ResourcesRegistry.* in GraalVM 22.2
exports.produce(new JPMSExportBuildItem("org.graalvm.nativeimage.builder", "com.oracle.svm.core.configure",
GraalVM.Version.VERSION_22_1_0));
GraalVM.Version.VERSION_22_1_0, GraalVM.Version.VERSION_22_3_0));

ResultHandle resourcesRegistrySingleton = overallCatch.invokeStaticMethod(IMAGE_SINGLETONS_LOOKUP,
overallCatch.loadClassFromTCCL("com.oracle.svm.core.configure.ResourcesRegistry"));
TryBlock tc = overallCatch.tryBlock();

ResultHandle currentThread = tc.invokeStaticMethod(ofMethod(Thread.class, "currentThread", Thread.class));
ResultHandle tccl = tc.invokeVirtualMethod(ofMethod(Thread.class, "getContextClassLoader", ClassLoader.class),
currentThread);
AssignableResultHandle resourcesArgTypes = tc.createVariable(Class[].class);
ResultHandle resourcesArgTypes = tc.marshalAsArray(Class.class, tc.loadClassFromTCCL(ConfigurationCondition.class),
tc.loadClassFromTCCL(String.class));
AssignableResultHandle resourcesArgs = tc.createVariable(Object[].class);
AssignableResultHandle argsIndex = tc.createVariable(int.class);

ResultHandle argTypes = tc.newArray(Class.class, tc.load(2));
ResultHandle configurationConditionClass = tc.invokeStaticMethod(FOR_NAME,
tc.load("org.graalvm.nativeimage.impl.ConfigurationCondition"),
tc.load(false), tccl);
tc.writeArrayValue(argTypes, 0, configurationConditionClass);
tc.writeArrayValue(argTypes, 1, tc.loadClassFromTCCL(String.class));
tc.assign(resourcesArgTypes, argTypes);
ResultHandle args = tc.newArray(Object.class, tc.load(2));
ResultHandle alwaysTrueMethod = tc.invokeStaticMethod(LOOKUP_METHOD,
configurationConditionClass,
tc.load("alwaysTrue"),
tc.newArray(Class.class, tc.load(0)));
ResultHandle alwaysTrueResult = tc.invokeVirtualMethod(INVOKE,
alwaysTrueMethod, tc.loadNull(),
tc.newArray(Object.class, tc.load(0)));
tc.writeArrayValue(args, 0, alwaysTrueResult);
tc.assign(resourcesArgs, args);
tc.assign(argsIndex, tc.load(1));

ResultHandle ignoreResourcesMethod = tc.invokeStaticMethod(LOOKUP_METHOD,
tc.loadClassFromTCCL("com.oracle.svm.core.configure.ResourcesRegistry"),
tc.load("ignoreResources"), resourcesArgTypes);
ResultHandle addResourcesMethod = tc.invokeStaticMethod(LOOKUP_METHOD,
tc.loadClassFromTCCL("com.oracle.svm.core.configure.ResourcesRegistry"),
tc.load("addResources"), resourcesArgTypes);
tc.assign(resourcesArgs,
tc.marshalAsArray(Object.class, tc.invokeStaticMethod(CONFIGURATION_ALWAYS_TRUE), tc.loadNull()));

AssignableResultHandle ignoreResourcesMethod = tc.createVariable(Method.class);
AssignableResultHandle addResourcesMethod = tc.createVariable(Method.class);
AssignableResultHandle resourcesSingleton = tc.createVariable(Object.class);

BranchResult graalVm22_3Test = tc.ifGreaterEqualZero(tc.invokeVirtualMethod(VERSION_COMPARE_TO,
tc.invokeStaticMethod(VERSION_CURRENT),
tc.marshalAsArray(int.class, tc.load(22), tc.load(3))));
/* GraalVM >= 22.3 */
try (BytecodeCreator greaterThan22_2 = graalVm22_3Test.trueBranch()) {

ResultHandle runtimeResourceSupportClass = greaterThan22_2.loadClassFromTCCL(RUNTIME_RESOURCE_SUPPORT);

greaterThan22_2.assign(resourcesSingleton, greaterThan22_2.invokeStaticMethod(IMAGE_SINGLETONS_LOOKUP,
runtimeResourceSupportClass));

greaterThan22_2.assign(ignoreResourcesMethod, greaterThan22_2.invokeStaticMethod(LOOKUP_METHOD,
runtimeResourceSupportClass, greaterThan22_2.load("ignoreResources"), resourcesArgTypes));
greaterThan22_2.assign(addResourcesMethod, greaterThan22_2.invokeStaticMethod(LOOKUP_METHOD,
runtimeResourceSupportClass, greaterThan22_2.load("addResources"), resourcesArgTypes));
}

/* GraalVM < 22.3 */
try (BytecodeCreator smallerThan22_3 = graalVm22_3Test.falseBranch()) {

ResultHandle resourceRegistryClass = smallerThan22_3
.loadClassFromTCCL("com.oracle.svm.core.configure.ResourcesRegistry");
smallerThan22_3.assign(resourcesSingleton, smallerThan22_3.invokeStaticMethod(IMAGE_SINGLETONS_LOOKUP,
resourceRegistryClass));

smallerThan22_3.assign(ignoreResourcesMethod, smallerThan22_3.invokeStaticMethod(LOOKUP_METHOD,
resourceRegistryClass, smallerThan22_3.load("ignoreResources"), resourcesArgTypes));
smallerThan22_3.assign(addResourcesMethod, smallerThan22_3.invokeStaticMethod(LOOKUP_METHOD,
resourceRegistryClass, smallerThan22_3.load("addResources"), resourcesArgTypes));
}

ResultHandle indexOne = tc.load(1);

for (NativeImageResourcePatternsBuildItem resourcePatternsItem : resourcePatterns) {
for (String pattern : resourcePatternsItem.getExcludePatterns()) {
tc.writeArrayValue(resourcesArgs, argsIndex, tc.load(pattern));
tc.invokeVirtualMethod(INVOKE, ignoreResourcesMethod, resourcesRegistrySingleton, resourcesArgs);
tc.writeArrayValue(resourcesArgs, indexOne, tc.load(pattern));
tc.invokeVirtualMethod(INVOKE, ignoreResourcesMethod, resourcesSingleton, resourcesArgs);
}
for (String pattern : resourcePatternsItem.getIncludePatterns()) {
tc.writeArrayValue(resourcesArgs, argsIndex, tc.load(pattern));
tc.invokeVirtualMethod(INVOKE, addResourcesMethod, resourcesRegistrySingleton, resourcesArgs);
tc.writeArrayValue(resourcesArgs, indexOne, tc.load(pattern));
tc.invokeVirtualMethod(INVOKE, addResourcesMethod, resourcesSingleton, resourcesArgs);
}
}
CatchBlockCreator cc = tc.addCatch(Throwable.class);
Expand All @@ -352,43 +360,60 @@ public void write(String s, byte[] bytes) {
// Needed to access LOCALIZATION_FEATURE
exports.produce(
new JPMSExportBuildItem("org.graalvm.nativeimage.builder", "com.oracle.svm.core.jdk.localization",
GraalVM.Version.VERSION_22_1_0));

AssignableResultHandle registerMethod = overallCatch.createVariable(Method.class);
AssignableResultHandle locClass = overallCatch.createVariable(Class.class);
TryBlock locTryBlock = overallCatch.tryBlock();
ResultHandle legacyLocClass = locTryBlock.loadClassFromTCCL(LEGACY_LOCALIZATION_FEATURE);
locTryBlock.assign(locClass, legacyLocClass);

ResultHandle legacyParams = locTryBlock.marshalAsArray(Class.class, locTryBlock.loadClassFromTCCL(String.class));
ResultHandle legacyRegisterMethod = locTryBlock.invokeVirtualMethod(
ofMethod(Class.class, "getDeclaredMethod", Method.class, String.class, Class[].class), legacyLocClass,
locTryBlock.load("addBundleToCache"), legacyParams);
locTryBlock.assign(registerMethod, legacyRegisterMethod);

CatchBlockCreator locCatchBlock = locTryBlock.addCatch(ClassNotFoundException.class);
ResultHandle newLocClass = locCatchBlock.loadClassFromTCCL(LOCALIZATION_FEATURE);
locCatchBlock.assign(locClass, newLocClass);

ResultHandle newParams = locCatchBlock.marshalAsArray(Class.class, locCatchBlock.loadClassFromTCCL(String.class));
ResultHandle newRegisterMethod = locCatchBlock.invokeVirtualMethod(
ofMethod(Class.class, "getDeclaredMethod", Method.class, String.class, Class[].class), newLocClass,
locCatchBlock.load("prepareBundle"), newParams);
locCatchBlock.assign(registerMethod, newRegisterMethod);

overallCatch.invokeVirtualMethod(ofMethod(AccessibleObject.class, "setAccessible", void.class, boolean.class),
registerMethod, overallCatch.load(true));

ResultHandle locSupport = overallCatch.invokeStaticMethod(
IMAGE_SINGLETONS_LOOKUP,
locClass);
for (NativeImageResourceBundleBuildItem i : resourceBundles) {
TryBlock et = overallCatch.tryBlock();

et.invokeVirtualMethod(ofMethod(Method.class, "invoke", Object.class, Object.class, Object[].class),
registerMethod, locSupport, et.marshalAsArray(Object.class, et.load(i.getBundleName())));
CatchBlockCreator c = et.addCatch(Throwable.class);
//c.invokeVirtualMethod(ofMethod(Throwable.class, "printStackTrace", void.class), c.getCaughtException());
GraalVM.Version.VERSION_22_1_0, GraalVM.Version.VERSION_22_3_0));

BranchResult graalVm22_3Test = overallCatch.ifGreaterEqualZero(overallCatch.invokeVirtualMethod(VERSION_COMPARE_TO,
overallCatch.invokeStaticMethod(VERSION_CURRENT),
overallCatch.marshalAsArray(int.class, overallCatch.load(22), overallCatch.load(3))));
/* GraalVM >= 22.3 */
try (BytecodeCreator greaterThan22_2 = graalVm22_3Test.trueBranch()) {

ResultHandle runtimeResourceSupportClass = greaterThan22_2.loadClassFromTCCL(RUNTIME_RESOURCE_SUPPORT);
ResultHandle addResourceBundlesParams = greaterThan22_2.marshalAsArray(Class.class,
greaterThan22_2.loadClassFromTCCL(ConfigurationCondition.class),
greaterThan22_2.loadClassFromTCCL(String.class));
ResultHandle addResourceBundlesMethod = greaterThan22_2.invokeStaticMethod(
LOOKUP_METHOD,
runtimeResourceSupportClass, greaterThan22_2.load("addResourceBundles"), addResourceBundlesParams);
ResultHandle runtimeResourceSupport = greaterThan22_2.invokeStaticMethod(
IMAGE_SINGLETONS_LOOKUP,
runtimeResourceSupportClass);
ResultHandle configAlwaysTrue = greaterThan22_2.invokeStaticMethod(CONFIGURATION_ALWAYS_TRUE);

for (NativeImageResourceBundleBuildItem i : resourceBundles) {
TryBlock et = greaterThan22_2.tryBlock();

et.invokeVirtualMethod(
INVOKE,
addResourceBundlesMethod, runtimeResourceSupport,
et.marshalAsArray(Object.class, configAlwaysTrue, et.load(i.getBundleName())));
CatchBlockCreator c = et.addCatch(Throwable.class);
//c.invokeVirtualMethod(ofMethod(Throwable.class, "printStackTrace", void.class), c.getCaughtException());
}
}

/* GraalVM < 22.3 */
try (BytecodeCreator smallerThan22_3 = graalVm22_3Test.falseBranch()) {

ResultHandle locClass = smallerThan22_3.loadClassFromTCCL(LOCALIZATION_FEATURE);
ResultHandle newParams = smallerThan22_3.marshalAsArray(Class.class,
smallerThan22_3.loadClassFromTCCL(String.class));
ResultHandle registerMethod = smallerThan22_3.invokeStaticMethod(
LOOKUP_METHOD,
locClass, smallerThan22_3.load("prepareBundle"), newParams);

ResultHandle locSupport = smallerThan22_3.invokeStaticMethod(
IMAGE_SINGLETONS_LOOKUP,
locClass);

for (NativeImageResourceBundleBuildItem i : resourceBundles) {
TryBlock et = smallerThan22_3.tryBlock();

et.invokeVirtualMethod(ofMethod(Method.class, "invoke", Object.class, Object.class, Object[].class),
registerMethod, locSupport, et.marshalAsArray(Object.class, et.load(i.getBundleName())));
CatchBlockCreator c = et.addCatch(Throwable.class);
//c.invokeVirtualMethod(ofMethod(Throwable.class, "printStackTrace", void.class), c.getCaughtException());
}
}
}
int count = 0;
Expand Down Expand Up @@ -554,48 +579,51 @@ public void write(String s, byte[] bytes) {
BranchResult graalVm22_3Test = tc.ifGreaterEqualZero(tc.invokeVirtualMethod(VERSION_COMPARE_TO,
tc.invokeStaticMethod(VERSION_CURRENT), tc.marshalAsArray(int.class, tc.load(22), tc.load(3))));
/* GraalVM >= 22.3 */
BytecodeCreator greaterThan22_2 = graalVm22_3Test.trueBranch();
greaterThan22_2.invokeStaticMethod(ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Class[].class),
carray);

if (jniAccessible.isConstructors()) {
greaterThan22_2.invokeStaticMethod(
ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
constructors);
}
try (BytecodeCreator greaterThan22_2 = graalVm22_3Test.trueBranch()) {
greaterThan22_2.invokeStaticMethod(ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Class[].class),
carray);

if (jniAccessible.isConstructors()) {
greaterThan22_2.invokeStaticMethod(
ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
constructors);
}

if (jniAccessible.isMethods()) {
greaterThan22_2.invokeStaticMethod(
ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
methods);
}
if (jniAccessible.isMethods()) {
greaterThan22_2.invokeStaticMethod(
ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
methods);
}

if (jniAccessible.isFields()) {
greaterThan22_2.invokeStaticMethod(
ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Field[].class),
fields);
if (jniAccessible.isFields()) {
greaterThan22_2.invokeStaticMethod(
ofMethod(JNI_RUNTIME_ACCESS, "register", void.class, Field[].class),
fields);
}
}
/* GraalVM < 22.3 */
BytecodeCreator smallerThan22_3 = graalVm22_3Test.falseBranch();
smallerThan22_3.invokeStaticMethod(ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, Class[].class),
carray);

if (jniAccessible.isConstructors()) {
try (BytecodeCreator smallerThan22_3 = graalVm22_3Test.falseBranch()) {
smallerThan22_3.invokeStaticMethod(
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
constructors);
}
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, Class[].class),
carray);

if (jniAccessible.isMethods()) {
smallerThan22_3.invokeStaticMethod(
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
methods);
}
if (jniAccessible.isConstructors()) {
smallerThan22_3.invokeStaticMethod(
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
constructors);
}

if (jniAccessible.isFields()) {
smallerThan22_3.invokeStaticMethod(
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, boolean.class, Field[].class),
smallerThan22_3.load(jniAccessible.isFinalFieldsWriteable()), fields);
if (jniAccessible.isMethods()) {
smallerThan22_3.invokeStaticMethod(
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, Executable[].class),
methods);
}

if (jniAccessible.isFields()) {
smallerThan22_3.invokeStaticMethod(
ofMethod(LEGACY_JNI_RUNTIME_ACCESS, "register", void.class, boolean.class, Field[].class),
smallerThan22_3.load(jniAccessible.isFinalFieldsWriteable()), fields);
}
}

CatchBlockCreator cc = tc.addCatch(Throwable.class);
Expand Down