Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide official API for commonly used svm.jar implementation details #4862

Closed
kristofdho opened this issue Sep 1, 2022 · 30 comments
Closed

Comments

@kristofdho
Copy link

kristofdho commented Sep 1, 2022

Copied from #4783 (comment):

We have some usages that I think are not yet covered by #4783 (workarounds are very welcome):

  1. com.oracle.svm.core.SubstrateOptions
    • Used to get the value of SubstrateOptions.Class, as we have multiple entrypoints within a module, and need to switch config based on which one is used.
  2. com.oracle.svm.core.jdk.NativeLibrarySupport, com.oracle.svm.hosted.c.NativeLibraries and com.oracle.svm.core.jdk.JNIRegistrationUtil
    • Used to write library support in a similar way to JNIRegistrationJava.
    • The JNIRegistrationUtil superclass can be substituted by our own implementation, but without the other classes we have no way to do the static linking ourselves.
  3. com.oracle.svm.core.jdk.Resources#registerResource(java.lang.String, java.lang.String, java.io.InputStream)
    • Used in our custom support for the fontmanager library, to add the fontconfig.bfc file to the resources, to be used from the substitution.
    • We can possibly work around this by loading the resource in a static byte array instead in our substitutions.
  4. com.oracle.svm.hosted.FeatureImpl.DuringSetupAccessImpl#registerClassReachabilityListener
    • Used to add reachability handlers based on conditions not currently available. For us specifically a listener that fires for all classes with a specific annotation.
  5. com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization
    • Hack to conditionally add resource bundles during analysis, used in the same way as here.
    • Any workaround suggestions to add simple resource-bundles on the fly would be much appreciated.

/cc @olpaw

@olpaw
Copy link
Member

olpaw commented Sep 7, 2022

In addition to the above here is a list of -H: options used by Quarkus: quarkusio/quarkus#27784
kindly provided by @zakkak

@olpaw
Copy link
Member

olpaw commented Sep 8, 2022

@kristofdho we discussed your suggestions:

  • com.oracle.svm.core.SubstrateOptions

    • Used to get the value of SubstrateOptions.Class, as we have multiple entrypoints within a module, and need to switch config based on which one is used.

As mentioned before use e.g. -Dmy.main.class=<MainClass> to get entrypoint information into the Feature

  • com.oracle.svm.core.jdk.NativeLibrarySupport, com.oracle.svm.hosted.c.NativeLibraries and com.oracle.svm.core.jdk.JNIRegistrationUtil

    • Used to write library support in a similar way to JNIRegistrationJava.
    • The JNIRegistrationUtil superclass can be substituted by our own implementation, but without the other classes we have no way to do the static linking ourselves.

Please copy whatever you need from there into your source base. We have no intention making theses helper methods public API.

  • com.oracle.svm.core.jdk.Resources#registerResource(java.lang.String, java.lang.String, java.io.InputStream)

    • Used in our custom support for the fontmanager library, to add the fontconfig.bfc file to the resources, to be used from the substitution.
    • We can possibly work around this by loading the resource in a static byte array instead in our substitutions.

This is an interesting usecase.

  • We need to add a new variant to the API (GR-40906):
org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResource(Module module, String resourcePath, byte[] resource)

that will allow you to do that. Thanks for bringing this up.

  • com.oracle.svm.hosted.FeatureImpl.DuringSetupAccessImpl#registerClassReachabilityListener

    • Used to add reachability handlers based on conditions not currently available. For us specifically a listener that fires for all classes with a specific annotation.

There are

org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerClassInitializerReachabilityHandler

and

org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerReachabilityHandler
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerMethodOverrideReachabilityHandler
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerSubtypeReachabilityHandler

in the API. The only thing left is to add API that allows you to query annotations without triggering JDK internal annotation handling. We have that internally since com.oracle.svm.util.GuardedAnnotationAccess was added in June.

  • Make GuardedAnnotationAccess accessible as public API (GR-40905)
  • com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization

    • Hack to conditionally add resource bundles during analysis, used in the same way as here.
    • Any workaround suggestions to add simple resource-bundles on the fly would be much appreciated.

This needs to be fixed as well:

  • Ensure org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResourceBundle API methods are usable during analysis (GR-40907)

@olpaw
Copy link
Member

olpaw commented Sep 8, 2022

@zakkak from the reported -H: options we have identified the following to be added as proper @APIOptions:

  • -H:IncludeLocales
  • -H:BuildOutputJSONFile
  • -H:+AddAllCharsets
  • -H:NativeLinkerOption

They will be added soon (GR-40904).

For the other option-use you mentioned there are the following remarks:

  • -H:-ParseOnce
    We recommend not using this option anymore.
  • -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime
    What is the reasoning behind that? Did you measure that this is actually beneficial?
  • -H:+AllowFoldMethods
    This should not be necessary. Please try to remove the option and report back if it causes problems.
  • -H:FallbackThreshold=5
    This is equivalent to API option --auto-fallback
  • -H:FallbackThreshold=0
    This is equivalent to API option --no-fallback
  • -H:EnableURLProtocols
    This is equivalent to API option --enable-url-protocols
  • -H:-InlineBeforeAnalysis
    This option should not be used anymore.
  • -H:-SpawnIsolates
    Except for native-debugging on Windows this option should not be used anymore.
  • -H:-StackTrace
    What is the reasoning behind that? Did you measure that this is actually beneficial?

@zakkak
Copy link
Collaborator

zakkak commented Sep 9, 2022

They will be added soon (GR-40904).

Thank you @olpaw

For the other option-use you mentioned there are the following remarks:

* `-H:-ParseOnce`
  We recommend not using this option anymore.

This is only used with GraalVM < 22.2 so there is no issue moving forward.

* `-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime`
  What is the reasoning behind that? Did you measure that this is actually beneficial?

It seems like this was based on some past evaluation of the then options and needs to be re-evaluated. There are people working on it.

* `-H:+AllowFoldMethods`
  This should not be necessary. Please try to remove the option and report back if it causes problems.

@galderz it looks like this was introduced by you in quarkusio/quarkus#13376. Can you please have a look?

* `-H:FallbackThreshold=5`
  This is equivalent to API option `--auto-fallback`

* `-H:FallbackThreshold=0`
  This is equivalent to API option `--no-fallback`

* `-H:EnableURLProtocols`
  This is equivalent to API option `--enable-url-protocols`

The above have been replaced in quarkusio/quarkus#27783

* `-H:-InlineBeforeAnalysis`
  This option should not be used anymore.

The option adding this flag has been deprecated in quarkusio/quarkus#27783

* `-H:-SpawnIsolates`
  Except for native-debugging on Windows this option should not be used anymore.

This was introduced back in Dec 2018, when isolates where spawned by default(?). We should mark this as deprecated and eventually remove it. Users (and testers) wil still disable full stack traces using quarkus.native.additional-build-args.

* `-H:-StackTrace`
  What is the reasoning behind that? Did you measure that this is actually beneficial?

@Sanne this was introduced as an option in quarkusio/quarkus@8d16a42 do you remember why it was needed for?
In any case I believe we should deprecate the corresponding option and eventually delete it. Users will still be able to disable full stack traces using quarkus.native.additional-build-args.

@Sanne
Copy link
Contributor

Sanne commented Sep 9, 2022

[Regarding -H:-StackTrace I've commented on https://github.com/quarkusio/quarkus/issues/27784#issuecomment-1241806173 ]

Unrelated to the -H options, I'm more concerned about the move of the annotations.

I'm understanding that many annotations are being moved from the svm artifact to the new API artifact - this is nice.

However it would seem that some of them have a different package; as we're currently building Quarkus with the old annotations (we have to until GraalVM releases this), this is compiling fine (no errors, no warnings), but then the new GraalVM version isn't going to apply these in practice. Examples: @NeverInline, @AlwaysInline, @Uninterruptible, @UnknownObjectField, ...

This concerns me as we can only observe the problem via obscure side-effects; could we maintain the existing annotations in the current package in addition to the new one, deprecate the old ones, and have the compiler take into account both flavours?

Alternatively, may I suggest to only keep them in the original package, mark them as deprecated (so to express they are still not meant as stable API) and address this problem in the future. Essentially postpone the move to a new package and treat that as a separate task to be addressed in a different version, that would keep the current migration simpler and give some more time to consider these implications.

@kristofdho
Copy link
Author

kristofdho commented Sep 11, 2022

@olpaw

  • com.oracle.svm.core.jdk.NativeLibrarySupport, com.oracle.svm.hosted.c.NativeLibraries and com.oracle.svm.core.jdk.JNIRegistrationUtil
    • Used to write library support in a similar way to JNIRegistrationJava.
    • The JNIRegistrationUtil superclass can be substituted by our own implementation, but without the other classes we have no way to do the static linking ourselves.

Please copy whatever you need from there into your source base. We have no intention making theses helper methods public API.

I agree that we should just copy whatever util function that we use, but that only covers the usages of JNIRegistrationUtil.

We're still doing calls like:

PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("some_prefix");
NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("library");
NativeLibraries nativeLibraries = ((FeatureImpl.BeforeAnalysisAccessImpl) a).getNativeLibraries();
nativeLibraries.addStaticJniLibrary("library");
nativeLibraries.addStaticNonJniLibrary("otherLibrary");
nativeLibraries.addDynamicNonJniLibrary("anotherLibrary");

  • com.oracle.svm.hosted.FeatureImpl.DuringSetupAccessImpl#registerClassReachabilityListener

There are

org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerClassInitializerReachabilityHandler

and

org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerReachabilityHandler
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerMethodOverrideReachabilityHandler
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerSubtypeReachabilityHandler

I'm probably missing something here, but are you suggesting the functionality I need is already available here?
These methods always have some sort of matching criteria, what I'm requesting is a method that returns classes that have a certain annotation on them, eg:

void registerAnnotatedClassReachabilityHandler(BiConsumer<DuringAnalysisAccess, Class<?>> callback,
                                               Class<? extends Annotation> annotation);

So it can be used like:

registerAnnotatedClassReachabilityHandler(MyFeature::annotatedClassReachable, javax.inject.Singleton.class);

Possibly extend it to allow searching for any annotated element:

void registerAnnotatedElementReachabilityHandler(BiConsumer<DuringAnalysisAccess, java.lang.reflect.AnnotatedElement> callback,
                                                 Class<? extends Annotation> annotation,
                                                 java.lang.annotation.ElementType elementType); // elementType could be optional

I don't follow exactly how GuardedAnnotationAccess fits into that. Just to return the instance of the annotation, instead of calling Class#getAnnotation? I guess that could still be kept internal, if the callback includes the annotation instance.

@andrejusc
Copy link

Side question - could we get/generate concise API doc/html with versions and since/deprecated notion in them assuming that list of those -H options continues to grow/mutate?

@olpaw
Copy link
Member

olpaw commented Sep 12, 2022

@kristofdho regarding:

Ensure org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResourceBundle API methods are usable during analysis (GR-40907)

Are you sure this does not work with the new RuntimeResourceAccess#addResourceBundle API method?

I tested the following:

Implemented DemoFeature that provides org.graalvm.nativeimage.hosted.Feature#duringAnalysis that uses

if (addBundle) {
   Module module = ModuleLayer.boot().findModule("jdk.compiler").orElseThrow();
   String bundleName = "com.sun.tools.javac.resources.javac";
   System.out.printf("Adding ResourceBundle %s:%s%n", module.getName(), bundleName);
   RuntimeResourceAccess.addResourceBundle(module, bundleName);
   System.out.printf("ResourceBundle added%n");
   addBundle = false;
}

Then in the Demo.main method I use:

Module compilerModule = ModuleLayer.boot().findModule("jdk.compiler").orElseThrow();
String bundleName = "com.sun.tools.javac.resources.javac";
ResourceBundle bundle = ResourceBundle.getBundle(bundleName, compilerModule);
System.out.println(bundle);
bundle.getKeys().asIterator().forEachRemaining(System.out::println);

This works as expected and gives me

com.sun.tools.javac.resources.javac@2d2802e4
javac.opt.Xlint.desc.synchronization
javac.opt.arg.key.equals.value
javac.opt.Xdoclint.subopts
javac.opt.g
javac.opt.Xlint.desc.unchecked
javac.msg.usage.header
javac.opt.d
javac.opt.X
javac.opt.sourcepath
....
javac.opt.Xlint.all
javac.opt.Xlint.desc.module
javac.opt.release
javac.opt.modulepath

at image runtime.

@olpaw
Copy link
Member

olpaw commented Sep 12, 2022

Implemented DemoFeature that provides org.graalvm.nativeimage.hosted.Feature#duringAnalysis that uses

Also works if I use ModuleLayer.boot().findModule("java.xml") and

String bundleName = "com.sun.org.apache.xml.internal.serializer.utils.SerializerMessages";

or

String bundleName = "com.sun.org.apache.xalan.internal.xsltc.compiler.util.ErrorMessages";

@kristofdho, can you provide me with a sample program where you cannot use RuntimeResourceAccess.addResourceBundle in org.graalvm.nativeimage.hosted.Feature#duringAnalysis?

@kristofdho
Copy link
Author

@olpaw I was merely reporting our usages of internal API and why they are used. I tried removing the setConfigurationSealed call (with GraalVM 22.2) and it indeed seems to compile fine. This was added in our codebase back when we were using GraalVM 21.0 and hasn't been updated since. It definitely wasn't allowed to add resource bundles during analysis back then, as evident from this internal feature as well. It must have been updated along the way.

So we can probably assume this is no longer an issue. (The internal usage I refer to can probably also be cleaned up a bit, and no longer call setConfigurationSealed).

@olpaw
Copy link
Member

olpaw commented Sep 12, 2022

So we can probably assume this is no longer an issue.

Thanks for confirming!

@galderz
Copy link
Contributor

galderz commented Sep 13, 2022

  • -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime
    What is the reasoning behind that? Did you measure that this is actually beneficial?

This goes back to quarkusio/quarkus@223288a in 2018. At the time the default GC policy (adaptive) was deemed to generate full GCs constantly compared to say the space/time one. I'm exploring if this is still the case today, and see what tuning can be done, if any, to improve things while staying with the default adaptive GC policy.

@Sanne
Copy link
Contributor

Sanne commented Sep 16, 2022

@christianwimmer please could you check #4862 (comment) as well - I think it was missed.

@olpaw
Copy link
Member

olpaw commented Sep 19, 2022

I don't follow exactly how GuardedAnnotationAccess fits into that. Just to return the instance of the annotation, instead of calling Class#getAnnotation? I guess that could still be kept internal, if the callback includes the annotation instance.

@kristofdho if you use

org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerSubtypeReachabilityHandler

with java.lang.Object as the subtype and you combine it with the recently merged
https://github.com/oracle/graal/pull/4989/files#diff-c30464fae56f58c5775804672939d2a4e75ab85a73420ba286099f2ceac5efea
you can do the same as what you would expect from registerAnnotatedElementReachabilityHandler

@olpaw
Copy link
Member

olpaw commented Sep 19, 2022

This is an interesting usecase.

We need to add a new variant to the API (GR-40906):

org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResource(Module module, String resourcePath, byte[] resource)

that will allow you to do that. Thanks for bringing this up.

@kristofdho this implemented and merged (will be in 22.3): #4945

@Sanne
Copy link
Contributor

Sanne commented Sep 21, 2022

@christianwimmer please could you check #4862 (comment) as well - I think it was missed.

For the record, I've discussed this particular case at the face to face in Zurich with @christianwimmer and I think our conclusion was that it's not a big problem; if something could be done to catch people relying on the old annotations (now silently ignored) that would be a "nice to have", but not a blocker.

@kristofdho
Copy link
Author

kristofdho commented Sep 24, 2022

@olpaw We have identified an additional internal API usage, which sort-of fits in the bucket of using the options directly:

File imagePath = NativeImageGenerator.generatedFiles(HostedOptionValues.singleton()).toFile();
// This ultimately boils down to the value of SubstrateOptions.Path

We use this to find where the image will be writen.
I am aware that org.graalvm.nativeimage.hosted.Feature.AfterImageWriteAccess#getImagePath exists. However, as we're building through Gluon Substrate, afterImageWrite never gets called because they run with the -H:+ExitAfterRelocatableImageWrite option:
https://github.com/gluonhq/substrate/blob/6b447be1317a51f3aecb68ba8a6ecada24aeac1a/src/main/java/com/gluonhq/substrate/target/AbstractTargetConfiguration.java#L144

Would it be possible to expose the parent directly where these files will get writen?

PS: from a library dev standpoint, it may be usefull to also expose com.oracle.svm.core.BuildArtefacts#add so these type of additions are also included in the artefacts file, but that's obviously more a nice-to-have kind of thing.

@kristofdho
Copy link
Author

I also don't think the PlatformNativeLibrarySupport and NativeLibraries class access has been adresses yet.

@olpaw
Copy link
Member

olpaw commented Sep 28, 2022

I also don't think the PlatformNativeLibrarySupport and NativeLibraries class access has been adresses yet.

@kristofdho we need to fully understand your need to use those low-level classes.

There is

  -H:NativeLinkerOption=<string>*              Pass the provided raw option that will be appended to the linker command to produce the final binary. The possible
                                               options are platform specific and passed through without any validation.

If you need more than that please explain your specific usecase in detail on a separate issue.

@kristofdho
Copy link
Author

kristofdho commented Sep 28, 2022

@olpaw I'm honestly not sure if I need more than that or not.

We're using it to statically link in jni libraries, one example is the fontmanager library:

PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_font");
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_java2d_loops_DrawGlyphList");
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_awt_Win32FontManager_getFontPath");
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_awt_Win32FontManager_populateFontFileNameMap0");

NativeLibraries nativeLibraries = ((FeatureImpl.BeforeAnalysisAccessImpl) a).getNativeLibraries();
NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("fontmanager");
NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("freetype");
nativeLibraries.addStaticJniLibrary("fontmanager", "awt");
nativeLibraries.addStaticNonJniLibrary("freetype");
nativeLibraries.addDynamicNonJniLibrary("gdi32");

This works in the same way as other JNI libraries are configured in GraalVM already (sunmscapi, sunec, prefs, mlib_image, lcms, ...)
I don't think that's possible entirely through -H:NativeLinkerOption?

@kristofdho
Copy link
Author

kristofdho commented Nov 4, 2022

@olpaw I have discovered a few more API usage cases that were hidden by configuration to allow the previously fixed internal usages:

  1. ImageSingletons.lookup(ResourcesRegistry.class).addResources(pattern);
    • Current API in RuntimeResourceAccess only allows registering resources one by one, suggestion to add addResourcePattern(Module module, String pattern) to allow the old behavior.
      Context: We have a resource jar with static content to serve in a webapp, but this is only conditionally included. The resources itself are randomized as part of the build, but we just need to include the entire folder anyway, so adding them without wildcard is currently not possible.
  2. ImageSingletons.lookup(SystemPropertiesSupport.class).initializeProperty("property.name", "value")
    • Used to bake in default values of system properties. Actually occurs in Micronaut's NettyFeature. However we have some use for it as well, as now we have a setup class configuring the required system properties that we need to call in every possible entrypoint ourselves. This API would help us centralize this setup.
      Suggestion to add API for initializeProperty(String property, String value) and initializePropertyLazy(String property, Supplier<String> value)
  3. Nice to have: BuildArtifacts.singleton().add(ArtifactType.JDK_LIB_SHIM, dll)
    • Libraries that depend on their own packaged dll, can export these as a resource to include in the build to avoid packaging them as resources and having to unpack them to temp folders to use. Having the option to register it as a build artifact makes it easier for external tools to correctly package everything up, as the build_artifacts.txt file contains everything correctly.
      Edit: I realise this is a repeat of this comment Provide official API for commonly used svm.jar implementation details #4862 (comment)

@olpaw
Copy link
Member

olpaw commented Nov 30, 2022

Current API in RuntimeResourceAccess only allows registering resources one by one, suggestion to add addResourcePattern(Module module, String pattern) to allow the old behavior.

At some point we want to get rid of pattern-based resource specification even in the json-file approach. The problem is that allowing patterns forces the builder to scan through the entirety of classpath and modulepath to determine for each individual entry in there if it matched the given pattern. It is a scalability issue.

But feel free to implemented that scanning through your resources on your side (e.g in your feature-code) and then call addResources for each element that you found.

@olpaw
Copy link
Member

olpaw commented Nov 30, 2022

ImageSingletons.lookup(SystemPropertiesSupport.class).initializeProperty("property.name", "value")

Would a META-INF/native-image/.../native-image.properties with

Args = -Dproperty.name=value ...

also do the trick for your usecase?

@olpaw
Copy link
Member

olpaw commented Nov 30, 2022

Nice to have: BuildArtifacts.singleton().add(ArtifactType.JDK_LIB_SHIM, dll)

Are you relying on the output in file build_artifacts.txt? @fniephaus is currently working on this and plans to get rid of it (by replacing it with equivalent information in some json file).

Allowing users to refer to arbitrary files in feature code and make them part of what constitutes the build-output is problematic and will conflict with replay-bundles. See #5473

@olpaw
Copy link
Member

olpaw commented Nov 30, 2022

Re: #4862 (comment)

This works in the same way as other JNI libraries are configured in GraalVM already (sunmscapi, sunec, prefs, mlib_image, lcms, ...)
I don't think that's possible entirely through -H:NativeLinkerOption?

It's not. But I don't think we want to make this official API as this goes way beyond what we want our users to do with public API (remember, we have to support public API). In your case, I guess, it makes more sense to either, 1) keep using our internals or 2) contribute your features to our codebase so that using internals is fine. cc @christianwimmer @wirthi

@kristofdho
Copy link
Author

kristofdho commented Nov 30, 2022

But feel free to implemented that scanning through your resources on your side (e.g in your feature-code) and then call addResources for each element that you found.

That was the obvious alternative, but we didn't do that for now as there was an internal API for that. We'll implement this on our side instead.

Would a META-INF/native-image/.../native-image.properties with

Args = -Dproperty.name=value ...

also do the trick for your usecase?

Unless I'm missing something, that would be a build-time only argument. What I'm looking for (and how it is used in the Micronaut NettyFeature) is a way to configure the default value of the property for the runtime of the native executable.
This also doesn't allow setting any lazilly computed values.

Are you relying on the output in file build_artifacts.txt?

Currently no. At this time we are blindly copying over all *.dll and *.exe files from the target folder (Maven). But since we have a custom feature that adds a few dlls to this output, it would've made sense to declare them as artifacts to native-image as well. But this was more as a general feature towards library developers which require native libraries, so they would have a proper official way of including these.
I don't think this would be problematic for replay-bundles, I would even argue that it would be necessary.
The replay bundle will contain all sources used for the original build anyway, so this includes the custom features that would add additional artifacts. So the replay would once again include these artifacts, as expected. The described image-building-as-a-service use-case would need to create an archive of all created artifacts for the user to download. So if native-image is not made aware of these additional artifacts, those would not be included.
What you're maybe referring to as problematic is if a user would try to include an arbitrary file from the host, which is not originating from one of the source bundles. On that I agree, that would be abuse of that specific API, but there is always room to misuse anything..

  1. contribute your features to our codebase so that using internals is fine.

I'm certainly willing to do so, I however currently only have a working setup for Windows, and I remember from previous interactions (years ago at this point) that cross-platform support is preferred over single-platform support. And specifically for the fontmanager and freetype libraries I have no knowledge how to test them on linux/macos.
And then there's the problem that this requires some substitutions that are in our current setup very minimalistic and remove conficuration flexibility as we don't use it, but would probably still need to be configurable in an official supporting solution. However that's nothing that a good round of reviews shouldn't be able to fix.

There are other issues that discuss or relate to such a potential API:
#4796
#4028
#3359

Edit: I do believe a first version of any native library support will likely start as a custom feature before it can reach a point where it can be made into a PR to include support for everyone. So without official API it makes the road towards general support ever so slightly more cumbersome.

@olpaw
Copy link
Member

olpaw commented Nov 30, 2022

Unless I'm missing something, that would be a build-time only argument. What I'm looking for (and how it is used in the Micronaut NettyFeature) is a way to configure the default value of the property for the runtime of the native executable.

I see. It makes sense to add an API for that.

@olpaw
Copy link
Member

olpaw commented Dec 2, 2022

Unless I'm missing something, that would be a build-time only argument. What I'm looking for (and how it is used in the Micronaut NettyFeature) is a way to configure the default value of the property for the runtime of the native executable.

I see. It makes sense to add an API for that.

Created #5557 for that.

@kristofdho
Copy link
Author

kristofdho commented Dec 2, 2022

Additional issue with contributing our fontmanager feature for Windows. It works on standard native-image, but there are linker issues on Gluon substrate: gluonhq/substrate#945
So for now we need to be able to switch between static and dynamic linking depending on if GluonFX is used or not.

In general anything awt related is a mess with javafx because we need the shim dlls, but substrate doesn't create them. So we need a second dummy project just to create the shims through the official build tools and provide a boatload of additional linker flags to substrate to make it work.

@wirthi
Copy link
Member

wirthi commented Aug 12, 2024

I don't see any obvious open issue here. #5557 that Paul created has been closed already.

Closing this ticket; if there is something concrete left to be done, please open a new ticket specifically for those parts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

7 participants