Cannot open native image resource using Path, NativeImageResourceFileSystemProvider not initialised

[credmond]

Describe the issue
One cannot open a resource using Java's Path. It seems NativeImageResourceFileSystemProvider is not initialised (fileSystem is null hence the exception is thrown).

Steps to reproduce the issue
Try to open any resource using this (where fortunes.json exists in /resource/fortunes.json, for example):

Path storeResourcePath = Path.of(Fortune.class.getResource("/fortunes.json").toURI());

... you will get the stacktrace provided below.

Build steps:

mvn clean compile
mvn -Pnative -Dagent exec:exec@java-agent
mvn -Pnative -Dagent package
target/fortune.exe OR ./target/fortune

E.g., using Graal's own examples @ https://github.com/graalvm/graalvm-demos/blob/master/fortune-demo/fortune/src/main/java/demo/Fortune.java, stick that line in main() and observe the same stacktrace pasted below.

It is possible to read the file this way:

InputStream stream = Fortune.class.getResource("/fortunes.json").openStream();

Obviously that's a workaround, but third party libraries using Path will still cause a problem.

I am surprised there isn't more noise about this issue.

Describe GraalVM and your environment:

• GraalVM versions:
  Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 17.0.9+11.1 (build 17.0.9+11-LTS-jvmci-23.0-b21, mixed mode, sharing)
  Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19, mixed mode, sharing)
• JDK major versions: 17, 21
• OS: Windows / Linux
• Architecture: x64

Stackrace:

Exception in thread "main" java.nio.file.FileSystemNotFoundException
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.resources.NativeImageResourceFileSystemProvider.getFileSystem(NativeImageResourceFileSystemProvider.java:144)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.resources.NativeImageResourceFileSystemProvider.getPath(NativeImageResourceFileSystemProvider.java:154)
        at [email protected]/java.nio.file.Path.of(Path.java:208)
        at demo.Fortune.main(Fortune.java:114)

[credmond]

Note: adding:

FileSystems.newFileSystem(URI.create("resource:/"), Collections.singletonMap("create", "true"));

...creates the filesystem. It only works for the native image run though, and fails when ran through an IDE or as a JAR as there's no provider for "resource".

What's going on here, why isn't this registered as a provider to auto init / and picked up during an agent run?

[credmond]

@jovanstevanovic: is this a bug?

[jovanstevanovic]

@credmond I'll have a look on Monday, and come back to you with an update.

[jovanstevanovic]

@credmond I couldn't reproduce it with the latest versions of both graalvm-demos and graal repositories. My version is:

native-image 21.0.1 2023-10-17
GraalVM Runtime Environment Oracle GraalVM 21.0.1-dev+11.1 (build 21.0.1+11-jvmci-23.1-b22)
Substrate VM Oracle GraalVM 21.0.1-dev+11.1 (build 21.0.1+11, serial gc)

[credmond]

Thanks for checking. But definitely is happening for me and I noticed it a couple of times (always just used workarounds). Let me see if I can get more details.

What operating system, by the way?

[jovanstevanovic]

I forgot to mention, on Linux.

[jovanstevanovic]

I see. There are two problems:

1. The code that you send me doesn't reflect the problem.
2. The code snippet that you send me can't work like that. You need first open the Native Image file system, and then you can use it. I've added a code to open and close the file system and everything works fine.
3. The same code works with HotSpot as well (build as jar and then run using that jar).

From my perspective, this is not a bug.

[credmond]

@jovanstevanovic: I mentioned above that manually opening the filesystem "works". However, I am very surprised to hear that that is by design and required.

The same code (Path.of(Fortune.class.getResource("/fortunes.json").toURI());) works in a JAR / IDE / locally, without needing to register any filesystem (e.g., all required filesystems are loaded by the JRE).

I thought the point of native image combined with the tracing agent (and overall sensible native image config) was to translate your bytecode to a native image without requiring adding specific code to your application. Is this an exception for some reason?

So I still thing this is a bug, or if not, it's an unclear design and is the only usage that requires adding hacky code, just for a native image to work.

Registering a "resource:/" filesystem fails outside of a native image run. There is no such "resource" filesystem. This means we need to add hacky try/catch filesystem register code just to get the same code to be able to run in native image or via a JAR or IDE.

I would have thought/expected that the filesystem could be marked for registering via native image config. The tracing agent surely should catch these odd requirements and add some configuration to register filesystems rather than requiring the developer to add them via application code, is that possible?

I feel like I am missing something here. I can't see how requiring specific native image code, is expected or desired design.

[credmond]

@jovanstevanovic

So you're telling me that if anyone wants to use basic standard Java code such as:

Path storeResourcePath = Path.of(Fortune.class.getResource("/fortunes.json").toURI());

...with native image, then they have to also manually add a hack like this first in order to get the NativeImageResourceFileSystem registered:

  // This hack registers NativeImageResourceFileSystemProvider & NativeImageResourceFileSystem when ran via Native Image
  // It allows lookups using "resource:/" URIs which means calls like Path.of(URI) will not fail.
  try {
       FileSystem filesystem = FileSystems.newFileSystem(URI.create("resource:/"), Collections.singletonMap("create", "true"));
       LOGGER.info("Created {} filesystem", filesystem.getClass().getSimpleName());
   } catch(Exception e) {
       // This will always happen outside of an native image; there no such thing as a "resource:/" file system outside of native image
       LOGGER.info("Not creating resource file system as not a native image.");
   }

... this is the reality. Is that the official expectation? I am not convinced this is a good idea. How native image looks up resources should be an internal problem and solved internally, Java application code should not need to consider this. I'm not sure if you're understanding my problem here.

[credmond]

This is what happens in any JRE (even GraalVM):

[jovanstevanovic]

I mentioned above that manually opening the filesystem "works". However, I am very surprised to hear that that is by design and required.

This is by design in Java as well. The Native Image FS operates similarly to how the Jar(Zip)FileSystem works in Java. If you add the same line, execute mvn clean package && java -jar target/fortune-1.0-SNAPSHOT-jar-with-dependencies.jar, you will encounter an exception like this:

Exception in thread "main" java.nio.file.FileSystemNotFoundException
  at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:156)
  at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:142)
  at java.base/java.nio.file.Path.of(Path.java:209)
  at demo.Fortune.readInputStream(Fortune.java:126)
  at demo.Fortune.<init>(Fortune.java:74)
  at demo.Fortune.main(Fortune.java:162)

I'll describe you a couple of more details in the next comment.

[jovanstevanovic]

Here we have a couple of things.

1. If you search for a resource that is in the jar with ClassA.class.getResource(resourceName), the resulting URL will have a jar scheme, and then If you want to operate further with that URL the corresponding FS is Jar(Zip)FileSystem.
2. If you run the same app and the resource is on a classpath that is not inside a jar, the scheme is file and the corresponding FS UnixFileSystem.
3. On Native Image (for the same way of resource access) the scheme is resource and for that, we have a special FS NativeImageResourceFileSystem that behaves the same as Jar(Zip)FileSystem.

[jovanstevanovic]

The reason NativeImageFileSystem behaves like JarFileSystem, not like UnixFileSystem, is that, unlike UnixFileSystem, you can't, for example, delete a file (resource) on both NativeImageFileSystem and JarFileSystem. The resources in NativeImageFileSystem are baked into the read-only section of the image heap

Additionally, when creating a file system in Java using a URL with a file scheme, the path should be set to '/'. However, if the scheme is jar, the path inside the JAR can be customized as desired. It's worth noting that the same code may not work consistently in Java, depending on whether the resource is located within a JAR file or not.

[credmond]

Thank you very much. I appreciate the details and yes, that makes sense.

Oddly, the project where I noticed this originally must have some third party lib registering ZipFileSystemProvider somewhere (I cannot find it :/), because the same lookup "works" when running as a JAR without any intentional registering (but not as a native image),

I see an old-ish enhancement request to auto-load filesystems:

https://bugs.openjdk.org/browse/JDK-8175243?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel

...where it complains about "Path" being limited in this sense. I did not know this behaviour, and I'm surprised I haven't encountered it before in various projects.

Anyway, although there's no bug as you said, hopefully this thread will help anyone with the same questions as myself.

My advice:

• register the filesystem as per my examples above
• or, do not use Path combined with getResource(); use getResource().openStream() and open using any stream reader class as normal

Either way will work.

Thanks again.

[jovanstevanovic]

@credmond You're welcome. I totally understand that these things with resources and file systems can be confusing sometimes. 🥲