Support statically linking user JNI libraries

[gradinac]

Why JNI Libraries for static linking need additional support?

Dynamic linking in the static context is not possible out of the box. The whole purpose of static linking is not to have any library dependencies at runtime, removing the need for the dynamic linker completely. In practice dynamic libraries depend on the underlying libc (and potentially other libraries) that would lead to very hard-to-diagnose issues. Furthermore, it would increase the footprint of the application as multiple libraries are loaded.

In glibc using dlopen will result in a static executable that still requires parts of libc to be available as dynamic libraries at runtime.

On musl, dlopen is replaced with a stub that does nothing when called:

static void *stub_dlopen(const char *file, int mode)
{
	__dl_seterr("Dynamic loading not supported");
	return 0;
}

A detailed discussion on what can go wrong when mixing dynamic linking in static executables: https://www.openwall.com/lists/musl/2012/12/08/4

Usage of static linking with JNI libraries from the user perspective

Users will have to go through two steps:

1. They need to compile their JNI libraries as static. JNI libraries intended for use in a static native-image should be compiled with a musl toolchain.
2. They must specify at build-time which JNI libraries are statically linked. For static executables, we can assume all JNI libraries will be statically linked. For dynamic executables, we can introduce a new hosted/API option.

Implementation

We already support statically linking JDK JNI libraries. Most of the required code infrastructure should already be in place, however, there are some caveats:

• We automatically try to find used JNI libraries with com.oracle.svm.hosted.jdk.JNIRegistrationSupport#registerLoadLibraryPlugin. Currently, this only works for JDK libraries, but we could extend it to also include other JNI libraries in case of static linking.

• JNI libraries can have an initializer function (JNI_OnLoad). As Java only supports dynamically linking JNI libraries, they can all contain the same symbol without causing issues. For static linking, that is not possible. The way this is handled in the JDK static libraries is by appending the library name to the OnLoad symbol during the JDK build (i.e., for libjava.a, the on-load function name is JNI_OnLoad_java. We should support the same functionality for user libraries.

• In case the automatic library detection doesn't work, users should have an option that would allow them to pass in the JNI library name.

• For automatically discovered libraries, a reasonable assumption would be that they have to be in the current working directory of the native-image tool that runs the image build.

• Symbol resolution works differently for builtin and user JNI libraries - for example, on Linux, see com.oracle.svm.core.posix.PosixNativeLibrarySupport.PosixNativeLibrary#findSymbol: the difference lies in the parameter to dlsym that specifies the shared library to be searched - for builtin libraries, we will search for the symbol in the executable itself and it's dependencies (RTLD_DEFAULT), while for user libraries, we will only search for the symbol in the JNI library itself.

Note that the above is not a complete list of tasks necessary for this ticket, we may run into more issues once we start implementing the feature.

Documentation

We should document this feature along with at least links to guides on how to create static libraries on different platforms. We should also document differences from standard JNI (i.e., the custom JNI_OnLoad function)

Testing

We should add tests with a small custom static JNI library that would cover the above implementation.

 

[kirillp]

Can I also compile and link my JNI into a libjni.so and later link it statically to the .so produced by GraalVM ?
So it loads without any callto dlopen

[smasher164]

Issue #3359 also discusses this feature.

I mention in #3359 (comment) that a flag like -H:RegisterBuiltin=<comma-delimited list of libraries to resolve internally> could be a convenient shortcut for doing all the native library stuff.

Otherwise, I agree that simply documenting the current method would provide some confidence around its stability.

[GrapeBaBa]

Any update for using native image static with JNI lib?