Extraction issues with native libraries on macOS and Linux

[AndyAtTV]

On Windows everything works as expected: The appropriate library is extracted to 'java.io.tmpdir'/flatlaf.temp, and when I use flatlaf.nativeLibraryPath it is picked up from there (or extracted if I deliberately use the wrong path)

On macOS and Linux nothing works as expected and both O/S "fail" in the same way: If I use flatlaf.nativeLibraryPath="." (or any relative path) and whether or not I manually put the native lib there then the flatlaf error message says it can't find the library (using the wrong name!) and says it will extract it. For example on Linux it says "Did not find external library ./libflatlaf-linux-x86_64.so, using extracted library instead". On macOS a similar message appears, again with 'lib' prepended to the library name.

If I then put the library in the correct place and rename it with 'lib' added to the name (e.g. libflatlaf-linux-x86_64.so) then no error message appears and it appears to work okay. So it seems that the code is pre-pending a spurious 'lib' to the library name.

If I use flatlaf.nativeLibraryPath="system" then it doesn't seem to matter if I put the native lib on any of the 'system' paths, or not, it always fails. I always get a flatlaf error message saying e.g. "Did not find external library flatlaf-linux-x86_64.so, using extracted library instead". Note that in this case the 'lib' is not prepended to the library name in the message, but it never seems to pick up the native library no matter where I place it.

Finally, with both Linux and macOS and with all the configurations I've tried, when the error message says it is extracting it (and when I don't use flatlaf.nativeLibraryPath), I've not yet been able to find an extracted library in the entire filesystem, and especially not within java.io.tmpdir. So it doesn't seems to be extracting the library when it fails to find it.

For the Linux/macOS tests I'm using a x64 laptop running the latest Mint, and an Apple M macmini running the latest macOS v14. I'm using the latest Adoptium Java 17 on both systems, and FlatLaf 3.3, and I've repeated every test on each O/S in Eclipse, standalone apps using jars, and standalone apps after using jlink.

[DevCharly]

First of all: FlatLaf automatically extracts native libraries from flatlaf.jar to temporary folders.
There is no need to set system property flatlaf.nativeLibraryPath for this use case.

If you like to ship the FlatLaf native libraries with your application (to avoid loading from temporary folder),
it is highly recommended to grab them from Maven central.
See https://www.formdev.com/flatlaf/native-libraries/
In this case, it is also not necessary to set flatlaf.nativeLibraryPath.

On Windows everything works as expected: The appropriate library is extracted to 'java.io.tmpdir'/flatlaf.temp, and when I use flatlaf.nativeLibraryPath it is picked up from there (or extracted if I deliberately use the wrong path)

Extracting to 'java.io.tmpdir'/flatlaf.temp is the default behavior on Windows.
Setting system property flatlaf.nativeLibraryPath for this case is not necessary!

On macOS and Linux nothing works as expected and both O/S "fail" in the same way: If I use flatlaf.nativeLibraryPath="." (or any relative path) and whether or not I manually put the native lib there then the flatlaf error message says it can't find the library (using the wrong name!) and says it will extract it. For example on Linux it says "Did not find external library ./libflatlaf-linux-x86_64.so, using extracted library instead". On macOS a similar message appears, again with 'lib' prepended to the library name.

The expected file names for the native libraries are the same as used in flatlaf.jar:
https://github.com/JFormDesigner/FlatLaf/tree/main/flatlaf-core/src/main/resources/com/formdev/flatlaf/natives

They start with lib on macOS and Linux.

Relative paths are relative to the working directory of your application.
Are you sure that you started your application in the right directory?
In Eclipse launch config dialog you can specify the working directory on "Arguments" tab.

If I then put the library in the correct place and rename it with 'lib' added to the name (e.g. libflatlaf-linux-x86_64.so) then no error message appears and it appears to work okay. So it seems that the code is pre-pending a spurious 'lib' to the library name.

Yes, this is usual library naming on macOS and Linux.

If I use flatlaf.nativeLibraryPath="system" then it doesn't seem to matter if I put the native lib on any of the 'system' paths, or not, it always fails. I always get a flatlaf error message saying e.g. "Did not find external library flatlaf-linux-x86_64.so, using extracted library instead". Note that in this case the 'lib' is not prepended to the library name in the message, but it never seems to pick up the native library no matter where I place it.

To be correct it says: Did not find library flatlaf-linux-x86_64 in ... (without .so)
So the leading lib and the trailing .so or .dylib are indeed missing in that error message.

For flatlaf.nativeLibraryPath="system", FlatLaf uses method System.loadLibrary(libName), which expects libraryname without leading lib and trailing .so or .dylib.

If you think that flatlaf.nativeLibraryPath="system" does not work, then you've either put the FlatLaf natives to the wrong directory, or they have wrong names, or there is a bug in Java.
FlatLaf only invokes System.loadLibrary("flatlaf-linux-x86_64") in that case.

Finally, with both Linux and macOS and with all the configurations I've tried, when the error message says it is extracting it (and when I don't use flatlaf.nativeLibraryPath), I've not yet been able to find an extracted library in the entire filesystem, and especially not within java.io.tmpdir. So it doesn't seems to be extracting the library when it fails to find it.

Yes, this is because they are immediately deleted after loading.

The source code of FlatLaf is freely available. You can simply debug into following method to find out whether or where it is extracted:

FlatLaf/flatlaf-core/src/main/java/com/formdev/flatlaf/util/NativeLibrary.java

Lines 102 to 154 in eed11d2

	private static boolean loadLibraryFromJar( String libraryName, ClassLoader classLoader ) {
	// add prefix and suffix to library name
	libraryName = decorateLibraryName( libraryName );
	// find library
	URL libraryUrl = (classLoader != null)
	? classLoader.getResource( libraryName )
	: NativeLibrary.class.getResource( "/" + libraryName );
	if( libraryUrl == null ) {
	LoggingFacade.INSTANCE.logSevere( "Library '" + libraryName + "' not found", null );
	return false;
	}
	File tempFile = null;
	try {
	// for development environment
	if( "file".equals( libraryUrl.getProtocol() ) ) {
	String binPath = libraryUrl.getPath();
	String srcPath = binPath.replace( "flatlaf-core/bin/main/", "flatlaf-core/src/main/resources/" );
	File libraryFile = new File( srcPath ); // use from 'src' folder if available
	if( !libraryFile.isFile() )
	libraryFile = new File( binPath ); // use from 'bin' or 'output' folder if available
	if( libraryFile.isFile() ) {
	// load library without copying
	System.load( libraryFile.getCanonicalPath() );
	return true;
	}
	}
	// create temporary file
	Path tempPath = createTempFile( libraryName );
	tempFile = tempPath.toFile();
	// copy library to temporary file
	try( InputStream in = libraryUrl.openStream() ) {
	Files.copy( in, tempPath, StandardCopyOption.REPLACE_EXISTING );
	}
	// load library
	System.load( tempFile.getCanonicalPath() );
	// delete library
	deleteOrMarkForDeletion( tempFile );
	return true;
	} catch( Throwable ex ) {
	LoggingFacade.INSTANCE.logSevere( ex.getMessage(), ex );
	if( tempFile != null )
	deleteOrMarkForDeletion( tempFile );
	return false;
	}
	}

For me it works fine.

[AndyAtTV]

Probably my bad explanation, but you appear to have misunderstood some of what I said. No matter, I've got what I need from your response.

FYI My confusion about the linux/macos library filenames arose because the files in Maven (where I manually downloaded them from) don't have lib in front of them, and there is no reference to the lib prefix in the documentation. I never thought to just look in the jar file (sorry!). In future if I have an issue I'll check out the source code first rather than wasting your time.

Thanks for all the great work you are doing.

[DevCharly]

Thanks for your feedback. I've fixed/improved the log messages and the system property docs:

FlatLaf/flatlaf-core/src/main/java/com/formdev/flatlaf/FlatSystemProperties.java

Lines 174 to 205 in 722dde6

	/**
	* Specifies a directory in which the FlatLaf native libraries are searched for.
	* The path can be absolute or relative to current application working directory.
	* This can be used to avoid extraction of the native libraries to the temporary directory at runtime.
	* <p>
	* If the value is {@code "system"} (supported since FlatLaf 2.6),
	* then {@link System#loadLibrary(String)} is used to load the native library.
	* This searches for the native library in classloader of caller
	* (using {@link ClassLoader#findLibrary(String)}) and in paths specified
	* in system properties {@code sun.boot.library.path} and {@code java.library.path}.
	* <p>
	* If the native library can not be loaded from the given path (or via {@link System#loadLibrary(String)}),
	* then the embedded native library is extracted to the temporary directory and loaded from there.
	* <p>
	* The file names of the native libraries must be either:
	* <ul>
	* <li>the same as in flatlaf.jar in package 'com/formdev/flatlaf/natives' (required for "system") or
	* <li>when downloaded from Maven central then as described here:
	* <a href="https://www.formdev.com/flatlaf/native-libraries/">https://www.formdev.com/flatlaf/native-libraries/</a>
	* (requires FlatLaf 3.4)
	* </ul>
	* <p>
	* <strong>Note</strong>: Since FlatLaf 3.1 it is recommended to download the
	* FlatLaf native libraries from Maven central and distribute them with your
	* application in the same directory as flatlaf.jar.
	* Then it is <strong>not necessary</strong> to set this system property.
	* See <a href="https://www.formdev.com/flatlaf/native-libraries/">https://www.formdev.com/flatlaf/native-libraries/</a>
	* for details.
	*
	* @since 2
	*/
	String NATIVE_LIBRARY_PATH = "flatlaf.nativeLibraryPath";

FYI My confusion about the linux/macos library filenames arose because the files in Maven (where I manually downloaded them from) don't have lib in front of them, ...

Yes, the Maven native file names do not work with flatlaf.nativeLibraryPath in FlatLaf 3.3 without renaming.
You need to remove the version and add lib.
In FlatLaf 3.4, they will work.

... and there is no reference to the lib prefix in the documentation.

But there is also no reference to the system property flatlaf.nativeLibraryPath on that page 😉

Is there a reason that you want use flatlaf.nativeLibraryPath?

[AndyAtTV]

My reasoning for using flatlaf.nativeLibraryPath may seem weak, but here goes:

Background info: I use jlink and so I don't directly distribute flatlaf.jar. Instead all the files it contains are automatically placed within the modules file.

My gut feel (based on 40 years experience in IT) is that at some point something may interfere with the extraction. Maybe an AV or other security system doing so deliberately, or by mistake, or just some weird permissions setup or problem on a user's computer. So I'd prefer to have the appropriate library pre-extracted. It also means I can prevent the native libraries from getting into modules if they ever start getting big.

So I manually put the appropriate library alongside all the other native libraries in my app: In runtime/lib, or runtime/bin as appropriate, and where my build system can easily re-sign them if necessary.

Finally, my control-freak/pessimistic nature tells me to explicitly set flatlaf.nativeLibraryPath to the appropriate folder. Even though you may tell me that FlatLaf will find them anyway, I just feel happier doing this. Not a great reason I'll admit, maybe it's just the way my brain's wired.