Crash at assertion on message length in dispatch.c

[hakanai]

We discovered an issue where on upgrading our build machines to macOS 12, tests started to crash with an assertion failure in JNA's dispatch.c - presumably the error from dlerror() is now longer than JNA's permitted maximum of 1024.

On investigation, it seems the error only gets triggered if loading the library fails - the test in particular which triggered this is possible to skip for macOS, but I don't yet know whether we'll also see the same thing in production.

One fix would be to stop asserting this - another would be to figure out what the actual maximum error message length is and raise the limit to accommodate it. I already checked the docs for dlerror() and it doesn't seem to say.

On further investigation, the error message from dlerror() includes full diagnostics for which paths it tried to load - so the longer your library search path is, the more likely you will hit the limit and crash.

Steps to reproduce

• Be on macOS 12+
• Have a relatively long DYLD_LIBRARY_PATH (we had 5 entries in it - issue didn't happen if I reduced it to 1 entry)
• Try to load a library which does not exist

What happens

Assertion failed: (count <= len && "snprintf() output has been truncated"), function LOAD_ERROR, file dispatch.c, line 74.

Followed by crashing with exit code 134.

Versions in use

• JNA: 5.10.0
• JVM: 11.0.12
• OS: macOS 12
• Architecture: x86_64

[dbwiddis]

I was able to reproduce this (JNA 5.12.1, openjdk 18.0.2, macOS 12.5, x86_64) with the following alteration to what used to be the string "IOKit":

IOKit INSTANCE = Native.load(String.join("JNA", Collections.nCopies(40, "JNA")), IOKit.class);

That's a 123-character filename, plus .dylib for 128 characters.
Longer filename strings triggered JDK IllegalArgumentException for too long of a name but this was apparently long enough, coupled with the multiple path searches, to trigger the issue.

Turns out only 17x3 = 51 character filename plus extension plus 5 different search paths and all their characters and the "(no such file)" string gets to 993 characters total.

While the 128-character filename limit suggests one maximum, the path can be 1024 bytes on macOS which would allow for over a hundred directories, so it's not a useful maximum.

I'm not sure what the max limit should be, but 1024 is probably too low.

[matthiasblaesing]

Instead of increasing the buffer, just dropping the assert (or compiling with -DNDEBUG) would be a solution, snprintf will create a string, that is shorter than the full message, but maybe that is enough? Is there anything usefull in this wall of text?

[dbwiddis]

Is there anything usefull in this wall of text?

A list of all the paths it looked in, which actually is useful for debugging, not unlike what JNA does on the Java side. Here's what a misnamed library currently prints:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'FOO':
dlopen(libFOO.dylib, 0x0009): tried: '/Library/Java/JavaVirtualMachines/jdk1.8.0_221.jdk/Contents/Home/bin/./libFOO.dylib' (no such file), 'libFOO.dylib' (no such file), '/usr/local/lib/libFOO.dylib' (no such file), '/usr/lib/libFOO.dylib' (no such file), '/Users/danielwiddis/git/oshi/oshi-core/libFOO.dylib' (no such file)
dlopen(libFOO.dylib, 0x0009): tried: '/Library/Java/JavaVirtualMachines/jdk1.8.0_221.jdk/Contents/Home/bin/./libFOO.dylib' (no such file), 'libFOO.dylib' (no such file), '/usr/local/lib/libFOO.dylib' (no such file), '/usr/lib/libFOO.dylib' (no such file), '/Users/danielwiddis/git/oshi/oshi-core/libFOO.dylib' (no such file)
dlopen(/Users/danielwiddis/Library/Frameworks/FOO.framework/FOO, 0x0009): tried: '/Users/danielwiddis/Library/Frameworks/FOO.framework/FOO' (no such file), '/Library/Frameworks/FOO.framework/FOO' (no such file), '/System/Library/Frameworks/FOO.framework/FOO' (no such file)
dlopen(/Library/Frameworks/FOO.framework/FOO, 0x0009): tried: '/Library/Frameworks/FOO.framework/FOO' (no such file), '/System/Library/Frameworks/FOO.framework/FOO' (no such file)
dlopen(/System/Library/Frameworks/FOO.framework/FOO, 0x0009): tried: '/System/Library/Frameworks/FOO.framework/FOO' (no such file), '/Library/Frameworks/FOO.framework/FOO' (no such file)
Native library (darwin-x86-64/libFOO.dylib) not found in resource path ([file:/Users/danielwiddis/git/oshi/oshi-core/target/test-classes/, file:/Users/danielwiddis/git/oshi/oshi-core/target/classes/, file:/Users/danielwiddis/.m2/repository/net/java/dev/jna/jna/5.12.1/jna-5.12.1.jar, file:/Users/danielwiddis/.m2/repository/net/java/dev/jna/jna-platform/5.12.1/jna-platform-5.12.1.jar, file:/Users/danielwiddis/.m2/repository/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar, file:/Users/danielwiddis/.m2/repository/org/slf4j/slf4j-simple/1.7.36/slf4j-simple-1.7.36.jar, file:/Users/danielwiddis/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.9.0/junit-jupiter-api-5.9.0.jar, file:/Users/danielwiddis/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar, file:/Users/danielwiddis/.m2/repository/org/junit/platform/junit-platform-commons/1.9.0/junit-platform-commons-1.9.0.jar, file:/Users/danielwiddis/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar, file:/Users/danielwiddis/.m2/repository/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar])
	at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:307)

[dbwiddis]

shorter than the full message, but maybe that is enough?

I think a truncated message with no crash would be sufficient.

[matthiasblaesing]

I pushed a first possible fix version to #1456. Instead of relying on a static message size, the required buffer is allocated when generated and then freed after exception is thrown.

[matthiasblaesing]

@hakanai can you test the changes from #1456? This should fix your problem.

[hakanai]

I took a shot at this but it looks like the way our build works makes it difficult to test this on the real thing without an artifact to point at. But, I grafted the right jnidispatch binary into a separate smaller build and that seemed to work fine.

[mikehearn]

Thanks for the fix. This bug can also occur when using JNA in an app that's in app translocation due to the very long paths that can result.

[EricCraigen]

When can we expect to see this fix in a release? We have implemented a workaround in our project that informs MacOS users they are not running the application from the Applications folder and would like to remove it when this fix is officially released. Thanks in advance!

[matthiasblaesing]

I'm looking at the end of year to get a release out: https://groups.google.com/g/jna-users/c/Vag9ydD0voY