/tmp mounted with noexec is not handled gracefully

[Equidamoid]

Version

3.3.3

Platform

Linux x64

JDK

11.0.14

Module

lwjgl core

Bug description

org.lwjgl.system.Library & co contain an assumption that if it can write an .so file to a certain path, then it can system.LoadLibrary() it, which is sometimes incorrect and causes unhandled exceptions.

A frequent example is having /tmp mounted with noexec flag. The user tries to run the application, lwjgl extracts its native library to /tmp/lwjgl_$USERNAME/..., and the following System.loadLibrary() throws because its underlying dlopen call is denied by the system.

It is possible to to use -Djava.io.tmpdir as a workaround, but it is near impossible for an average (non-programmer) user to figure out. Even adding this flag may be a challenge, as some applications (e.g. games) don't provide an easy way to add jvm flags. Moreover, sometimes the path is just ignored (I guess due to some application-dependent configuration).

Denying write access to /tmp/lwjgl_$USERNAME also helps (and causes the .so to be written to $HOME/.lwjgl/...), but again, not obvious and user-friendly.

Some examples of the problem occurring in various games:

• https://bugs.mojang.com/browse/MC-255235
• https://steamcommunity.com/app/108600/discussions/0/3198117312258029950/
• https://www.reddit.com/r/songsofsyx/comments/137xa1p/linux_noexec_issue_with_syx_and_tmp/

I'm not sure what is the best way of fixing this problem. Some options are:

• try another path when system.LoadLibrary() fails (in the same way as if extracting the library failed)
• add more info to the exception
• advise the application developers to place .so to a known "good" location (e.g. withing the game files) and not to rely on the auto-unpacking mechanism

Stacktrace or crash log output

java.lang.UnsatisfiedLinkError: /tmp/lwjgl_xxxxx/3.3.3-snapshot/x64/liblwjgl.so: /tmp/lwjgl_xxxxx/3.3.3-snapshot/x64/liblwjgl.so: failed to map segment from shared object
        at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
        at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442)
        at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498)
        at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2627)
        at java.base/java.lang.Runtime.load0(Runtime.java:768)
        at java.base/java.lang.System.load(System.java:1837)
        at org.lwjgl.system.Library.loadSystem(Library.java:189)
        at org.lwjgl.system.Library.loadSystemFromLibraryPath(Library.java:179)
        at org.lwjgl.system.Library.loadSystem(Library.java:132)
        at org.lwjgl.system.Library.loadSystem(Library.java:64)
        at org.lwjgl.system.Library.<clinit>(Library.java:52)

[Spasi]

Thank you @Equidamoid!