Add option to configure environment variables

[Zomono]

It would be very nice, if this great plugin has an option to set environment variables.

In my case I have to start my java app with LD_LIBRARY_PATH=lib java -jar myapp.jar. because my program is using native libraries and the only way to configure the directory which contains those libs is by setting a system specific environment variable.

When the app is bundled as native package it is no longer possible to offer environment variables during startup. Environment variables set with the start of the binary file don't have any effect to the java app.

Thanks in advanced

[FibreFoX]

For this I've added parameter-support on javafx-gradle-plugin 8.6.0, just define your parameters inside the new runJavaParameter-property:

jfx {
    // ....
    runJavaParameter = "-Djava.library.path=lib"
}

Does this work for you?

[Zomono]

Nope this doesn't work because the java-parameter "-Djava.library.path=lib" has only effect within the JVM. This is ok when you have a simple library with no other dependencies. But my library A has another dependency B. Those secondary dependencies will be handled by the native load library function (dlopen on mac and linux). This native function does not know about settings within the JVM. Thus it fails to load the secondary library even if the java.library.path option is set.
The dlopen function only cares about the evironment variable DL_LIBRARY_PATH

[FibreFoX]

So you are not talking about the jfxRun-task, but the generated native Launcher, correct?
As this is a bit special, can you provide some stripped down example project?

[FibreFoX]

If you mean something like manipulating the system environment, you might end up using something like this: http://stackoverflow.com/a/38073822/1961102

For me your setup isn't that clear, would be very good to have some reproducable project, where all components are open-sourced. I won't use precompiled stuff, especially not when having some binaries for own protection purpose.

Manipulating environment does not work cross-OS, so this might take a while to recreate or find a final solution. Maybe it would be better to change your binary libs to get their stuff right. This is a special question about general development of loading native libraries in addition to the native launcher. Am I wrong here?

[FibreFoX]

When the app is bundled as native package it is no longer possible to offer environment variables during startup

you can adjust the native launcher containing that -Djava.library.path=lib too, just set your configuration like this:

using jvmArgs

jfx {
    // ....
    runJavaParameter = "-Djava.library.path=lib"
    jvmArgs = [
        '-Djava.library.path=lib'
    ]
}

using jvmProperties

jfx {
    // ....
    runJavaParameter = "-Djava.library.path=lib"
    jvmProperties = [
        'java.library.path': 'lib'
    ]
}

These will be added to the internal call of the native launcher (you will find these inside the generated .cfg-file.

[Zomono]

Ok I have made a test project.
(https://www.dropbox.com/s/9kq4jzzw5wj9obc/Testwiese.zip?dl=0)
I have include the library z3 as a test library, where z3java references z3 and z3 references nothing.
Z3 is under MIT license and could be downloaded here (https://github.com/Z3Prover/z3/releases).

If you run the jar or the binary without any path-specification the error should happen in the first line "Failed to load z3".
If you use the java.library.path=lib argument you will notice, that z3 will be loaded but z3java fails (no error on windows because windows loads libraries a bit different).
If you use the environment variable

• DYLD_LIBRARY_PATH=lib on mac
• LD_LIBRARY_PATH=lib on linux
• PATH=lib on windows

like LD_LIBRARY_PATH=lib java -jar example.jar the example finished without errors.

The goal is to make the binary 'example' including such environment variables automatically when starting the JVM.

Beside my use case, there are many other ones where you want to use environment variables on your java call.

Thanks for your fast help.

[Zomono]

http://stackoverflow.com/a/38073822/1961102
I found this hack before but it is not working. It is absolutely impossible to change environment variables like LD_LIBRARY_PATH after the programm has started (ldopen uses the old value). This is because those variables are cached. Even in pure c/c++ it is not possible (i tried it by calling the native setenv functions).

I cant (and don't want to) modify my native libs because I'm not the author and I may want to update to newer version of those libraries.

[FibreFoX]

Sorry to be a bit pushy ... but I dont see any project I can download :) I need your example project involving the javafx-gradle-plugin and your native lib.

[Zomono]

Hmm where is that link gone https://www.dropbox.com/s/9kq4jzzw5wj9obc/Example.zip?dl=0

[FibreFoX]

@Zerlono thanks a lot, maybe I can find something useful 😄

[FibreFoX]

Hi @Zerlono

I have played a bit with the project you provided. On windows it works for me when using:

jfx {
    // minimal requirement for jfxJar-task
    mainClass = 'example.Main'
    nativeReleaseVersion = "$version"
    vendor = 'Meee'

    bundler = 'windows.app' // set this to some specific, if your don't want all bundlers running, examples 'windows.app', 'jnlp', ...

    appName = 'example' // this is used for files below 'src/main/deploy', e.g. 'src/main/deploy/windows/project.ico'
    additionalAppResources = 'addRes' // path to some additional resources when creating application-bundle

    bundleArguments = [
        runtime: null
    ]
    jvmArgs = [
        '-Djava.library.path=lib'
    ]
}

Could you try to print out the stacktrace when loading the library? I needed to modify the class-file a bit by adding "lib" as part of the libraryname (because thats what the filename is) and required to use different DLL-files.

package example;

import java.io.IOException;
import java.nio.file.StandardOpenOption;
import com.sun.javafx.application.LauncherImpl;
import javafx.application.Application;
import javafx.stage.Stage;

public class Main extends Application {

    public static void main(String[] args) {
        LauncherImpl.launchApplication(Main.class, args);
    }

    public static void appendToLogFile(String something) throws IOException {
        if( something != null ){
            java.nio.file.Files.write(java.nio.file.Paths.get("test.txt"), something.getBytes(), StandardOpenOption.APPEND, StandardOpenOption.CREATE);
        }
    }

    @Override
    public void start(Stage primaryStage) throws Exception {
        appendToLogFile("\n" + "System.getenv: java.library.path > " + System.getenv("java.library.path"));
        appendToLogFile("\n" + "System.getProperty: java.library.path > " + System.getProperty("java.library.path"));
        try{
            System.loadLibrary("libz3");
        } catch(Throwable e){
            e.printStackTrace();
            appendToLogFile("Failed to load z3");
            System.out.println("Failed to load z3");
            System.exit(0);
            return;
        }

        try{
            System.loadLibrary("libz3java");
        } catch(Throwable e){
            e.printStackTrace();
            appendToLogFile("Failed to load z3java");
            System.out.println("Failed to load z3java");
            System.exit(0);
            return;
        }
        appendToLogFile("SUCCESS");
        System.out.println("SUCCESS");
        System.exit(0);
    }
}

Please note that the jfxRun-task does NOT work, because the additional files won't be copied on that task, only when creating a native bundle (for me the fastest way was using windows.app) the files inside the additionalAppResources-folder will be copied.

I will check this on my ubuntu vm, because as you said, different systems behave different ;)

[Zomono]

Windows uses a different native library to do the dependency loading stuff. It differs to the dlopen function from linux/mac in the way how already loaded libs are handled. Windows recognizes if z3 was already loaded before and when it comes to z3java, z3 will not be loaded again. Thus the secondary dependency problem does not exists any more as long as you load your libs in the right order.
Linux in contrast loads z3 again (while loading z3java) even if you have loaded z3 (standalone) before.
And as always when it comes to the automatic dependency loading from within dlopen, it crashs because 'java-library.path' is unknown to this native function.

The naming convention on windows and linux/mac are not the same. This is the reason while you have to rename it ('lib'). On linux every libraryname is prefixed by 'lib' and appended by '.so' to find the right file. On windows there is no prefixing. The name is just appended by '.dll'

I'm not using the jfxrun task. It is not a problem for me.

[FibreFoX]

So you are saying, if I'm calling System.loadLibrary("libz3java"); this should load the libz3 itself? (haven't looked into it yet)

[Zomono]

Yeah the System.loadLibrary uses the native function System.mapLibraryName to get the right file name.
On windows use libz3 to load libz3.dll or rename the file to z3.dll

[FibreFoX]

:) that I'm aware of, I was more asking: do I require to load libz3 before libz3java? Or is libz3java loading the libz3 itself?

[Zomono]

z3java should load z3 automatically.

[FibreFoX]

Thanks for the input, I now do see the problem ... just asking: why aren't you loading that libz3 before loading libz3java your self? Seems like a very easy workaround ... do you see that the same way?
Manipulating "environment variables" seems pretty hard, on linux-systems it might be possible to exchange the native launcher by some "wrapper"-script/-file ... but on windows it might be a better solution to "just load it yourself" :(

[FibreFoX]

Some note: very interesting workaround about that "lib"-prefix:
Z3Prover/z3@d8d0b21

[Zomono]

The preloading workaround works on windows only (see comment above)

[FibreFoX]

On Mac having such wrapper might not work properly:
oracle/node-oracledb#231

This even was a problem on the z3 project itself: Z3Prover/z3#294

The outcome of the OpenJDK:

David's first recommendation is correct: don't use DYLD_LIBRARY_PATH. It's a debugging tool, not something that production software should be relying on (because it can be so easily tinkered with, because security, etc). Setting java.library.path to include the library directory is the ideal solution.

I will do further investigations, but I'm very sorry that when I don't find anything else, there is no real option to provide this, as the java.library.path was ment to work like this ... so it seems like a "not well designed" workaround of the z3 itself IMHO.

[Zomono]

Poorly DYLD_LIBRARY_PATH and LD_LIBRARY_PATH are the only option to load libs which have further dependencies. I must use these environment variables

[FibreFoX]

Just a stupid question: how is this solved in other languages?! I assume you are working in C/C++ too, so how is it solved in these languages without having LD_LIBRARY_PATH set?

[Zomono]

They restart their application with all needed environment variables set. This is not a solution for me because this would double the startup time.

[FibreFoX]

Searching through the web does not show any good news for LD_LIBRARY_PATH, especially because it is not cross-system compatible and harmful:

http://linuxmafia.com/faq/Admin/ld-lib-path.html

LD_LIBRARY_PATH is used in preference to any run time or default system linker path. If (God forbid) you had it set to something like /dcs/spod/baduser/lib, if there was a hacked version of libc in that directory (for example) your account could be compromised. It is for this reason that set-uid programs completely ignore LD_LIBRARY_PATH.

http://xahlee.info/UnixResource_dir/_/ldpath.html

If you are forced to set LD_LIBRARY_PATH, do so only as part of a wrapper.

http://stackoverflow.com/a/695539/1961102

Looks like -R is Solaris only, and you're on Linux.

https://web.archive.org/web/20050204070034/http://www.visi.com/~barr/ldpath.html

For security reasons, LD_LIBRARY_PATH is ignored at runtime for executables that have their setuid or setgid bit set. This severely limits the usefulness of LD_LIBRARY_PATH.

http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

LD_LIBRARY_PATH is handy for development and testing, but shouldn't be modified by an installation process for normal use by normal users

Workaround by loading the libz3/z3 first does work, so no need to fix some bug inside native loaders/files, especially not from this gradle-plugin.

[Zomono]

Workaround by loading the libz3/z3 first does work, so no need to fix some bug inside native loaders/files, especially not from this gradle-plugin.

Only works on Windows.

Wont fix

Environment variables are a common tool to configure programs. There are maybe a lot of other use cases, where it would be great to make this gradle-plugin do the thing.

Thanks for your time and your investigations!

[FibreFoX]

@Zerlono sorry to not providing you such feature, as it is nothing even possible with normal tools provided by the JDK itself using javapackager. This is very hacky and is different on every system, and I can't find a good and working way to provide this. You are required to implement your own wrapper-scripts for this.

[Zomono]

Yeah I see. Maybe the javapackager project is the one that should implement such a feature first.