Java bindings not loading in OS X 10.11 (El Capitan)

[shurickdaryin]

OS X 10.11 does not allow DYLD_LIBRARY_PATH environment variable to be propagated to child processes. As such, it is not possible to find libmpi.dylib:

mpirun -np 4 java -jar myapp.jar
Java bindings failed to load libmpi: dlopen(libmpi.dylib, 10): image not found
Java bindings failed to load libmpi: dlopen(libmpi.dylib, 10): image not found
Java bindings failed to load libmpi: dlopen(libmpi.dylib, 10): image not found
Java bindings failed to load libmpi: dlopen(libmpi.dylib, 10): image not found
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[9970,1],3]
  Exit code:    1
--------------------------------------------------------------------------

See oracle/node-oracledb#231, http://apple.stackexchange.com/questions/215030/el-capitan-make-check-dyld-library-path

A workaround is to copy libmpi.dylib to the current directory.

Probably a full path name should be used to load libmpi.dylib to avoid using DYLD_LIBRARY_PATH.

[ggouaillardet]

@shurickdaryin did you configure with --enable-mpirun-prefix-by-default ?
iirc, that should fix the issue

[shurickdaryin]

@ggouaillardet I reconfigured with --enable-mpirun-prefix-by-default and rebuilt openmpi and my app, but still get the same error.

[ggouaillardet]

were you able to test with Yosemite ? and did you get the same error ?
btw, do you run on only one node ?(vs several nodes)
a possible fix is to update ompi/mpi/java/c/mpi_MPI.c and dlopen with the full path if ompi is configured with --enabke-mpirun-prefix-by-default
I am still running yosemite, so I might ask you to test a fix tomorrow

[shurickdaryin]

No, it's El Capitan all around me. But I have tried to hard code the path to libmpi.dylib in the indicated file and it works. So the proposed fix should also work.

[shurickdaryin]

This has been run on one node.

[jsquyres]

@hppritcha Can you confirm? If so, it may be necessary to use installdirs to find libmpi at run time.

[hppritcha]

As I recall one needs to pretend LD_LIBRARY_PATH with location of OMPI install lib on os-x. It's owing to the dlopen of libmpi iin the JNI code. This isn't necessary on Linux. I don't think this is specific to el capitan. I will update the java FAQ.

[shurickdaryin]

Setting LD_LIBRARY_PATH or DYLD_LIBRARY_PATH does not change anything, as these variables are not exported to child processes. This behaviour is new with El Capitan.

I have both LD_LIBRARY_PATH and DYLD_LIBRARY_PATH set to /opt/openmpi/lib. With this setting, "otool -L libmpi.dylib" returns an error, while "otool -L /opt/openmpi/lib/libmpi.dylib" correctly reports library and its dependencies.

Another manifestation of this behaviour is that "LD_LIBRARY_PATH=foo env | grep LD" doesn't print anything, while "_LD_LIBRARY_PATH=foo env | grep LD" prints "_LD_LIBRARY_PATH=foo".

[jsquyres]

@hppritcha Ouch -- it sounds like El Cap made a Big Change in this area! This code might need to be updated per what I said above (i.e., use installdirs to find libmpi)...