Issues when building HDF5 with libgfortran on Mac

[astrofrog]

I'm trying to get the HDF5 conda-forge to build with the fortran bindings:

https://travis-ci.org/conda-forge/hdf5-feedstock/builds/145963897

This isn't working on MacOS X due to the following error:

dyld: Library not loaded: @rpath/./libgfortran.3.dylib
  Referenced from: /Users/travis/miniconda3/conda-bld/work/hdf5-1.8.17/fortran/src/./H5test_kind
  Reason: image not found
/bin/sh: line 1: 28842 Trace/BPT trap: 5       ./H5test_kind > H5fortran_detect.f90
make[2]: *** [H5fortran_detect.f90] Error 133
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1
Command failed: /bin/bash -x -e /Users/travis/miniconda3/conda-bld/work/hdf5-1.8.17/conda_build.sh

However, the same recipe passes on CircleCI/Linux. The main difference between the two (apart from OS) is that the CIrcleCI build used libgfortran from the defaults channel:

https://circleci.com/gh/conda-forge/hdf5-feedstock/67

I'm not sure if this is related, but I'm opening this in case any of the libgfortran feedstock maintainers have any ideas. I'll also continue to investigate this (cc @pelson @jakirkham @msarahan)

[jakirkham]

This is not a problem with libgfortran. This is an issue with the rpath not being fixed yet. So potentially conda-build, but it really doesn't lie there either as conda-build hasn't had a chance to work its magic with install_name_tool. I would recommend setting the DYLD_FALLBACK_LIBRARY_PATH on OS X to $PREFIX/lib for this step.

[pelson]

P.S. You can use @conda-forge/libgfortran mention to get the full team pinged.

Not having looked at the recipe, I think @jakirkham is telling you that the make check is happening in the build phase, not the test phase, so conda-build hasn't had a chance to modify the RPATH within the installed shared objects. The standard workaround for this is to put those things in the LD search path at build time. It isn't a particularly nice solution, but it does the trick for the time being. I'm pretty sure there is functionality we could add to conda-build to allow the make check usecase.

[pelson]

Given the above, I think we can close this, right @astrofrog?

[astrofrog]

@pelson - yes, we can close the specific issue of it not being picked up at build time. However, see the latest comments in conda-forge/hdf5-feedstock#28 - it looks like this error is happening during the testing phase too, but I thought that at that point all the library paths should be correctly set up?

[astrofrog]

@pelson @jakirkham - I think this is actually a bug in the conda gcc package: conda/conda#3241

By the way, what is the relation between the libgfortran and gcc conda packages - don't they conflict since they both install libgfortran library files?

[jakirkham]

Looked at this the other day and it appears the loader_path may be improperly set with this package. So am going to reopen this issue. It is my suspicion that if we fix the loader_path some of these issues should go away.

@pelson @jakirkham - I think this is actually a bug in the conda gcc package: conda/conda#3241

I think I agree with you. I don't know this, but it is my suspicion that the gcc package from defaults was not actually built with conda-build. If it were built with conda-build, it would have had the loader_path set correctly, which does not appear to be the case. Also it would have properly handled the PREFIX in the libtool files (*.la). See this comment and the one below.

By the way, what is the relation between the libgfortran and gcc conda packages - don't they conflict since they both install libgfortran library files?

We made this package just for OS X. defaults provides one for Linux and we decided we did not want to enter that territory. It simply keeps the needed files from gcc. So if gcc and libgfortran are installed on OS X there is no conflict as libgfortran is part of gcc.

[jakirkham]

Maybe PR ( #3 ) will fix it. Though there are a few cases it might still not work. If you have access to OS X, could you, @astrofrog, please try testing this with something (presumably HDF5) and see if it fixes the problem?

[jakirkham]

Are these still issues @astrofrog ?

[astrofrog]

I haven't had any issues recently - maybe we can close and re-open if this comes up again?

[jakirkham]

Thanks for letting us know. Happy to reopen if these recur.

[bhomass]

I am not sure why this is closed. Is there some fix in conda that took care of this, because I am having issues with this right now. Is there something a user can do to get around this problem? Do I need a newer version of conda?

[jakirkham]

If you are seeing issues with hdf5, @bhomass, can you please open a new issue on that repo with a MCVE? Thanks.

[QM-Fitz]

I find this is helpful, I have solve the problem http://wusun.name/blog/2018-04-27-conda-libgfortran-issue/
"The fix is simple. Switch the default gfortran back to the Anaconda one. Check the path of DYLD_FALLBACK_LIBRARY_PATH. If it is empty, set it to the Anaconda lib directory, e.g., something like /Users/<your_name>/anaconda/lib. That's it. You may want to add it to the .bash_profile because macOS seems to constantly clear it. (Though I suspect that there may be a more elegant solution, at the moment this solved my problem.)" I add "export DYLD_FALLBACK_LIBRARY_PATH=$DYLD_FALLBACK_LIBRARY_PATH:/Users//miniconda3/lib" to .bash_profile and then source ~/.bash_profile .