PyCaffe import failing "Library not loaded ... Reason: image not found"

[damienstanton]

Running OS X 10.10.4 I have compiled successfully against Anaconda and Matlab, completed all installation steps and fixed the known boost incompatibility problem.

However Caffe still fails to import properly in Python (executed again in iPython for readability):

Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, May 28 2015, 17:04:42)
Type "copyright", "credits" or "license" for more information.

IPython 3.2.0 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import caffe
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-1cca3aa1f8c5> in <module>()
----> 1 import caffe

/Users/damien/caffe/python/caffe/__init__.py in <module>()
----> 1 from .pycaffe import Net, SGDSolver
      2 from ._caffe import set_mode_cpu, set_mode_gpu, set_device, Layer, get_solver
      3 from .proto.caffe_pb2 import TRAIN, TEST
      4 from .classifier import Classifier
      5 from .detector import Detector

/Users/damien/caffe/python/caffe/pycaffe.py in <module>()
     11 import numpy as np
     12
---> 13 from ._caffe import Net, SGDSolver
     14 import caffe.io
     15

ImportError: dlopen(/Users/damien/caffe/python/caffe/_caffe.so, 2): Library not loaded: @rpath/libcudart.7.0.dylib
  Referenced from: /Users/damien/caffe/python/caffe/_caffe.so
  Reason: image not found

Since everything tested and compiled correctly, and the caffe module is located, I can't figure out why the library is not loading properly. I've tried reinstalling the Cuda driver and toolkit which did not make a difference.

[mitchellzen]

almost same issue here. followed all instructions and fixed the boost incompatibility problem, import gives me this error, though hd5 is installed:

Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, May 28 2015, 17:04:42) 
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import caffe
Traceback (most recent call last):
  File "", line 1, in 
  File "caffe/__init__.py", line 1, in 
    from .pycaffe import Net, SGDSolver
  File "caffe/pycaffe.py", line 13, in 
    from ._caffe import Net, SGDSolver
ImportError: dlopen(caffe/_caffe.so, 2): Library not loaded: libhdf5_hl.10.dylib
  Referenced from: /Users/z/python/caffe/python/caffe/_caffe.so
  Reason: image not found

[alexander-cohen]

Has anyone figured this out? I have the same issue:

Traceback (most recent call last): File "", line 1, in File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/caffe/**init**.py", line 1, in from .pycaffe import Net, SGDSolver File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/caffe/pycaffe.py", line 13, in from ._caffe import Net, SGDSolver ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/caffe/_caffe.so, 2): Library not loaded: @rpath/libcaffe.so Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/caffe/_caffe.so Reason: image not found

I am using macports installed caffe, all the dependencies also form macports. I did not do any of the compilation stuff on their site, because I did not know how to make it work with the macports install. I would suspect that this is the source of the issue. Any help/advice?

[damienstanton]

So I was able to fix the libcudart issue by adding export DYLD_LIBRARY_PATH='/usr/local/cuda/lib/' to my bash source file. I am now getting the same hdf5 error that @mitchellzen sees.

Will investigate this one now.

[this-sam]

I am also on OSX 10.10 and encountering the "Library not loaded: @rpath/libcaffe.so" issue. I followed this install guide by robertsdionne, fixed the boost issue like this, and have tried every combination of manually setting the DYLD_LIBRARY_PATH and DYLD_FALLBACK_LIBRARY_PATH that I can think of (or have come across online). Has anyone used other solutions?

I have the following relevant changes to my .bash_profile:

export PYTHONPATH=$PYTHONPATH:/Users/X/Development/caffe/distribute/python/
export DYLD_LIBRARY_PATH='/usr/local/cuda/lib/'
export DYLD_FALLBACK_LIBRARY_PATH=/usr/local/cuda/lib:/usr/local/lib:/usr/lib:
export LD_LIBRARY_PATH=/usr/local/cuda/lib:/usr/local/lib:/Users/X/Develop/caffe/distribute/lib

[damienstanton]

Okay no fix yet but I think I found the source of this issue: a glance at /usr/local/lib shows libhdf5_hl.9.dylib while as shown above PyCaffe is looking for libhdf5_hl.10.dylib.

I tried brew-editing homebrew/OpenCV and also recompiled Caffe, but still no change.

Does anyone know how to get the appropriate version of libhdf5 installed? Brew seems to be showing the most up to date package so this is a bit odd.

[this-sam]

I have solved the issue I personally was having by reviewing this guide. I changed the following in my bash_profile:

export PYTHONPATH=$PYTHONPATH:/Users/X/Development/caffe/distribute/python/

to

export PYTHONPATH=$PYTHONPATH:/Users/X/Development/caffe/python/

(Removed the distribute part of the path.)

Hopefully this will help someone solving the same issue.

[damienstanton]

That didn't work for me (I followed the same guide, thanks for posting here). My PYTHONPATH is the same as the latter in your example.

[creynold]

Try removing the brew install of hdf5 if you're using anaconda. Anaconda comes packaged with hdf5 (~/anaconda/lib has the dylib files). You can alternatively copy the files from the anaconda directory into /usr/local/lib.

[damienstanton]

@creynold that was smart, thanks. Resolves the hdf5 dylib error. Getting a segfault now, however:

/Users/damien/anaconda/bin/python.app: line 3:  1811 Segmentation fault: 11  /Users/damien/anaconda/python.app/Contents/MacOS/python "$@"

I tried resolving per the solutions here but I believe I am already referencing the correct (Anaconda) Python. Any ideas?

[YearnyeenHo]

Hello, I have encountered the problem:
dyld: Library not loaded: libhdf5_hl.10.dylib
and it is so weird thatbrew install hdf5 can only installed the 1.8.14 version,
So，I directly downloaded the source code of 1.8.15 version from HDF website,
followed the sample install instructions and installed it, and could find a new folder hdf5 under /usr/local.
We can see that there is already a folder called '1.8.14' under /usr/local/Cellar/hdf5 if you have installed the hdf5 through brew.
Clone the folder /usr/local/hdf5 into the /usr/local/Cellar/hdf5, and rename it to1.8.15.
run brew switch hdf5 1.8.15 to make brew switch to the specific version.
And now I can run make runtest successfully.

[shelhamer]

Closing as this looks like a usage issue/installation question.

This tracker is reserved for specific Caffe development issues and bugs; please ask usage questions on the caffe-users list.

For more information, see our contributing guide.

Thanks!

[kunalghosh]

Just incase anyone else is stuck with this issue.

I was facing the same problem dyld: Library not loaded: libhdf5_hl.10.dylib while running make runtest and also while importing caffe in my python interpreter.

I am using the anaconda python distribution and adding the libs to DYLD_FALLBACK_LIBRARY_PATH fixed the problem.

export ANACONDA_HOME=$HOME/anaconda
export DYLD_FALLBACK_LIBRARY_PATH=$ANACONDA_HOME/lib:/usr/local/lib:/usr/lib

I followed this wiki.

[compartia]

thanks, kunalghosh!

[xingdi1990]

thanks, kunalghosh

[RenfengLiu]

I have the same error on OS X 10.11.1, with anaconda. I tried all the methods list above, but none of them worked.

$ make runtest
.build_release/tools/caffe
dyld: Library not loaded: libhdf5.10.dylib
  Referenced from: /project/caffe/.build_release/tools/caffe
  Reason: image not found
make: *** [runtest] Trace/BPT trap: 5

[gsabran]

similar issue here (OSX 10.11 with anaconda as well)

ImportError                               Traceback (most recent call last)
<ipython-input-1-1cca3aa1f8c5> in <module>()
----> 1 import caffe

/Users/guigou/caffe/distribute/python/caffe/__init__.py in <module>()
----> 1 from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver
      2 from ._caffe import set_mode_cpu, set_mode_gpu, set_device, Layer, get_solver, layer_type_list
      3 from .proto.caffe_pb2 import TRAIN, TEST
      4 from .classifier import Classifier
      5 from .detector import Detector

/Users/guigou/caffe/distribute/python/caffe/pycaffe.py in <module>()
     11 import numpy as np
     12 
---> 13 from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
     14         RMSPropSolver, AdaDeltaSolver, AdamSolver
     15 import caffe.io

ImportError: dlopen(/Users/guigou/caffe/distribute/python/caffe/_caffe.so, 2): Library not loaded: @rpath/libcaffe.so
  Referenced from: /Users/guigou/caffe/distribute/python/caffe/_caffe.so
  Reason: image not found

No idea of what's going on...

[jpipas]

Ran into this as well after the upgrade to 10.11 - decided to try and rebuild caffe - huge headache.

After fixing - e.g. downgrading to - boost 1.57 (and boost-python), messing around with OpenBLAS (because I couldn't get it to find atlas), and having to use cmake - I was finally able to get make all, and make runtests to finish without major errors (not without warnings though).

Running make pycaffe completes - but when I try import caffe from python cli - I get the following:

$ python
Python 2.7.10 (default, Sep 23 2015, 04:34:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Fatal Python error: PyThreadState_Get: no current thread
Abort trap: 6

I think I'm now running into issues where OSx python and Homebrew python are crossing the streams... resulting in total protonic reversal...

Scratching my head on what to do next.

[orsharir]

It appears that El Capitan made changes to how the OS treat DYLD_ prefix variables - specifically, it seems that the OS unset the variables when running scripts (e.g. the ipython command is usually a script, or the runtests command). I couldn't find any official documentation about this behavior, but there are references from other open source projects:

• https://github.com/kubo/fix_oralib_osx
• http://www.wine-reviews.net/2015/10/winebottler-1752-has-been-released-now.html

It seems that the way to solve this is to remove the dependency on the DYLD_FALLBACK_LIBRARY_PATH environment variable, as suggested by the second link above:

First, Apple got rid of LD_LIBRARY_PATH and DYLD_FALLBACK_LIBRARY_PATH. Forcing us to “otool -L” and “install_name_tool -change” thru all the bins and libs of Wine to use proper @ rpath relative paths for all Wine related libraries. The result is, of course, a correct linking through out the app – thank you Apple. I guess somebody has to kick you once a while to come up with proper solutions.

[gsabran]

Thanks @orsharir for finding this. That seems super relevant. I'm a bit lost now: is there something I should do on my side or wait for a kind of fix from caffe (or try to make one)?

[orsharir]

Long term I guess this should be fix officially by caffe, by removing the dependence on DYLD_FALLBACK_LIBRARY_PATH. I'm afraid I don't have the proficiency to make that fix myself.

For the near term you can turn off System Integrity Protection, which return the system to the previous behavior, but leave your system vulnerable to all kind of attacks. You can also try to work around some of the issues by creating .bashenv script (similar to .bash_profile) where you set your DYLD_FALLBACK_LIBRARY_PATH variable, and add to your .bash_profile the line 'export BASH_ENV=~/.bashenv' - this forces the system to run .bashenv before every non-interactive bash session (i.e. bash scripts). This last part resolved the problems I had with ipython, but still doesn't resolve the issue of running 'make runtest'.

[shelhamer]

@orsharir could you post a new issue with these El Capitan details so that we have an OS X build TODO to track? Thanks for looking into this.

[orsharir]

Of course. This is the issue I've opened #3227.

[dgs0323]

@jpipas I am running into the same issue that you posted. Did you find a way to resolve this?

[anbai106]

I have the same problem for my Macpro os x 10.11.6, .build_release/tools/caffe
dyld: Library not loaded: libhdf5_hl.10.dylib
Referenced from: /Users/junhao.wen/Hao/Docker/Caffe/Caffe_source/caffe/.build_release/tools/caffe
Reason: image not found
make: *** [runtest] Trace/BPT trap: 5

And i tried the insall_name_tool solution and the DYLD_FALLBACK_LIBRARY_PATH, neither of them works.

I guess this is a common problem for Mac users, dont you have some solution for this????
Really need your help,
Thanks in advance

[chm123]

1. Open up a terminal
2. Locate the path of the libhdf5_hl.10.dylib:
   locate libhdf5_hl.10.dylib
3. Add the path. For me, it is:
   install_name_tool -add_rpath '/usr/local/anaconda/lib' /usr/local/caffe/.build_release/tools/caffe

This solved my problem.

[shenhaihua002]

mac os,I see the http://akmetiuk.com/blog/2016/03/29/compiling-caffe.html to solve my problem.

[BluebirdStory]

I have solved this problem, does anyone need help?

[BluebirdStory]

It's sort of complicated I mean, so if there is no one needs help, I would not bother to board the solution.

[abouzaidan-tony]

@BluebirdStory i think i have a similar problem with pytorch v1.0.0
and i didnt have it with pytorch 0.4.1
when calling "import torch" in python 2.7
i get the following error, i think there's a linking error

import torch
Traceback (most recent call last):
File "", line 1, in
File "/Library/Python/2.7/site-packages/torch/init.py", line 84, in
from torch._C import *
ImportError: dlopen(/Library/Python/2.7/site-packages/torch/_C.so, 9): Library not loaded: @rpath/libc++.1.dylib
Referenced from: /Library/Python/2.7/site-packages/torch/_C.so
Reason: image not found

can you please help me fix this problem?
i am using python2.7 on my mac (maos mojave)

[abouzaidan-tony]

i have fix it running this command :
sudo install_name_tool -change @rpath/libc++.1.dylib /usr/lib/libc++.1.dylib /Library/Python/2.7/site-packages/torch/_C.so