Segmentation fault when running extract_feature

[kerolos]

hello ,
i got a segmentation fault when extracting features i dont know how can i solve this problem.

caffe-master> ./extract_features models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc8 examples/_temp/features 10 leveldb GPU
E0705 15:04:16.180382 6323 extract_features.cpp:52] Using GPU
E0705 15:04:16.180963 6323 extract_features.cpp:58] Using Device_id=0
Segmentation fault

thanks in advance.

[seanbell]

From https://github.com/BVLC/caffe/blob/master/CONTRIBUTING.md:

When reporting a bug, it's most helpful to provide the following information, where applicable:

• What steps reproduce the bug?
• Can you reproduce the bug using the latest master, compiled with the DEBUG make option?
• What hardware and operating system/distribution are you running?
• If the bug is a crash, provide the backtrace (usually printed by Caffe; always obtainable with gdb).

[kerolos]

-Dug :

caffe-master> gdb --args ./extract_features models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc8 examples/_temp/features 10 leveldb GPU
GNU gdb (GDB; openSUSE Leap 42.1) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://bugs.opensuse.org/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./extract_features...done.
(gdb) run
Starting program: /home_local/caffe-master/extract_features models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc8 examples/_temp/features 10 leveldb GPU
Missing separate debuginfos, use: zypper install glibc-debuginfo-2.19-19.1.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
E0708 13:40:08.751108 15738 extract_features.cpp:52] Using GPU
E0708 13:40:08.751610 15738 extract_features.cpp:58] Using Device_id=0
Detaching after fork from child process 15742.
[New Thread 0x7fffd7fff700 (LWP 15744)]
[New Thread 0x7fffdfec1700 (LWP 15743)]

Program received signal SIGSEGV, Segmentation fault.
0x00007fffea1c7990 in std::string::append(std::string const&) () from /usr/lib64/libstdc++.so.6
Missing separate debuginfos, use: zypper install Mesa-libGL1-debuginfo-11.0.8-148.1.x86_64 Mesa-libglapi0-debuginfo-11.0.8-148.1.x86_64 fontconfig-debuginfo-2.11.0-3.2.x86_64 libHalf12-debuginfo-2.2.0-37.1.x86_64 libICE6-debuginfo-1.0.9-3.1.x86_64 libIex-2_2-12-debuginfo-2.2.0-37.1.x86_64 libIlmImf-2_2-22-debuginfo-2.2.0-37.1.x86_64 libIlmThread-2_2-12-debuginfo-2.2.0-37.1.x86_64 libSM6-debuginfo-1.2.2-5.1.x86_64 libX11-6-debuginfo-1.6.3-3.1.x86_64 libX11-xcb1-debuginfo-1.6.3-3.1.x86_64 libXau6-debuginfo-1.0.8-6.1.x86_64 libXcursor1-debuginfo-1.1.14-6.1.x86_64 libXdamage1-debuginfo-1.1.4-9.1.x86_64 libXext6-debuginfo-1.3.3-3.1.x86_64 libXfixes3-debuginfo-5.0.1-5.1.x86_64 libXi6-debuginfo-1.7.5-1.1.x86_64 libXinerama1-debuginfo-1.1.3-5.1.x86_64 libXrandr2-debuginfo-1.5.0-1.1.x86_64 libXrender1-debuginfo-0.9.9-1.1.x86_64 libXxf86vm1-debuginfo-1.1.4-1.1.x86_64 libblas3-debuginfo-3.5.0-8.2.x86_64 libbz2-1-debuginfo-1.0.6-32.1.x86_64 libcblas3-debuginfo-20110120-3.2.x86_64 libdrm2-debuginfo-2.4.65-1.1.x86_64 libexpat1-debuginfo-2.1.0-17.1.x86_64 libffi4-debuginfo-5.3.1+r233831-6.1.x86_64 libfreetype6-debuginfo-2.5.5-8.2.x86_64 libgcc_s1-debuginfo-5.3.1+r233831-6.1.x86_64 libgfortran3-debuginfo-5.3.1+r233831-6.1.x86_64 libglib-2_0-0-debuginfo-2.44.1-2.3.x86_64 libgobject-2_0-0-debuginfo-2.44.1-2.3.x86_64 libhdf5-10-debuginfo-1.8.15-3.9.x86_64 libhdf5_hl10-debuginfo-1.8.15-3.9.x86_64 libjasper1-debuginfo-1.900.1-164.1.x86_64 libjbig2-debuginfo-2.0-3.2.x86_64 libjpeg8-debuginfo-8.0.2-31.3.x86_64 liblapack3-debuginfo-3.5.0-8.2.x86_64 libleveldb1-debuginfo-1.18-2.3.x86_64 liblmdb-0_9_14-debuginfo-0.9.14-4.1.x86_64 liblzma5-debuginfo-5.0.5-3.5.x86_64 libopencv3_1-debuginfo-3.1.0-88.12.x86_64 libpcre1-debuginfo-8.33-3.5.x86_64 libpng16-16-debuginfo-1.6.8-7.1.x86_64 libprotobuf9-debuginfo-2.6.1-5.1.x86_64 libpython2_7-1_0-debuginfo-2.7.9-21.3.x86_64 libqt4-debuginfo-4.8.6-10.2.x86_64 libqt4-x11-debuginfo-4.8.6-10.2.x86_64 libquadmath0-debuginfo-5.3.1+r233831-6.1.x86_64 libsnappy1-debuginfo-1.1.3-1.1.x86_64 libstdc++6-debuginfo-5.3.1+r233831-6.1.x86_64 libtiff5-debuginfo-4.0.6-3.1.x86_64 libuuid1-debuginfo-2.25-15.2.x86_64 libxcb-dri2-0-debuginfo-1.11.1-1.1.x86_64 libxcb-dri3-0-debuginfo-1.11.1-1.1.x86_64 libxcb-glx0-debuginfo-1.11.1-1.1.x86_64 libxcb-present0-debuginfo-1.11.1-1.1.x86_64 libxcb-sync1-debuginfo-1.11.1-1.1.x86_64 libxcb1-debuginfo-1.11.1-1.1.x86_64 libxshmfence1-debuginfo-1.2-5.3.x86_64 libz1-debuginfo-1.2.8-6.4.x86_64
(gdb) quit
A debugging session is active.

Inferior 1 [process 15738] will be killed.

Quit anyway? (y or n) y
ghob_ke@rmc-lx0181:/home_local/caffe-master> echo $ARCH

ghob_ke@rmc-lx0181:/home_local/caffe-master> gdb --args ./extract_features models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc8 examples/_temp/features 10 leveldb GPU
GNU gdb (GDB; openSUSE Leap 42.1) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://bugs.opensuse.org/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./extract_features...done.
(gdb) run
Starting program: /home_local/caffe-master/extract_features models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc8 examples/_temp/features 10 leveldb GPU
Missing separate debuginfos, use: zypper install glibc-debuginfo-2.19-19.1.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
E0708 13:42:17.459518 16078 extract_features.cpp:52] Using GPU
E0708 13:42:17.460166 16078 extract_features.cpp:58] Using Device_id=0
Detaching after fork from child process 16082.
[New Thread 0x7fffd7fff700 (LWP 16084)]
[New Thread 0x7fffdfec1700 (LWP 16083)]

Program received signal SIGSEGV, Segmentation fault.
0x00007fffea1c7990 in std::string::append(std::string const&) () from /usr/lib64/libstdc++.so.6
Missing separate debuginfos, use: zypper install Mesa-libGL1-debuginfo-11.0.8-148.1.x86_64 Mesa-libglapi0-debuginfo-11.0.8-148.1.x86_64 fontconfig-debuginfo-2.11.0-3.2.x86_64 libHalf12-debuginfo-2.2.0-37.1.x86_64 libICE6-debuginfo-1.0.9-3.1.x86_64 libIex-2_2-12-debuginfo-2.2.0-37.1.x86_64 libIlmImf-2_2-22-debuginfo-2.2.0-37.1.x86_64 libIlmThread-2_2-12-debuginfo-2.2.0-37.1.x86_64 libSM6-debuginfo-1.2.2-5.1.x86_64 libX11-6-debuginfo-1.6.3-3.1.x86_64 libX11-xcb1-debuginfo-1.6.3-3.1.x86_64 libXau6-debuginfo-1.0.8-6.1.x86_64 libXcursor1-debuginfo-1.1.14-6.1.x86_64 libXdamage1-debuginfo-1.1.4-9.1.x86_64 libXext6-debuginfo-1.3.3-3.1.x86_64 libXfixes3-debuginfo-5.0.1-5.1.x86_64 libXi6-debuginfo-1.7.5-1.1.x86_64 libXinerama1-debuginfo-1.1.3-5.1.x86_64 libXrandr2-debuginfo-1.5.0-1.1.x86_64 libXrender1-debuginfo-0.9.9-1.1.x86_64 libXxf86vm1-debuginfo-1.1.4-1.1.x86_64 libblas3-debuginfo-3.5.0-8.2.x86_64 libbz2-1-debuginfo-1.0.6-32.1.x86_64 libcblas3-debuginfo-20110120-3.2.x86_64 libdrm2-debuginfo-2.4.65-1.1.x86_64 libexpat1-debuginfo-2.1.0-17.1.x86_64 libffi4-debuginfo-5.3.1+r233831-6.1.x86_64 libfreetype6-debuginfo-2.5.5-8.2.x86_64 libgcc_s1-debuginfo-5.3.1+r233831-6.1.x86_64 libgfortran3-debuginfo-5.3.1+r233831-6.1.x86_64 libglib-2_0-0-debuginfo-2.44.1-2.3.x86_64 libgobject-2_0-0-debuginfo-2.44.1-2.3.x86_64 libhdf5-10-debuginfo-1.8.15-3.9.x86_64 libhdf5_hl10-debuginfo-1.8.15-3.9.x86_64 libjasper1-debuginfo-1.900.1-164.1.x86_64 libjbig2-debuginfo-2.0-3.2.x86_64 libjpeg8-debuginfo-8.0.2-31.3.x86_64 liblapack3-debuginfo-3.5.0-8.2.x86_64 libleveldb1-debuginfo-1.18-2.3.x86_64 liblmdb-0_9_14-debuginfo-0.9.14-4.1.x86_64 liblzma5-debuginfo-5.0.5-3.5.x86_64 libopencv3_1-debuginfo-3.1.0-88.12.x86_64 libpcre1-debuginfo-8.33-3.5.x86_64 libpng16-16-debuginfo-1.6.8-7.1.x86_64 libprotobuf9-debuginfo-2.6.1-5.1.x86_64 libpython2_7-1_0-debuginfo-2.7.9-21.3.x86_64 libqt4-debuginfo-4.8.6-10.2.x86_64 libqt4-x11-debuginfo-4.8.6-10.2.x86_64 libquadmath0-debuginfo-5.3.1+r233831-6.1.x86_64 libsnappy1-debuginfo-1.1.3-1.1.x86_64 libstdc++6-debuginfo-5.3.1+r233831-6.1.x86_64 libtiff5-debuginfo-4.0.6-3.1.x86_64 libuuid1-debuginfo-2.25-15.2.x86_64 libxcb-dri2-0-debuginfo-1.11.1-1.1.x86_64 libxcb-dri3-0-debuginfo-1.11.1-1.1.x86_64 libxcb-glx0-debuginfo-1.11.1-1.1.x86_64 libxcb-present0-debuginfo-1.11.1-1.1.x86_64 libxcb-sync1-debuginfo-1.11.1-1.1.x86_64 libxcb1-debuginfo-1.11.1-1.1.x86_64 libxshmfence1-debuginfo-1.2-5.3.x86_64 libz1-debuginfo-1.2.8-6.4.x86_64
(gdb) bt
#0 0x00007fffea1c7990 in std::string::append(std::string const&) () at /usr/lib64/libstdc++.so.6
#1 0x00007ffff75ff60c in std::operator+<char, std::char_traits, std::allocator >(std::basic_string<char, std::char_traits, std::allocator > const&, std::basic_string<char, std::char_traits, std::allocator > const&) (lhs="", rhs=<error reading variable: Cannot access memory at address 0x0>) at /usr/include/c++/4.8/bits/basic_string.h:2369
#2 0x00007ffff7600a1f in caffe::ImageDataLayer::DataLayerSetUp(std::vectorcaffe::Blob<float, std::allocatorcaffe::Blob<float> > const&, std::vectorcaffe::Blob<float, std::allocatorcaffe::Blob<float> > const&) (this=0x552e740, bottom=..., top=std::vector of length 2, capacity 2 = {...}) at /volume/USERSTORE/ried_sa/software/deeplearning/caffe/src/caffe/layers/image_data_layer.cpp:66
#3 0x00007ffff75827c3 in caffe::BasePrefetchingDataLayer::LayerSetUp(std::vectorcaffe::Blob<float_, std::allocatorcaffe::Blob<float_> > const&, std::vectorcaffe::Blob<float_, std::allocatorcaffe::Blob<float_> > const&) (this=0x552e740, bottom=..., top=...) at /volume/USERSTORE/ried_sa/software/deeplearning/caffe/src/caffe/layers/base_data_layer.cpp:48
#4 0x00007ffff76620c5 in caffe::Net::Init(caffe::NetParameter const&) (top=std::vector of length 2, capacity 2 = {...}, bottom=std::vector of length 0, capacity 0, this=0x552e740)
at /volume/USERSTORE/ried_sa/software/deeplearning/caffe/include/caffe/layer.hpp:71
#5 0x00007ffff76620c5 in caffe::Net::Init(caffe::NetParameter const&) (this=this@entry=0x51ee5d0, in_param=...) at /volume/USERSTORE/ried_sa/software/deeplearning/caffe/src/caffe/net.cpp:148
#6 0x00007ffff7663825 in caffe::Net::Net(std::string const&, caffe::Phase, caffe::Net const_) (this=0x51ee5d0, param_file=..., phase=caffe::TEST, root_net=)
at /volume/USERSTORE/ried_sa/software/deeplearning/caffe/src/caffe/net.cpp:36
#7 0x000000000040a67a in feature_extraction_pipeline(int, char_*) (argc=, argv=0x7fffffffe328) at /volume/USERSTORE/ried_sa/software/deeplearning/caffe/tools/extract_features.cpp:98
#8 0x00007fffe9854b05 in __libc_start_main () at /lib64/libc.so.6
#9 0x00000000004062ce in _start () at ../sysdeps/x86_64/start.S:122
(gdb) quit
A debugging session is active.