Compiling with CUDA on Ubuntu 16.04 + Solution

[kleinric]

Compiling on a clean install of Ubuntu 16.04 with cmake and using CUDA from the repo.

I get the following error:

/usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ /usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ /usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ /usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ /usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ /usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ /usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’: /usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope return (char *) memcpy (__dest, __src, __n) + __n; ^ CMake Error at cuda_compile_generated_sgd_solver.cu.o.cmake:266 (message): Error generating file /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/solvers/./cuda_compile_generated_sgd_solver.cu.o
Seems TensorFlow has similar issues: https://github.com/fayeshine/tensorflow/commit/6c8c572c12521d706eda692fa7793f90b45dde20

By adding that flag to my CMakeLists.txt it fixed the problem and compiles fine. ie. Add this near the front:
set(${CMAKE_CXX_FLAGS} "-D_FORCE_INLINES ${CMAKE_CXX_FLAGS}")

Someone might be able to add this to the cmake file.

[lukeyeager]

This helped me and several other users on 16.04 as well:
https://groups.google.com/d/msg/caffe-users/Tm3OsZBwN9Q/XKGRKNdmBAAJ

EDIT

The CUDA 8.0 RC works great. Just use that.

[neouyghur]

I have same problem, but the solution is not working for me.

[neouyghur]

At first this solution does not work, however after I have download the newest Caffe, then compile with this modification it works. But I still got some error at undefined reference to `lzma_index_end@XZ_5.0'.

[macfer]

It worked for me on 16.04 but I added the flag to Makefile rather then to CMakeLists.txt as I had some other workarounds applied to Makefile.config and wasn't interested in switching to cmake build.

If you are compiling with make then edit Makefile and replace the line
NVCCFLAGS += -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
with
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS).

[swearos]

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORCE_INLINES"),
add the line in the front of the CMakeLists.txt
the gcc is too new;

[bipedalBit]

@macfer thanks a lot. It seems Makefile configuring methods really counts.

[Gwyki]

Ubuntu 16.04, GCC 5.3.1

Ok, so I have applied all the workarounds I can find:

1. NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
2. Removed GCC version fence in /usr/local/cuda/include/host_config.h
3. CXXFLAGS += -D_GLIBCXX_USE_CXX11_ABI=0

The protobuf builds coming from the 16.04 repo would just not link. In the end I had to grab the source and compile manually:

cd /usr/lib/X86_64-linux-gnu
sudo mkdir libprotobufOLD
mv libprotobuf* libprotobufOLD
cd ~
apt-get source libprotobuf-dev
cd protobuf-2.6.1
./configure
make -j8
sudo make install
sudo ldconfig

I was somewhat desperate to get it running ASAP and would prefer to not have to butcher the original protobuf libs but it didn't seem like uninstalling them cleanly was possible. Anyone have comments on why this was necessary?

edit: Don't be silly like me. Don't move the original protobuf libs they are of course needed. For whatever reason after moving them back and removing the ones I compiled from source, everything still compiles, with ldd libcaffe.so showing it is linked with the original system protobuf libs... Puzzling.

[hikarido]

Similar answer described here https://github.com/BVLC/caffe/wiki/Ubuntu-16.04-or-15.10-Installation-Guide

[raffienficiaud]

Actually, tweaking CUDA_NVCC_FLAGS would be a better choice. I have this working:

cmake ... -DCUDA_NVCC_FLAGS="-D_FORCE_INLINES" ..

and it works perfectly.

[dozyc]

the CMAKE_CXX_FLAGS fix didn't seem to have an effect, so I ended up adding some lines to cmake/Cuda.cmake around line 260:

# memcpy workaround
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} \"-D_FORCE_INLINES\" ")

[shelhamer]

This Ubuntu 16.04 issue is resolved by CUDA 8.