Compilation failed, 'stddef.h' file not found

[jgreitemann]

I tried to run the example provided in the README.md file. However, the compilation of the mpi crate failed for me with the following output:

   Compiling mpi v0.1.7
failed to run custom build command for `mpi v0.1.7`
Process didn't exit successfully: `/home/jgreitemann/Documents/Programming/rust/rsmpi-test/target/debug/build/mpi-4dd1b45724a1a7b9/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("0")
PROFILE = Some("debug")
TARGET = Some("x86_64-unknown-linux-gnu")
debug=true opt-level=0
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = Some("mpicc")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
running: "mpicc" "-O0" "-c" "-ffunction-sections" "-fdata-sections" "-g" "-m64" "-fPIC" "-o" "/home/jgreitemann/Documents/Programming/rust/rsmpi-test/target/debug/build/mpi-4dd1b45724a1a7b9/out/src/rsmpi.o" "src/rsmpi.c"
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
AR_x86_64-unknown-linux-gnu = None
AR_x86_64_unknown_linux_gnu = None
HOST_AR = None
AR = None
TARGET = Some("x86_64-unknown-linux-gnu")
running: "ar" "crus" "/home/jgreitemann/Documents/Programming/rust/rsmpi-test/target/debug/build/mpi-4dd1b45724a1a7b9/out/librsmpi.a" "/home/jgreitemann/Documents/Programming/rust/rsmpi-test/target/debug/build/mpi-4dd1b45724a1a7b9/out/src/rsmpi.o"
cargo:rustc-link-lib=static=rsmpi
cargo:rustc-link-search=native=/home/jgreitemann/Documents/Programming/rust/rsmpi-test/target/debug/build/mpi-4dd1b45724a1a7b9/out
cargo:rustc-link-search=native=/usr/lib64/openmpi/lib

--- stderr
ar: `u' modifier ignored since `D' is the default (see `U')
warning: argument unused during compilation: '-L/usr/lib64/openmpi/lib' [-Wunused-command-line-argument]
/usr/include/openmpi-x86_64/mpi.h:223:10: fatal error: 'stddef.h' file not found
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: ()', ../src/libcore/result.rs:732

I'm running Fedora 22, Rust 1.3.0, and openmpi 1.8.7-1 from the Fedora repos.

[bsteinb]

Hm, can you compile a simple C program that includes mpi.h using mpicc?

[jgreitemann]

I did have an issue with my $PATH, but now compiling with mpicc works. The problem persists, though.

[bsteinb]

You are right. From the initial listing you provided it seems like mpicc is able to compile the small C shim library that rsmpi uses. The file not found error seems to come from bindgen which uses libclang while mpicc uses gcc on Fedora. I will investigate further.

[bsteinb]

I just installed Fedora 22 in a VM and tried to reproduce the problem without success (that is, I could successfully build rsmpi and also a small test). I have installed the packages openmpi, openmpi-devel, llvm and clang (and git) and I am also running rust 1.3.0. Could you make sure that you have those packages installed.

I also had an issue about mpicc not being in $PATH at first. Apparently that is solved by doing module load mpi.

[jgreitemann]

I didn't have llvm and clang installed. That did the trick. I didn't use LLVM before (obviously) and sticked with GCC. I should have realized it was required as rustc is based on it (or is that not the reason?). Anyway, I think there should be checks in place to ensure all dependencies are installed. Thanks for your time and effort.

[bsteinb]

rustc builds its own copy of LLVM and runs out of the box. rsmpi parses the MPI C header file using the bindgen crate which in turn uses libclang. I am a bit surprised that this did not cause any problems when bindgen was built as a dependency. Anyway, the error message in this case did not seem helpful at all, so maybe I will check whether all dependencies are installed or otherwise mention all dependencies in the README explicitly.

[cillian64]

I had this same problem on Ubuntu 16.04.1 -- If I installed libclang, the build failed with the stddef.h missing. Installing clang fixed the problem. If this is a known issue it would be useful to update the dependencies section to refer to clang rather than libclang.

[Rufflewind]

Ran into this problem while trying this on our HPC cluster, even though I installed clang and it works just fine by itself. My guess is that Clang is utilizing GCC's standard library, but on the HPC cluster the GCC standard library is installed in some weird location, which the frontend clang is able to find, but the libclang isn't able to.

What I ended up doing was to

(1) Run

echo '#include <stddef.h>' | mpicc -M -E -

to find out where the standard headers are. It should print something like:

-.o: \
  /some/weird/path/include/stddef.h

(2) Before building rsmpi, set the variable via

export C_INCLUDE_PATH=/some/weird/path/include"${C_INCLUDE_PATH+:}${C_INCLUDE_PATH-}"

Edit: Changed command to use mpicc instead of clang.

[Libbum]

Sorry to raise the dead here, but this seems to still be an issue. Trying to compile on arch linux, clang, openmpi etc are installed. I hit the 'stddef.h' file not found error and add the C_INCLUDE_PATH which Rufflewind suggests (this is /usr/lib/clang/4.0.0/include for me -- which isn't so odd really?) and generate the following warnings when running mpicc on src/ffi/rsmpi.c:

cargo:warning=In file included from /usr/include/mpi.h:225:0,
cargo:warning=                 from src/ffi/rsmpi.h:3,
cargo:warning=                 from src/ffi/rsmpi.c:1:
cargo:warning=/usr/lib/clang/4.0.0/include/stddef.h:34:19: error: missing binary operator before token "("
cargo:warning= #if !__has_feature(modules)
cargo:warning=                   ^
cargo:warning=/usr/lib/clang/4.0.0/include/stddef.h:46:42: error: missing binary operator before token "("
cargo:warning= #if !defined(_PTRDIFF_T) || __has_feature(modules)
cargo:warning=                                          ^
cargo:warning=/usr/lib/clang/4.0.0/include/stddef.h:57:39: error: missing binary operator before token "("
cargo:warning= #if !defined(_SIZE_T) || __has_feature(modules)
cargo:warning=                                       ^
cargo:warning=/usr/lib/clang/4.0.0/include/stddef.h:71:42: error: missing binary operator before token "("
cargo:warning=      !defined(_RSIZE_T)) || __has_feature(modules)
cargo:warning=                                          ^
cargo:warning=/usr/lib/clang/4.0.0/include/stddef.h:83:40: error: missing binary operator before token "("
cargo:warning= #if !defined(_WCHAR_T) || __has_feature(modules)
cargo:warning=                                        ^
cargo:warning=In file included from src/ffi/rsmpi.h:3:0,
cargo:warning=                 from src/ffi/rsmpi.c:1:
cargo:warning=/usr/include/mpi.h:323:9: error: unknown type name ‘ptrdiff_t’
cargo:warning= typedef OPAL_PTRDIFF_TYPE MPI_Aint;
cargo:warning=         ^
cargo:warning=In file included from src/ffi/rsmpi.h:3:0,
cargo:warning=                 from src/ffi/rsmpi.c:1:
cargo:warning=/usr/include/mpi.h:358:5: error: unknown type name ‘size_t’
cargo:warning=     size_t _ucount;
cargo:warning=     ^~~~~~

Any ideas on how this can be fixed?

[Rufflewind]

The MPI compiler on Arch Linux uses GCC by default (not Clang). Therefore setting C_INCLUDE_PATH=/usr/lib/clang/4.0.0/include is likely to cause problems since GCC does not understand the Clang-specific things like __has_feature, which are found in internal Clang headers such as /usr/lib/clang/4.0.0/include/stddef.h.

What happens if you don’t set C_INCLUDE_PATH at all?

[Libbum]

a stddef.h not found error:

--- stderr
warning: argument unused during compilation: '-L/usr/lib/openmpi' [-Wunused-command-line-argument]
/usr/include/mpi.h:225:10: fatal error: 'stddef.h' file not found
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ()', /checkout/src/libcore/result.rs:859

If I do the same trick as you mentioned before but use gcc instead, I get further it seems, and I end up with:

error: no field `MPI_SOURCE` on type `std::os::raw::c_void`
   --> /home/tim/.cargo/git/checkouts/rsmpi-d28083ffcf8044e3/33ffd96/src/point_to_point.rs:770:16
    |
770 |         self.0.MPI_SOURCE
    |                ^^^^^^^^^^

error: no field `MPI_TAG` on type `std::os::raw::c_void`
   --> /home/tim/.cargo/git/checkouts/rsmpi-d28083ffcf8044e3/33ffd96/src/point_to_point.rs:775:16
    |
775 |         self.0.MPI_TAG
    |                ^^^^^^^

error: no field `MPI_ERROR` on type `std::os::raw::c_void`
   --> /home/tim/.cargo/git/checkouts/rsmpi-d28083ffcf8044e3/33ffd96/src/point_to_point.rs:780:16
    |
780 |         self.0.MPI_ERROR
    |                ^^^^^^^^^

error: aborting due to 3 previous errors

[Rufflewind]

That’s odd. I tested this on a fresh Arch container with only rustup, openmpi, gcc, and clang installed through Pacman, and rsmpi works out of the box with no configuration required.

Did you install clang through Pacman, or obtain it through some other source? In the latter case, you could either:

• Use the gcc -M -E trick to find out where stddef.h is for GCC and set C_INCLUDE_PATH properly.
• Or, if you prefer to stick to Clang you can set OMPI_MPICC=clang, in which case /usr/lib/clang/4.0.0/include should work (and mpicc will therefore use Clang instead of GCC).

[Libbum]

OK, thanks - good to know. Neither of those suggestions give me any joy, but if the container works then it's certainly something on my own system that's the problem. I'll investigate further on my end. Thanks for the help.

[bsteinb]

This might be easier to debug if you try to apply bindgen (which rsmpi uses) to the header directly. You could try cargo install bindgen and then env RUST_LOG=bindgen bindgen src/ffi/rsmpi.h to see whether the output tells you anything useful.