Build fails on osx with /usr/bin/cc is not able to compile a simple test program.

[hu55a1n1]

My build script fails only on Mac (macOS Mojave 10.14.6). It works fine on linux.
Also tried setting CMAKE_C_COMPILER_WORKS but then I get errors like fatal error: 'stdio.h' file not found.

Here's the relevant part of my build.rs -

    let dst = Config::new("mylib-core")
        .profile("MinSizeRel")
        .define("USE_CURL", "OFF")
        .build_target("mylib")
        .build();

    println!("cargo:rustc-link-search=native={}/build/lib", dst.display());
    println!("cargo:rustc-link-lib=static=mylib");

Here's the output -

error: failed to run custom build command for `mylib-sys v0.1.0 (/Users/dev/mylib-core/rust/mylib-sys)`

Caused by:
  process didn't exit successfully: `/Users/dev/mylib-core/rust/target/debug/build/mylib-sys-f177755e646ed510/build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-env-changed=MACOSX_DEPLOYMENT_TARGET
running: "cmake" "/Users/dev/mylib-core/rust/mylib-sys/mylib-core" "-DCMAKE_OSX_ARCHITECTURES=x86_64" "-DCMAKE_OSX_SYSROOT=macosx" "-DCMAKE_OSX_DEPLOYMENT_TARGET=" "-DUSE_CURL=OFF" "-DDEV_NO_INTRN_PTR=OFF" "-DCMAKE_INSTALL_PREFIX=/Users/dev/mylib-core/rust/target/debug/build/mylib-sys-83508180fe1bf192/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=MinSizeRel"
-- The C compiler identification is AppleClang 10.0.1.10010046
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- broken
-- Configuring incomplete, errors occurred!
See also "/Users/dev/mylib-core/rust/target/debug/build/mylib-sys-83508180fe1bf192/out/build/CMakeFiles/CMakeOutput.log".
See also "/Users/dev/mylib-core/rust/target/debug/build/mylib-sys-83508180fe1bf192/out/build/CMakeFiles/CMakeError.log".

--- stderr
CMake Error at /usr/local/Cellar/cmake/3.16.1/share/cmake/Modules/CMakeTestCCompiler.cmake:60 (message):
  The C compiler

    "/usr/bin/cc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /Users/dev/mylib-core/rust/target/debug/build/mylib-sys-83508180fe1bf192/out/build/CMakeFiles/CMakeTmp
    
    Run Build Command(s):/usr/bin/make cmTC_b026a/fast && /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/cmTC_b026a.dir/build.make CMakeFiles/cmTC_b026a.dir/build
    Building C object CMakeFiles/cmTC_b026a.dir/testCCompiler.c.o
    /usr/bin/cc   -ffunction-sections -fdata-sections -fPIC -m64  -arch x86_64 -isysroot macosx   -o CMakeFiles/cmTC_b026a.dir/testCCompiler.c.o   -c /Users/dev/mylib-core/rust/target/debug/build/mylib-sys-83508180fe1bf192/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
    clang: warning: no such sysroot directory: 'macosx' [-Wmissing-sysroot]
    Linking C executable cmTC_b026a
    /usr/local/Cellar/cmake/3.16.1/bin/cmake -E cmake_link_script CMakeFiles/cmTC_b026a.dir/link.txt --verbose=1
    /usr/bin/cc  -ffunction-sections -fdata-sections -fPIC -m64  -arch x86_64 -isysroot macosx -Wl,-search_paths_first -Wl,-headerpad_max_install_names   CMakeFiles/cmTC_b026a.dir/testCCompiler.c.o  -o cmTC_b026a 
    clang: warning: no such sysroot directory: 'macosx' [-Wmissing-sysroot]
    ld: library not found for -lSystem
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make[1]: *** [cmTC_b026a] Error 1
    make: *** [cmTC_b026a/fast] Error 2
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:39 (project)


thread 'main' panicked at '
command did not execute successfully, got: exit code: 1

build script failed, must exit now', /Users/dev/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.43/src/lib.rs:1104:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

warning: build failed, waiting for other jobs to finish...
error: build failed

[alexcrichton]

The error message there indicates ld: library not found for -lSystem which means that something is wrong with the linker, which means that there may be something wrong with cross-compilation or something like that. I don't believe it's an issue with this crate, though, so closing.

[JonathanHenson]

I don't think this is a problem with the linker. I build C and C++ on my machine all day via cmake. something is wrong with the emited cmake. Here's my output:

running: "cmake" "/Users/henso/source/aws-crt-rust/aws-common-runtime/aws-c-common" "-DCMAKE_OSX_ARCHITECTURES=x86_64" "-DCMAKE_OSX_SYSROOT=macosx" "-DCMAKE_OSX_DEPLOYMENT_TARGET=" "-DCMAKE_INSTALL_PREFIX=/Users/henso/source/aws-crt-rust/target/debug/build/aws-crt-rust-1e31957a34af6229/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Debug"
-- The C compiler identification is AppleClang 10.0.1.10010046
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- broken
-- Configuring incomplete, errors occurred!
See also "/Users/henso/source/aws-crt-rust/target/debug/build/aws-crt-rust-1e31957a34af6229/out/build/CMakeFiles/CMakeOutput.log".
See also "/Users/henso/source/aws-crt-rust/target/debug/build/aws-crt-rust-1e31957a34af6229/out/build/CMakeFiles/CMakeError.log".

--- stderr
CMake Error at /usr/local/Cellar/cmake/3.16.4/share/cmake/Modules/CMakeTestCCompiler.cmake:60 (message):
The C compiler

"/usr/bin/cc"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: /Users/henso/source/aws-crt-rust/target/debug/build/aws-crt-rust-1e31957a34af6229/out/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_25798/fast && /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/cmTC_25798.dir/build.make CMakeFiles/cmTC_25798.dir/build
Building C object CMakeFiles/cmTC_25798.dir/testCCompiler.c.o
/usr/bin/cc   -ffunction-sections -fdata-sections -fPIC -m64  -arch x86_64 -isysroot macosx   -o CMakeFiles/cmTC_25798.dir/testCCompiler.c.o   -c /Users/henso/source/aws-crt-rust/target/debug/build/aws-crt-rust-1e31957a34af6229/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
clang: warning: no such sysroot directory: 'macosx' [-Wmissing-sysroot]
Linking C executable cmTC_25798
/usr/local/Cellar/cmake/3.16.4/bin/cmake -E cmake_link_script CMakeFiles/cmTC_25798.dir/link.txt --verbose=1
/usr/bin/cc  -ffunction-sections -fdata-sections -fPIC -m64  -arch x86_64 -isysroot macosx -Wl,-search_paths_first -Wl,-headerpad_max_install_names   CMakeFiles/cmTC_25798.dir/testCCompiler.c.o  -o cmTC_25798 
clang: warning: no such sysroot directory: 'macosx' [-Wmissing-sysroot]
ld: library not found for -lSystem
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [cmTC_25798] Error 1
make: *** [cmTC_25798/fast] Error 2

This exact cmake project builds perfectly fine for c and c++.

[JonathanHenson]

fwiw, adding the following line:

aws_common_config.define("CMAKE_OSX_SYSROOT", "PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk");

fixed my build. I pulled that directly from the CMakeCache.txt on the same exact project that was build using vanilla cmake:

cmake <source>

I think the issue is the default value you're adding is incorrect.

[hu55a1n1]

Thanks for the workaround @JonathanHenson... It's working for me as well.

set(CMAKE_OSX_SYSROOT "PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk")

Btw, I wasn't cross compiling either, but I do have two versions of cmake on my system - one installed using brew and the other compiled from sources. Not sure if this has something to do with the issue.

[alexcrichton]

Oh oops I missed that, sorry about that! This looks like it's a regression from #93 unfortunately. I've reverted that so this should be fixed now, sorry for the breakage!

I'll open a follow-up for re-landing.

[Dushistov]

@hu55a1n1 @JonathanHenson

Any opensource repo with which it is possible to reproduce problem?

[JonathanHenson]

Should work for any cmake project. Mine won’t be ready for a little while.

[Dushistov]

@JonathanHenson

Should work for any cmake project

Oh, I should mention this before. I can not reproduce this issue on macOS 10.15.4 with recent XCode.
So I suspect the previous version of XCode/macos 10.14 is part of problem.
But before going via hard way to get machine with 10.14 I want to be sure.

Can you try then this cmake hello world: https://github.com/Dushistov/cmake_rs_hello_world
I explicitly setted revsion for cmake deps to bad via cmake = "=0.1.43".

I can build it for macos and cross-compile for iOS without any issues.

[JonathanHenson]

I think the issue has already been fixed as of a week ago.

[Dushistov]

@JonathanHenson

I think the issue has already been fixed as of a week ago.

From my perspective is was broken a week ago :)
The version 0.1.43 fixes build for iOS and the same time break build for some unknown for me configuration, then "iOS fix" was reverted and version 0.1.44 was released.

So I need find out what is wrong with 0.1.43 to get my iOS build back.

[JonathanHenson]

If you grab GitHub.com/awslabs/aws-c-common and try to build it on 10.14, it should break. I suspect would break regardless of the cmake project though.

[Dushistov]

@JonathanHenson

I updated my test project
https://github.com/Dushistov/cmake_rs_hello_world to include aws-c-common as submodule and it builds without any errors on fresh install of macOS 10.14:

 sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.14.6
BuildVersion:	18G103

Any other hints to reproduce this build problem, please?

what cmake version do you use,
and what with output ls /Library/Developer/CommandLineTools/SDKs,
my show this: "MacOSX.sdk MacOSX10.14.sdk"?

[MichaelHills]

Hi, I'd like to get the iOS support re-instated. I'm currently trying to add iOS support for the Bevy Engine here and had some trouble building shaderc-rs until I stumbled across this revert. By pinning to version 0.1.43 I was able to get the cmake-rs build in shaderc-sys to work.

I have done some research on specifying sysroot as macosx and have found that various software will invoke xcodebuild or more recently xcrun to resolve the short-hand names like macosx into the full SDK path.

CMake does it here Kitware/CMake@777d364#diff-5464a2a0787149bc084bebd2dbfe2f31 (example of them switching to xcrun in CMake).

Rust itself does it here https://github.com/rust-lang/rust/blob/9f3c96b869b48ecd0bb556c6ad9cd603b4dacfb9/src/librustc_target/spec/apple_sdk_base.rs

LLVM doesn't do it https://github.com/llvm/llvm-project/blame/f6510a01a663a2e249462ef0ca98a5b94c22f59b/clang/lib/Driver/ToolChains/Darwin.cpp#L1178 but the comment in the Rust code says For clang, xcrun sets SDKROOT; for rustc, the user or build system can set it, or we can fall back to checking for xcrun on PATH.

The logs in the OP show that macosx was not substituted with the full SDK path, /usr/bin/cc ... -isysroot macosx when clang was invoked. Now, based on my understanding, CMake should have been the one to substitute macosx for the full SDK path, but without a repro of the issue this is about as far as I'll get. Possibly it was a buggy cmake version? Maybe it's to do with full XCode install vs just the CLI tools?

@hu55a1n1 @JonathanHenson do either of you still have a repro of the problem? The only guess fix I can come up with is to just omit SYSROOT entirely for mac builds since it seems to work fine without it.

diff --git a/src/lib.rs b/src/lib.rs
index cd776e4..876c8f3 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -958,7 +958,7 @@ impl AppleTarget {
                 }
             }
         } else if self.is_osx_target() {
-            Some("macosx".to_owned())
+            None
         } else {
             eprintln!(
                 "Warning: could not determine sdk name for Apple target: {}",

This way, normal mac builds should hopefully be unaffected, and iphone builds can work for others like @Dushistov and myself. Thoughts?

[MichaelHills]

I also ran into some issues with CMake ios defaults (see https://gitlab.kitware.com/cmake/cmake/-/issues/20956) that if we do re-instate the reverted code I'd like to suggest adding in

            cmd.arg("-DCMAKE_MACOSX_BUNDLE=FALSE");

so that it doesn't by default try to build a "GUI executable that can be launched from the Finder" for library builds. Given that cmake-rs is generally used for building Rust libraries, and as of right now doesn't support iOS builds, I think it would be a better step forward to force bundles off and work-around the CMake default for the time being. I posted a comment on that CMake issue so one can look there for more context.

[Dushistov]

I have done some research on specifying sysroot

cc-rs is dependency of cmake-rs, and it is also uses xcrun:

https://github.com/alexcrichton/cc-rs/blob/facf617200dbec2ffc6f11528b5b4b6022ef82d2/src/lib.rs#L2496

and also, may be it is completly underlated, but I found out during fixing rust-lang/cc-rs#530,
that different cc work in different way in spite of they are part of the same xcode.
/usr/bin/cc works just fine without SDKROOT,
while /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc is failed to find
includes if SDKROOT is not stetted. And xcodebuild uses the last one.