OS X toolchain broke CLI without XCode

[dhalperi]

Description of the problem / feature request:

If I build bazel from HEAD (5736381), then that version of bazel cannot compile bazel on OS X.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

See #7397 for more info -- about once daily I pull and build bazel, then try to build bazel again with the built binary before I upgrade my local install.

Key difference in my setup from Bazel CI for OS X I only have the command-line tools installed, and not the XCode IDE:

[..] On our CI we use mac workers that have xcode installed (so we can also test objc stuff). It seems that you only have CLT installed [...]

#7397 (comment)

What operating system are you running Bazel on?

macOS 10.14.5

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

git pull && bazel build //src:bazel && cp -f $(bazel info bazel-bin)/src/bazel ~/bin/bazel-head && bazel-head build //src:bazel && cp -f ~/bin/bazel-head ~/bin/bazel

I have bazel-0.25.2 in /usr/local/bin/bazel, and latest bazel that worked from head in ~/bin/bazel (~last Friday HEAD). I have tried the above command with both.

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

[email protected]:bazelbuild/bazel.git
57363813d3c72129c61c35a2b03c3033b62cb49d
57363813d3c72129c61c35a2b03c3033b62cb49d

Have you found anything relevant by searching the web?

Just my series of issues finding problems in Bazel with my common setup that is not tested in your CI :)

• Bazel built at HEAD cannot compile Bazel on OS X #7397
• Bazel@HEAD cannot build Bazel on OS X #7575

If you make XCode an official requirement for developing on Mac, I'll happily install it. I figured for now I'm a useful canary?

[keith]

Can you post the specific log for this failure? (I'm assuming it's not exactly the same as last time?)

[dhalperi]

Thanks @keith -- sorry, was too busy making sure I filled all the fields in and starting git bisect to .. fill all the fields in.

ERROR: /private/var/tmp/_bazel_dhalperin/84f5772a14c30e3cb841f904746049ac/external/bazel_tools/src/main/cpp/util/BUILD:174:1: Target '@bazel_tools//src/main/cpp/util:blaze_exit_code' depends on toolchain '@local_config_cc//:cc-compiler-darwin_x86_64', which cannot be found: no such target '@local_config_cc//:cc-compiler-darwin_x86_64': target 'cc-compiler-darwin_x86_64' not declared in package '' defined by /private/var/tmp/_bazel_dhalperin/84f5772a14c30e3cb841f904746049ac/external/local_config_cc/BUILD'
ERROR: /private/var/tmp/_bazel_dhalperin/84f5772a14c30e3cb841f904746049ac/external/bazel_tools/src/main/cpp/util/BUILD:36:1: Target '@bazel_tools//src/main/cpp/util:filesystem' depends on toolchain '@local_config_cc//:cc-compiler-darwin_x86_64', which cannot be found: no such target '@local_config_cc//:cc-compiler-darwin_x86_64': target 'cc-compiler-darwin_x86_64' not declared in package '' defined by /private/var/tmp/_bazel_dhalperin/84f5772a14c30e3cb841f904746049ac/external/local_config_cc/BUILD'

[dhalperi]

Perhaps unsurprisingly, git bisect blames b4c0fb0 cc: @hlopko

[dhalperi]

More possibly-useful info:

(base) ➜  bazel git:(b4c0fb0031) cat /private/var/tmp/_bazel_dhalperin/84f5772a14c30e3cb841f904746049ac/external/local_config_cc/BUILD

# Copyright 2016 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This becomes the BUILD file for @local_config_cc// under non-FreeBSD unixes.

package(default_visibility = ["//visibility:public"])

load(":cc_toolchain_config.bzl", "cc_toolchain_config")
load(":armeabi_cc_toolchain_config.bzl", "armeabi_cc_toolchain_config")

licenses(["notice"])  # Apache 2.0

cc_library(
    name = "malloc",
)

filegroup(
    name = "empty",
    srcs = [],
)

filegroup(
    name = "cc_wrapper",
    srcs = ["cc_wrapper.sh"],
)

filegroup(
    name = "compiler_deps",
    srcs = glob(["extra_tools/**"]) + [":cc_wrapper"],
)

# This is the entry point for --crosstool_top.  Toolchains are found
# by lopping off the name of --crosstool_top and searching for
# the "${CPU}" entry in the toolchains attribute.
cc_toolchain_suite(
    name = "toolchain",
    toolchains = {
        "darwin|compiler": ":cc-compiler-darwin",
        "darwin": ":cc-compiler-darwin",
        "armeabi-v7a|compiler": ":cc-compiler-armeabi-v7a",
        "armeabi-v7a": ":cc-compiler-armeabi-v7a",
    },
)

cc_toolchain(
    name = "cc-compiler-darwin",
    toolchain_identifier = "local",
    toolchain_config = ":local",
    all_files = ":compiler_deps",
    ar_files = ":empty",
    as_files = ":empty",
    compiler_files = ":compiler_deps",
    dwp_files = ":empty",
    linker_files = ":compiler_deps",
    objcopy_files = ":empty",
    strip_files = ":empty",
    supports_param_files = 0,
)

cc_toolchain_config(
    name = "local",
    cpu = "darwin",
    compiler = "compiler",
    toolchain_identifier = "local",
    host_system_name = "local",
    target_system_name = "local",
    target_libc = "macosx",
    abi_version = "local",
    abi_libc_version = "local",
    cxx_builtin_include_directories = ["/usr/local/include",
    "/Library/Developer/CommandLineTools/usr/lib/clang/10.0.1/include",
    "/Library/Developer/CommandLineTools/usr/include",
    "/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include",
    "/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks",
    "/Library/Developer/CommandLineTools/usr/include/c++/v1"],
    tool_paths = {"ar": "/usr/bin/libtool",
        "ld": "/usr/bin/ld",
        "cpp": "/usr/bin/cpp",
        "gcc": "cc_wrapper.sh",
        "dwp": "/usr/bin/dwp",
        "gcov": "/usr/bin/gcov",
        "nm": "/usr/local/bin/nm",
        "objcopy": "/usr/local/bin/objcopy",
        "objdump": "/usr/local/bin/objdump",
        "strip": "/usr/local/bin/strip"},
    compile_flags = ["-U_FORTIFY_SOURCE",
    "-fstack-protector",
    "-Wall",
    "-Wthread-safety",
    "-Wself-assign",
    "-fcolor-diagnostics",
    "-fno-omit-frame-pointer"],
    opt_compile_flags = ["-g0",
    "-O2",
    "-D_FORTIFY_SOURCE=1",
    "-DNDEBUG",
    "-ffunction-sections",
    "-fdata-sections"],
    dbg_compile_flags = ["-g"],
    cxx_flags = ["-std=c++0x"],
    link_flags = ["-undefined",
    "dynamic_lookup",
    "-headerpad_max_install_names",
    "-lstdc++",
    "-lm"],
    opt_link_flags = [""],
    unfiltered_compile_flags = ["-no-canonical-prefixes",
    "-Wno-builtin-macro-redefined",
    "-D__DATE__=\"redacted\"",
    "-D__TIMESTAMP__=\"redacted\"",
    "-D__TIME__=\"redacted\""],
    coverage_compile_flags = ["-fprofile-instr-generate",  "-fcoverage-mapping"],
    coverage_link_flags = ["-fprofile-instr-generate"],
    supports_start_end_lib = False,
)

# Android tooling requires a default toolchain for the armeabi-v7a cpu.
cc_toolchain(
    name = "cc-compiler-armeabi-v7a",
    toolchain_identifier = "stub_armeabi-v7a",
    toolchain_config = ":stub_armeabi-v7a",
    all_files = ":empty",
    ar_files = ":empty",
    as_files = ":empty",
    compiler_files = ":empty",
    dwp_files = ":empty",
    linker_files = ":empty",
    objcopy_files = ":empty",
    strip_files = ":empty",
    supports_param_files = 1,
)

armeabi_cc_toolchain_config(name = "stub_armeabi-v7a")

[hlopko]

That's a great bug!!! Thanks for catching it. I'll fix it right tomorrow (it's eod in MUC already).

[hlopko]

I can repro on a mac with Xcode uninstalled. I couldn't repro with just CLT selected, or with BAZEL_USE_CPP_ONLY_TOOLCHAIN=1.

[hlopko]

I believe I have a fix in #8492, could you take a look to see if that fixes the issue for you too?

[dhalperi]

Works for me:

(base) ➜  bazel git:(master) git fetch origin pull/8492/head:pr-8492 && git checkout pr-8492
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
Unpacking objects: 100% (33/33), done.
remote: Total 33 (delta 23), reused 23 (delta 23), pack-reused 10
From github.com:bazelbuild/bazel
 * [new ref]               refs/pull/8492/head -> pr-8492
Switched to branch 'pr-8492'

(base) ➜  bazel git:(pr-8492) bazel build //src:bazel && cp -f $(bazel info bazel-bin)/src/bazel ~/bin/bazel-head && bazel-head build //src:bazel && cp -f ~/bin/bazel-head ~/bin/bazel

[hlopko]

Thanks for checking!