Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--incompatible_enable_cc_toolchain_resolution still tries to access local_config_cc #12712

Open
AustinSchuh opened this issue Dec 16, 2020 · 13 comments
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: bug

Comments

@AustinSchuh
Copy link
Contributor

Splitting off of #7260,

I flipped --incompatible_enable_cc_toolchain_resolution today with 4.0.0-rc7 (essentially), provided my own toolchains, removed the --crosstool_top flag, and promptly got the following:

$ BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 bazel build -c opt --platforms=@aos//tools/platforms:linux_aarch64 //...
ERROR: /home/austin/.cache/bazel/_bazel_austin/71ec74b5bb538661db62db908afb04ce/external/local_config_cc/BUILD:29:19: in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'aarch64'
ERROR: Analysis of target '//build:docker_rootfs' failed; build aborted: Analysis of target '@local_config_cc//:toolchain' failed
INFO: Elapsed time: 0.836s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

We don't have an android or IOS build, so I would have thought I could flip it without waiting.

INFO: ToolchainResolution: Target platform @aos//tools/platforms:linux_x86: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/cpp:toolchain_type -> toolchain @llvm_toolchain//:cc-clang-linux-k8
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @llvm_toolchain//:cc-clang-linux-k8
INFO: ToolchainResolution:     Type @bazel_tools//tools/python:toolchain_type: execution platform @local_config_platform//:host: Rejected toolchain @io_bazel_rules_docker//toolchains:default_container_py_runtime_pair; mismatching values: run_in_container
INFO: ToolchainResolution: Target platform @aos//tools/platforms:linux_aarch64: Selected execution platform @io_bazel_rules_docker//platforms:local_container_platform, type @bazel_tools//tools/python:toolchain_type -> toolchain @io_bazel_rules_docker//toolchains:default_container_py_runtime_pair, type @bazel_tools//tools/cpp:toolchain_type -> toolcha    in @llvm_toolchain//:cc-clang-linux-aarch64
INFO: ToolchainResolution: Target platform @aos//tools/platforms:linux_aarch64: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/python:toolchain_type -> toolchain @bazel_tools//tools/python:_autodetecting_py_runtime_pair, type @bazel_tools//tools/cpp:toolchain_type -> toolchain @llvm_toolchain//:cc-clang-linux-aarc    h64
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform @aos//tools/platforms:linux_aarch64: execution @io_bazel_rules_docker//platforms:local_container_platform: Selected toolchain @llvm_toolchain//:cc-clang-linux-aarch64
ERROR: /home/austin/.cache/bazel/_bazel_austin/71ec74b5bb538661db62db908afb04ce/external/local_config_cc/BUILD:29:19: in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'aarch64'

suggests that toolchain resolution succeeded, and the correct toolchains were selected. But, for some reason, the default local_config_cc is still being looked at, maybe because it is the default for --crosstool_top or something like that. This ends up being a problem because I'm cross-compiling for aarch64, and local_config_cc isn't going to be able to find a compiler for that.

@AustinSchuhBRT
Copy link

FYI, @katre @gregestren, split from the other ticket.

@eagleonhill
Copy link

We encounters the same problem: whenever we want to cross-compile anything, with a platform mapping that doesn't map to host cpu(k8), I get the same error.

Looks like the reference is from CppHelper.getToolchainUsingDefaultCcToolchainAttribute calls, which always read crosstool_top.

@katre katre added team-Rules-CPP Issues for C++ rules untriaged labels Dec 17, 2020
@katre
Copy link
Member

katre commented Dec 17, 2020

As a workaround, can you create a cc_toolchain_suite and set --crosstool_top to keep the legacy CC machinery happy?

@oquenchil, can we make this check ignorable when toolchain resolution is enabled, or is it going to be necessary until we can flip the flag and remove the legacy code?

@AustinSchuhBRT
Copy link

That's what I ended up doing, but it didn't feel right. Hence the bug.

@katre
Copy link
Member

katre commented Dec 17, 2020

Definitely a bug, I'm just glad to know there's a workaround if anyone hits this in the future.

@carpenterjc
Copy link

carpenterjc commented Mar 29, 2022

I have found you can delay setting the crosstool_top to the platform_mappings file which avoids having it globally.

This is with Bazel 5.1

For example:

platforms:
  //:my_platform
    --cpu=x64_windows
    --crosstool_top=//:my_toolchains 

@github-actions
Copy link

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

@github-actions github-actions bot added the stale Issues or PRs that are stale (no activity for 30 days) label Jun 13, 2023
@limdor
Copy link
Contributor

limdor commented Jun 13, 2023

@bazelbuild/triage please keep, this is still something that does not make sense

@github-actions github-actions bot removed the stale Issues or PRs that are stale (no activity for 30 days) label Jun 14, 2023
@comius
Copy link
Contributor

comius commented Sep 13, 2023

This was possibly resolve with 0013115
After the commit crosstool_top is ignored with C++ toolchain resolution enabled (before it was still a dependency of C++ rules).

@AustinSchuh could you verify and close?

@AustinSchuh
Copy link
Contributor Author

@comius , I can't tell for certain, but I think I'm getting worse errors now on bazel 7. They may be different, I can't tell easily. With both --incompatible_enable_cc_toolchain_resolution and BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, I now unconditionally get local_config_cc as the result of toolchain resolution.

INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: Rejected toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-armeabi-v7a; mismatching values: armv7, android
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: execution //tools/platforms:linux_x86: Selected toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-k8
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: Rejected toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-armeabi-v7a; mismatching values: armv7, android
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: execution platform //tools/platforms:linux_x86: Skipping toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-k8; execution platform already has selected toolchain
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: execution platform //tools/platforms:linux_x86: Skipping toolchain @llvm_toolchain//:cc-clang-x86_64-linux; execution platform already has selected toolchain
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: Rejected toolchain @llvm_toolchain//:cc-clang-aarch64-linux; mismatching values: aarch64
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: Rejected toolchain @local_config_cc//:cc-compiler-armeabi-v7a; mismatching values: armv7, android
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform //tools/platforms:linux_x86: execution platform //tools/platforms:linux_x86: Skipping toolchain @local_config_cc//:cc-compiler-k8; execution platform already has selected toolchain
INFO: ToolchainResolution: Target platform //tools/platforms:linux_x86: Selected execution platform //tools/platforms:linux_x86, type @bazel_tools//tools/python:toolchain_type -> toolchain @bazel_tools//tools/python:_autodetecting_py_runtime_pair, type @bazel_tools//tools/cpp:toolchain_type -> toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-k8

You can see here that Bazel picks @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-k8 which I don't want it to do. I don't want that toolchain to exist at all, and want it to pick @llvm_toolchain//:cc-clang-x86_64-linux instead.

(I can probably persuade it to work with `--extra_toolchains or one of the other flags, but that seems like it leaves me at risk of accidentally getting the wrong toolchain somewhere else.)

Any ideas?

@fmeum
Copy link
Collaborator

fmeum commented Nov 28, 2023

@AustinSchuh Do you have a reproducer you could share? On an empty project with just a single cc_library target, I get:

$ USE_BAZEL_VERSION=7.0.0rc4 bazel build //:lib --repo_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 --incompatible_enable_cc_toolchain_resolution --toolchain_resolution_debug=.*
INFO: Invocation ID: 5f1cf9a1-29a6-496e-bcf0-0e5a764647b9
WARNING: Build option --toolchain_resolution_debug has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: ToolchainResolution: Target platform @@local_config_platform//:host: Selected execution platform @@local_config_platform//:host, 
INFO: ToolchainResolution: Performing resolution of @@bazel_tools//tools/cpp:toolchain_type for target platform @@local_config_platform//:host
      ToolchainResolution: No @@bazel_tools//tools/cpp:toolchain_type toolchain found for target platform @@local_config_platform//:host.
INFO: ToolchainResolution: Target platform @@local_config_platform//:host: Selected execution platform @@local_config_platform//:host, 
ERROR: /home/fhenneke/.cache/bazel/_bazel_fhenneke/e9921a0e334967d7d44291dbeea196de/external/bazel_tools/tools/cpp/BUILD:58:19: in cc_toolchain_alias rule @@bazel_tools//tools/cpp:current_cc_toolchain: 
Traceback (most recent call last):
	File "/virtual_builtins_bzl/common/cc/cc_toolchain_alias.bzl", line 26, column 48, in _impl
	File "/virtual_builtins_bzl/common/cc/cc_helper.bzl", line 219, column 17, in _find_cpp_toolchain
Error in fail: Unable to find a CC toolchain using toolchain resolution. Target: @@bazel_tools//tools/cpp:current_cc_toolchain, Platform: @@local_config_platform//:host, Exec platform: @@local_config_platform//:host

which looks right to me: no toolchain from @bazel_tools is registered.

@carpenterjc
Copy link

@AustinSchuh @comius Speculation here, but could this be because some rules have switched to building some tools rather than using pre-build binaries?
For example they had to switch a python launcher to a pre-built one for windows with this defect? #19977

@AustinSchuh
Copy link
Contributor Author

I feel a bit embarrassed, but I'm having trouble reproducing the failure too. I'm getting this feeling like I somehow didn't add BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN like I thought I did... Thanks for digging in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: bug
Projects
None yet
Development

No branches or pull requests

9 participants