Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong assembler selected for @local_config_cc//:cc-toolchain-x64_windows-clang-cl #23128

Closed
michaelsiegrist opened this issue Jul 26, 2024 · 11 comments
Labels

Comments

@michaelsiegrist
Copy link
Contributor

michaelsiegrist commented Jul 26, 2024

Description of the bug:

When compiling a cc_library with assembly (.S) files as input, the @local_config_cc//:cc-toolchain-x64_windows-clang-cl toolchain selects clang-cl.exe as the executable. However, clang-cl is not intended to handle assembly; as with cl.exe from the MSVC toolchain, it only accepts C/C++ code as input.

Comparing the Windows clang-cl and MSVC toolchains, it looks like the MSVC toolchain properly selects ml64.exe for assembly files, but the clang-cl toolchain uses clang-cl.

MSVC toolchain, tool_paths["ml"] = "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/ml64.exe":

>bazel query --output=build @local_config_cc//:msvc_x64
# C:/users/siegrist/_bazel_siegrist/njqmoezo/external/local_config_cc/BUILD:217:20
cc_toolchain_config(
  name = "msvc_x64",
  cpu = "x64_windows",
  compiler = "msvc-cl",
  toolchain_identifier = "msvc_x64",
  host_system_name = "local",
  target_system_name = "local",
  target_libc = "msvcrt",
  abi_version = "local",
  abi_libc_version = "local",
  tool_paths = {"ar": "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/lib.exe", "ml": "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/ml64.exe", "cpp": "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/cl.exe", "gcc": "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/cl.exe", "gcov": "wrapper/bin/msvc_nop.bat", "ld": "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/link.exe", "nm": "wrapper/bin/msvc_nop.bat", "objcopy": "wrapper/bin/msvc_nop.bat", "objdump": "wrapper/bin/msvc_nop.bat", "strip": "wrapper/bin/msvc_nop.bat"},
  cxx_builtin_include_directories = ["C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\include", "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\ATLMFC\\include", "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Auxiliary\\VS\\include", "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\ucrt", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\um", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\shared", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\winrt", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\cppwinrt", "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um"],
  archiver_flags = ["/MACHINE:X64"],
  default_link_flags = ["/MACHINE:X64"],
  msvc_env_tmp = "C:\\Users\\siegrist\\AppData\\Local\\Temp\\3",
  msvc_env_path = "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\bin\\HostX64\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\VC\\VCPackages;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\TeamFoundation\\Team Explorer;C:\\Tools\\ms\\vs\\2022\\17.4.11\\MSBuild\\Current\\bin\\Roslyn;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Team Tools\\Performance Tools\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Team Tools\\Performance Tools;C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v10.0A\\bin\\NETFX 4.8 Tools\\x64\\;C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.22000.0\\\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\bin\\\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\\\MSBuild\\Current\\Bin\\amd64;C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\Tools\\;;C:\\Windows\\system32;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\Ninja;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\VC\\Linux\\bin\\ConnectionManagerExe",
  msvc_env_include = "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\include;C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\ATLMFC\\include;C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Auxiliary\\VS\\include;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\ucrt;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\um;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\shared;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\winrt;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\cppwinrt;C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um",
  msvc_env_lib = "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\ATLMFC\\lib\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\lib\\x64;C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\lib\\um\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.22000.0\\ucrt\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\\\lib\\10.0.22000.0\\\\um\\x64",
  msvc_cl_path = "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/cl.exe",
  msvc_ml_path = "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/ml64.exe",
  msvc_link_path = "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/link.exe",
  msvc_lib_path = "C:/Tools/ms/vs/2022/17.4.11/VC/Tools/MSVC/14.34.31933/bin/HostX64/x64/lib.exe",
  dbg_mode_debug_flag = "/DEBUG:FULL",
  fastbuild_mode_debug_flag = "/DEBUG:FASTLINK",
  supports_parse_showincludes = True,
)
# Rule msvc_x64 instantiated at (most recent call last):
#   C:/users/siegrist/_bazel_siegrist/njqmoezo/external/local_config_cc/BUILD:217:20 in <toplevel>
# Rule cc_toolchain_config defined at (most recent call last):
#   C:/users/siegrist/_bazel_siegrist/njqmoezo/external/local_config_cc/windows_cc_toolchain_config.bzl:1414:27 in <toplevel>

Clang-cl toolchain, tool_paths["ml"] = "C:/Tools/llvm/16.0.2/bin/clang-cl.exe":

>bazel query --output=build @local_config_cc//:clang_cl_x64
# C:/users/siegrist/_bazel_siegrist/njqmoezo/external/local_config_cc/BUILD:501:20
cc_toolchain_config(
  name = "clang_cl_x64",
  cpu = "x64_windows",
  compiler = "clang-cl",
  toolchain_identifier = "clang_cl_x64",
  host_system_name = "local",
  target_system_name = "local",
  target_libc = "msvcrt",
  abi_version = "local",
  abi_libc_version = "local",
  tool_paths = {"ar": "C:/Tools/llvm/16.0.2/bin/llvm-lib.exe", "ml": "C:/Tools/llvm/16.0.2/bin/clang-cl.exe", "cpp": "C:/Tools/llvm/16.0.2/bin/clang-cl.exe", "gcc": "C:/Tools/llvm/16.0.2/bin/clang-cl.exe", "gcov": "wrapper/bin/msvc_nop.bat", "ld": "C:/Tools/llvm/16.0.2/bin/lld-link.exe", "nm": "wrapper/bin/msvc_nop.bat", "objcopy": "wrapper/bin/msvc_nop.bat", "objdump": "wrapper/bin/msvc_nop.bat", "strip": "wrapper/bin/msvc_nop.bat"},
  cxx_builtin_include_directories = ["C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\include", "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\ATLMFC\\include", "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Auxiliary\\VS\\include", "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\ucrt", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\um", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\shared", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\winrt", "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\cppwinrt", "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um", "C:\\Tools\\llvm\\16.0.2\\lib\\clang\\16\\include"],
  archiver_flags = ["/MACHINE:X64"],
  default_link_flags = ["/MACHINE:X64"],
  msvc_env_tmp = "C:\\Users\\siegrist\\AppData\\Local\\Temp\\3",
  msvc_env_path = "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\bin\\HostX64\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\VC\\VCPackages;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\TeamFoundation\\Team Explorer;C:\\Tools\\ms\\vs\\2022\\17.4.11\\MSBuild\\Current\\bin\\Roslyn;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Team Tools\\Performance Tools\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Team Tools\\Performance Tools;C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v10.0A\\bin\\NETFX 4.8 Tools\\x64\\;C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.22000.0\\\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\bin\\\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\\\MSBuild\\Current\\Bin\\amd64;C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\Tools\\;;C:\\Windows\\system32;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\Ninja;C:\\Tools\\ms\\vs\\2022\\17.4.11\\Common7\\IDE\\VC\\Linux\\bin\\ConnectionManagerExe",
  msvc_env_include = "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\include;C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\ATLMFC\\include;C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Auxiliary\\VS\\include;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\ucrt;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\um;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\shared;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\winrt;C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\cppwinrt;C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um;C:\\Tools\\llvm\\16.0.2\\lib\\clang\\16\\include",
  msvc_env_lib = "C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\ATLMFC\\lib\\x64;C:\\Tools\\ms\\vs\\2022\\17.4.11\\VC\\Tools\\MSVC\\14.34.31933\\lib\\x64;C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\lib\\um\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.22000.0\\ucrt\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\\\lib\\10.0.22000.0\\\\um\\x64;C:\\Tools\\llvm\\16.0.2\\lib\\clang\\16\\lib\\windows",
  msvc_cl_path = "C:/Tools/llvm/16.0.2/bin/clang-cl.exe",
  msvc_ml_path = "C:/Tools/llvm/16.0.2/bin/clang-cl.exe",
  msvc_link_path = "C:/Tools/llvm/16.0.2/bin/lld-link.exe",
  msvc_lib_path = "C:/Tools/llvm/16.0.2/bin/llvm-lib.exe",
  dbg_mode_debug_flag = "/DEBUG",
  fastbuild_mode_debug_flag = "/DEBUG",
  supports_parse_showincludes = True,
)
# Rule clang_cl_x64 instantiated at (most recent call last):
#   C:/users/siegrist/_bazel_siegrist/njqmoezo/external/local_config_cc/BUILD:501:20 in <toplevel>
# Rule cc_toolchain_config defined at (most recent call last):
#   C:/users/siegrist/_bazel_siegrist/njqmoezo/external/local_config_cc/windows_cc_toolchain_config.bzl:1414:27 in <toplevel>

It seems that the clang-cl toolchain should also be using ml64.exe or perhaps llvm-as to properly compile assembly files.

Which category does this issue belong to?

C++ Rules

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

bazel-test.zip

The example contains a single cc_library() target //:test that depends on @boost//:context, which in turn has some pre-generated assembly files listed in its srcs. The project is configured to use the Windows clang-cl toolchain described in the Bazel Windows documentation. After setting %BAZEL_VC% and %BAZEL_LLVM% to the appropriate paths (MSVC version 17.4.11 and LLVM 16.0.2), I compile and get the following:

>bazel build -j 1 //...
INFO: Analyzed target //:test (0 packages loaded, 0 targets configured).
ERROR: C:/users/siegrist/_bazel_siegrist/njqmoezo/external/rules_boost~~non_module_dependencies~boost/BUILD.bazel:144:14: Compiling external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S failed: (Exit 1): clang-cl.exe failed: error executing CppCompile command (from target @@rules_boost~~non_module_dependencies~boost//:context) C:\Tools\llvm\16.0.2\bin\clang-cl.exe @bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/_objs/context/make_x86_64_ms_pe_masm.obj.params
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:2:20: error: unexpected token in argument list
; Copyright Oliver Kowalke 2009.
                   ^
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:3:21: error: unexpected token in argument list
; Distributed under the Boost Software License, Version 1.0.
                    ^
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:4:3: error: unexpected token at start of statement
; (See accompanying file LICENSE_1_0.txt or copy at
  ^
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:7:3: error: unexpected token at start of statement
; ----------------------------------------------------------------------------------
  ^
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:8:3: error: unexpected token at start of statement
; | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
  ^

<.... snip ...>

bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:159:7: error: invalid instruction mnemonic 'exit'
    ; exit application
      ^~~~
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:162:1: error: invalid instruction mnemonic 'make_fcontext'
make_fcontext ENDP
^~~~~~~~~~~~~
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:163:1: error: invalid instruction mnemonic 'end'
END
^~~
Target //:test failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.117s, Critical Path: 1.64s
INFO: 3 processes: 2 internal, 1 local.
ERROR: Build did NOT complete successfully

Looking at the params file used for the compilation shows the following:

>type "bazel-out\x64_windows-fastbuild\bin\external\rules_boost~~non_module_dependencies~boost\_objs\context\make_x86_64_ms_pe_masm.obj.params"
/nologo
/DCOMPILER_MSVC
/DNOMINMAX
/D_WIN32_WINNT=0x0601
/D_CRT_SECURE_NO_DEPRECATE
/D_CRT_SECURE_NO_WARNINGS
/bigobj
/Zm500
/EHsc
/wd4351
/wd4291
/wd4250
/wd4996
/Iexternal/rules_boost~~non_module_dependencies~boost
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/context/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/assert/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/assert/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/current_function/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/current_function/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/config/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/config/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/cstdint/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/cstdint/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/limits/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/limits/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/detail/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/detail/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/winapi/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/winapi/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/smart_ptr/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/smart_ptr/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/align/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/align/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/core/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/core/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/predef/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/predef/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/scoped_array/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/scoped_array/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/checked_delete/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/checked_delete/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/scoped_ptr/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/scoped_ptr/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/shared_array/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/shared_array/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/shared_ptr/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/shared_ptr/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/throw_exception/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/throw_exception/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/utility/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/utility/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/container_hash/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/container_hash/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/describe/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/describe/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/mp11/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/mp11/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/integer/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/integer/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/static_assert/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/static_assert/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/type_traits/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/type_traits/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/mpl/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/mpl/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/move/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/move/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/preprocessor/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/preprocessor/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/io/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/io/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/swap/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/swap/include
/Iexternal/rules_boost~~non_module_dependencies~boost/libs/pool/include
/Ibazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/pool/include
/DBOOST_ALL_NO_LIB
/DBOOST_NO_CXX98_FUNCTION_BASE
/DBOOST_CONTEXT_EXPORT=
/showIncludes
/W0
/Fobazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/_objs/context/make_x86_64_ms_pe_masm.obj
/c
bazel-out/x64_windows-fastbuild/bin/external/rules_boost~~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S

Which operating system are you running Bazel on?

Windows Server 2016

What is the output of bazel info release?

release 7.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

According to this SO answer, cl.exe does not support compiling assembly; ml64.exe is needed for that.

Any other information, logs, or outputs that you want to share?

No response

@michaelsiegrist
Copy link
Contributor Author

michaelsiegrist commented Jul 26, 2024

It looks like this can indeed be fixed by changing the paths for both msvc_ml_path and tool_paths["ml"] to use the path to ml64.exe from the MSVC installation. Since MSVC needs to be available anyway for the clang-cl toolchain to work, that seems like a doable change, I believe.

Here is a attached a modified version of the test, but this time with a modified version of @local_config_cc//:clang_cl_x64 with the paths updated, and it appears to work just fine now (note that the same warnings appear when using the MSVC toolchain, so they are orthogonal to this issue).

bazel-test-working.zip

>bazel build //...
INFO: Analyzed 3 targets (79 packages loaded, 2890 targets configured).
INFO: From Compiling external/rules_boost~override~non_module_dependencies~boost/libs/context/src/asm/ontop_x86_64_ms_pe_masm.S:
MASM : warning A4018:invalid command-line option : /bigobj
MASM : warning A4018:invalid command-line option : /Zm500
MASM : warning A4018:invalid command-line option : /Z500
MASM : warning A4018:invalid command-line option : /Z00
MASM : warning A4018:invalid command-line option : /Z0
MASM : warning A4018:invalid command-line option : /EHsc
MASM : warning A4018:invalid command-line option : /wd4351
MASM : warning A4018:invalid command-line option : /wd4291
MASM : warning A4018:invalid command-line option : /wd4250
MASM : warning A4018:invalid command-line option : /wd4996
MASM : warning A4018:invalid command-line option : /showIncludes
 Assembling: bazel-out/x64_windows-fastbuild/bin/external/rules_boost~override~non_module_dependencies~boost/libs/context/src/asm/ontop_x86_64_ms_pe_masm.S
INFO: From Compiling external/rules_boost~override~non_module_dependencies~boost/libs/context/src/asm/jump_x86_64_ms_pe_masm.S:
MASM : warning A4018:invalid command-line option : /bigobj
MASM : warning A4018:invalid command-line option : /Zm500
MASM : warning A4018:invalid command-line option : /Z500
MASM : warning A4018:invalid command-line option : /Z00
MASM : warning A4018:invalid command-line option : /Z0
MASM : warning A4018:invalid command-line option : /EHsc
MASM : warning A4018:invalid command-line option : /wd4351
MASM : warning A4018:invalid command-line option : /wd4291
MASM : warning A4018:invalid command-line option : /wd4250
MASM : warning A4018:invalid command-line option : /wd4996
MASM : warning A4018:invalid command-line option : /showIncludes
 Assembling: bazel-out/x64_windows-fastbuild/bin/external/rules_boost~override~non_module_dependencies~boost/libs/context/src/asm/jump_x86_64_ms_pe_masm.S
INFO: From Compiling external/rules_boost~override~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S:
MASM : warning A4018:invalid command-line option : /bigobj
MASM : warning A4018:invalid command-line option : /Zm500
MASM : warning A4018:invalid command-line option : /Z500
MASM : warning A4018:invalid command-line option : /Z00
MASM : warning A4018:invalid command-line option : /Z0
MASM : warning A4018:invalid command-line option : /EHsc
MASM : warning A4018:invalid command-line option : /wd4351
MASM : warning A4018:invalid command-line option : /wd4291
MASM : warning A4018:invalid command-line option : /wd4250
MASM : warning A4018:invalid command-line option : /wd4996
MASM : warning A4018:invalid command-line option : /showIncludes
 Assembling: bazel-out/x64_windows-fastbuild/bin/external/rules_boost~override~non_module_dependencies~boost/libs/context/src/asm/make_x86_64_ms_pe_masm.S
INFO: Found 3 targets...
INFO: Elapsed time: 8.296s, Critical Path: 2.15s
INFO: 12 processes: 4 internal, 8 local.
INFO: Build completed successfully, 12 total actions

Note that my system has MSVC and LLVM installed into non-default locations, so the paths will likely need to be adjusted for other machines. However, the principle is the same.

@fmeum
Copy link
Collaborator

fmeum commented Jul 26, 2024

Would you be interested in sending a PR?

@michaelsiegrist
Copy link
Contributor Author

Sure, I'll try to have something up soon.

iancha1992 pushed a commit to iancha1992/bazel that referenced this issue Aug 22, 2024
Assembly files are not valid inputs for `clang-cl.exe`; the MSVC `ml64.exe` must be used instead.

Fixes bazelbuild#23128.

Closes bazelbuild#23337.

PiperOrigin-RevId: 666406544
Change-Id: Ia7a5fc4702f08a5754145ca286c079d1a4f0e204
iancha1992 pushed a commit to iancha1992/bazel that referenced this issue Aug 23, 2024
Assembly files are not valid inputs for `clang-cl.exe`; the MSVC `ml64.exe` must be used instead.

Fixes bazelbuild#23128.

Closes bazelbuild#23337.

PiperOrigin-RevId: 666406544
Change-Id: Ia7a5fc4702f08a5754145ca286c079d1a4f0e204
github-merge-queue bot pushed a commit that referenced this issue Aug 28, 2024
Assembly files are not valid inputs for `clang-cl.exe`; the MSVC
`ml64.exe` must be used instead.

Fixes #23128.

Closes #23337.

PiperOrigin-RevId: 666406544
Change-Id: Ia7a5fc4702f08a5754145ca286c079d1a4f0e204

Commit
e5a083d

---------

Co-authored-by: Michael Siegrist <[email protected]>
@iancha1992
Copy link
Member

A fix for this issue has been included in Bazel 7.4.0 RC1. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.4.0rc1. Thanks!

@fhanau
Copy link

fhanau commented Oct 23, 2024

Unfortunately, the fix PR for this broke building assembly code for me... With Bazel 7.4.0, compiling a .S file on Windows with the clang-cl toolchain set up results in MASM : fatal error A1013:invalid numerical command-line argument : /W errors when there are clang-style arguments defined in copts (e.g. --copt='-Wno-macro-redefined'), whereas building "just worked" beforehand. It would be non-trivial for us to drop these flags without getting lots of warnings for compiling C++ code. I believe that this is being caused by #23406, so it looks like here the opposite of your case happens and the file compiles well with clang-cl but MASM fails since it can't digest the input arguments.
I would file an issue for this, but I don't develop on Windows and only see this on CI so I can't easily provide a minimal reproducer. Maybe this points to another bug in the Windows toolchain configuration: Are copts supposed to be passed through to assemble? FWIW the error message says Compiling external/v8/icu/embedded.S failed: (Exit 1): ml64.exe failed: error executing CppCompile command so maybe assembling the .S file is still considered a cpp compile action somehow?

@meteorcloudy
Copy link
Member

@fhanau Can you please file an issue and provide a reproducible case?

@michaelsiegrist Have you encountered similar problem after #23337 ?

@michaelsiegrist
Copy link
Contributor Author

We see warnings being generated, but not errors. For instance, when compiling @llvm-project//llvm:Support, we see several actions generate warnings about unsupported options like this:

Assembling: external/_main~_repo_rules~llvm-project/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_windows_msvc.asm
INFO: From Compiling llvm/lib/Support/BLAKE3/blake3_avx512_x86-64_windows_msvc.asm:
MASM : warning A4018:invalid command-line option : /bigobj
MASM : warning A4018:invalid command-line option : /Zm500
MASM : warning A4018:invalid command-line option : /Z500
MASM : warning A4018:invalid command-line option : /Z00
MASM : warning A4018:invalid command-line option : /Z0
MASM : warning A4018:invalid command-line option : /EHsc
MASM : warning A4018:invalid command-line option : /wd4351
MASM : warning A4018:invalid command-line option : /wd4291
MASM : warning A4018:invalid command-line option : /wd4250
MASM : warning A4018:invalid command-line option : /wd4996

Maybe this points to another bug in the Windows toolchain configuration: Are copts supposed to be passed through to assemble?

This is what I assume is happening as well. It looks like the code has been recently migrated from here into rules_cc, but I think the relevant section is:
https://github.com/bazelbuild/rules_cc/blob/faeafdb82814b4f7295c555781e800f080607bdd/cc/private/toolchain/windows_cc_toolchain_config.bzl#L169-L191

It looks to me like that code is using the contents of compiler_input_flags and compiler_output_flags, which are defined in the same file here and here.

I would guess that the right thing would be to refactor those features/flag_sets to separate compilation options from assembly options, but I don't know nearly enough about Windows tooling to know what the right thing would be.

@meteorcloudy
Copy link
Member

@fhanau @michaelsiegrist Do you mind filing an new issue to track this problem? Unfortunately, the Bazel team doesn't have resource to polish our Windows support, but we welcome contributions from the community to address those issues.

@fhanau
Copy link

fhanau commented Oct 29, 2024

We see warnings being generated, but not errors. For instance, when compiling @llvm-project//llvm:Support, we see several actions generate warnings about unsupported options like this:

Assembling: external/_main~_repo_rules~llvm-project/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_windows_msvc.asm
INFO: From Compiling llvm/lib/Support/BLAKE3/blake3_avx512_x86-64_windows_msvc.asm:
MASM : warning A4018:invalid command-line option : /bigobj
MASM : warning A4018:invalid command-line option : /Zm500
MASM : warning A4018:invalid command-line option : /Z500
MASM : warning A4018:invalid command-line option : /Z00
MASM : warning A4018:invalid command-line option : /Z0
MASM : warning A4018:invalid command-line option : /EHsc
MASM : warning A4018:invalid command-line option : /wd4351
MASM : warning A4018:invalid command-line option : /wd4291
MASM : warning A4018:invalid command-line option : /wd4250
MASM : warning A4018:invalid command-line option : /wd4996

Maybe this points to another bug in the Windows toolchain configuration: Are copts supposed to be passed through to assemble?

This is what I assume is happening as well. It looks like the code has been recently migrated from here into rules_cc, but I think the relevant section is: https://github.com/bazelbuild/rules_cc/blob/faeafdb82814b4f7295c555781e800f080607bdd/cc/private/toolchain/windows_cc_toolchain_config.bzl#L169-L191

It looks to me like that code is using the contents of compiler_input_flags and compiler_output_flags, which are defined in the same file here and here.

I would guess that the right thing would be to refactor those features/flag_sets to separate compilation options from assembly options, but I don't know nearly enough about Windows tooling to know what the right thing would be.

I see the same warnings, followed by a fatal error for options with a leading dash like -Wno-macro-redefined. I'm afraid that the warnings/errors point to the opposite conclusion from the premise of #23337: That at least for our use case (compiling multi-platform C++ code and some assembly using clang-cl because it is more compatible with compiling with clang on Unix), assembly files are valid inputs for clang-cl.exe and it is the right choice for this. The warnings at least should be a clear sign that the updated toolchain configuration using the MSVC assembler for asm and clang-cl tools for everything else is inconsistent.

@meteorcloudy I'll try to file an issue today or tomorrow but we're in a similar situation – no one on my team is based on Windows so I'll describe the issue as well as I can while also having an incomplete understanding of it.

@fhanau
Copy link

fhanau commented Oct 31, 2024

@fhanau @michaelsiegrist Do you mind filing an new issue to track this problem? Unfortunately, the Bazel team doesn't have resource to polish our Windows support, but we welcome contributions from the community to address those issues.

#24152

@michaelsiegrist
Copy link
Contributor Author

@fhanau, would it be possible for you to test the target I use in my reproduction in your environment? The issue arose for us because we needed to compile @boost//:context, which relies on a .S file that failed to compile with the Windows clang-cl toolchain. My change resolved that issue for us, and I'd be interested to know if your environment would have the same problem (supposing you tested with a version of Bazel that does not contain #23337). If this is indeed environment-specific, I think we can back out my change, but I'd hate to fix one bug by reintroducing another.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants