Feature: add settings for clang-cl (clang on Windows)

[SSE4]

Changelog: Feature: Add settings for clang-cl (clang on Windows).
Docs: conan-io/docs#1784

related:
#1839
#6789
https://github.com/conan-community/community/issues/242
https://github.com/conan-community/community/issues/236
bincrafters/community#680

as clang on Windows is getting more popular (Clang is now used to build Chrome for Windows, Firefox switching to clang-cl for Windows builds), this configuration needs to have a first-class support in conan. this demand is also shown by growing number of issues related to Clang on Windows in conan-community/bincrafters.

currently, there are several possibilities to use Clang on Windows:

• Clang/C2 toolset, which isn't something we consider, as The Clang/C2 experiment for C++ is ending
• install LLVM binaries from the llvm.org (section Pre-Built Binaries, Windows)
• install LLVM via pacman in MSYS2 (pacman -S mingw64/mingw-w64-x86_64-clang)

NOTE: users may currently use llmv* based toolsets for Visual Studio compiler, e.g. if they install corresponding extension or use preview version, but it has some disadvantages, as it works only for Visual Studio (MSBuild) projects, while the majority of packages use plain makefiles, autotools, CMake or other build systems like b2 which have no concept of MSVC toolsets.

to add more complexity, Clang on Windows may operates in two modes (--driver-mode):

• cl - in this mode Clang tries to be compatible with Microsoft Visual Studio (MSVC) and accepts MSVC-style command line arguments, such as /Zi, /MT, etc. used by default ibyclang-cl.exe.
• g++ - classic mode, where clang tries to accept GCC-style arguments, such as -m32, -g and so on. in other words, same behaviors, as observer on Linux and Mac. used by default by clang.exe

we clearly have to distinguish mentioned two modes, so:

• conan doesn't try to add unsupported compiler flags, and we avoid warnings like clang-cl: warning: argument unused during compilation: '-O3' [-Wunused-command-line-argument]
• recipe authors may distinguish easily between these two modes

besides these two modes, I also propose to add compiler.runtime for Clang, as it uses the same compiler runtime options (/MT, /MD, /MDd, /MTd) as MSVC

finally, I propose to add None to compiler.libcxx for the Clang on Windows, so we don't use -stdlib option, as it's not supported and redundant in the mentioned case (Clang uses the same standard library implementation as MSVC does, and doesn't use libc++ or libstdc++). right now conan adds -stdlib=libstdc++ which results in compiler warning: clang-cl: warning: unknown argument ignored in clang-cl: '-stdlib=libstdc++' [-Wunknown-argument]
possible alternative: use msstl to indicate Microsoft STL implementation?

this PR only propose settings changes, it doesn't include:

• compiler flag management changes
• compiler detection changes

these things will be added as soon as we agree on settings

annex 1:
typical conan profile for clang-cl currently looks like:

include(default)
[settings]
compiler=clang
compiler.version=8
compiler.libcxx=libstdc++
[options]
[env]
CC=clang-cl  -mrtm
CXX=clang-cl  -mrtm
CFLAGS= -fms-compatibility-version=1800
CXXFLAGS= -fms-compatibility-version=1800
CONAN_CMAKE_GENERATOR=Ninja
PATH=[C:\Program Files\LLVM\bin]
[build_requires]
ninja_installer/1.8.2@bincrafters/stable

or for clang.exe instead of clang-cl.exe:

include(default)
[settings]
compiler=clang
compiler.version=8
compiler.libcxx=libstdc++
[options]
[env]
CC=clang  -mrtm
CXX=clang  -mrtm
CFLAGS= -fms-compatibility-version=1800
CXXFLAGS= -fms-compatibility-version=1800
CONAN_CMAKE_GENERATOR=Ninja
PATH=[C:\Program Files\LLVM\bin]
[build_requires]
ninja_installer/1.8.2@bincrafters/stable

after changes made to the conan, profile will not change significantly, for clang-cl.exe:

include(default)
[settings]
compiler=clang
compiler.version=8
compiler.runtime=MD
compiler.driver=cl
[options]
[env]
CC=clang-cl  -mrtm
CXX=clang-cl  -mrtm
CFLAGS= -fms-compatibility-version=1800
CXXFLAGS= -fms-compatibility-version=1800
CONAN_CMAKE_GENERATOR=Ninja
PATH=[C:\Program Files\LLVM\bin]
[build_requires]
ninja_installer/1.8.2@bincrafters/stable

and for clang.exe:

include(default)
[settings]
compiler=clang
compiler.version=8
compiler.runtime=MD
compiler.driver=g++
[options]
[env]
CC=clang  -mrtm
CXX=clang  -mrtm
CFLAGS= -fms-compatibility-version=1800
CXXFLAGS= -fms-compatibility-version=1800
CONAN_CMAKE_GENERATOR=Ninja
PATH=[C:\Program Files\LLVM\bin]
[build_requires]
ninja_installer/1.8.2@bincrafters/stable

• Refer to the issue that supports this Pull Request.
• If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
• I've read the Contributing guide.
• I've followed the PEP8 style guides for Python code.
• I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

^{Note: By default this PR will skip the slower tests and will use a limited set of python versions. Check here how to increase the testing level by writing some tags in the current PR body text.}

[memsharded]

I would say that the main issue is the driver. It totally seems an argument that should be provided and managed accordingly by the build helpers, depending on the build system, for example. It doesn't really seem that the binary is different if being driven by the cl or the g++ interface. It seems possible to build some third party library package with makefiles, using the g++ interface and other package with cmake, that is using the cl interface. Not only possible, but also quite probable.

I am fine with adding runtime and libcxx with None.

What would be the way to manage it with VS toolset? Just adding the toolset? Will it be binary compatible with cmake clang compiled libraries? If yes, then we should probably take it into account, we can not design it in a way that will block native VS clang support.

[dmn-star]

@SSE4 Might be of interest to you VS2019 support clang out-of-the-box.
https://devblogs.microsoft.com/cppblog/clang-llvm-support-in-visual-studio/

The “Desktop development with C++” workload with the Visual Studio installer now includes full support for targeting Clang/LLVM based toolsets.

[memsharded]

Hi @SSE4

There were some comments, questions and concerns above. I feel this is not ready to be in 1.22, so lets move it for 1.23, and lets find some time to keep discussing this.

[SSE4]

the newly added platform toolset is named LLVM (clang-cl).
the one provided by extension is just llvm.
previously it was LLVM-<vs.version> (e.g. llvm-vs2017).
the problem, if we want to add compatibility quirks (like vs_toolset_compatible() one), we can't deduce Clang version to match just from the settings (toolset name). we need to somehow locate compiler executable (non-trivial in case of VS toolset) and asks it for its version.

side note: Intel compiler adds its own toolsets as well (e.g. in my case it's Intel C++ Compiler 19.0, which is good, as it includes compiler version, so we can at least match it). but we currently don't do any toolset <-> compiler version mapping for Intel, right?

[SSE4]

NOTE: driver-mode doesn't introduce a new setting on its own, e.g. produced binaries are identical for clang.exe and clang-cl.exe. as well as for clang.exe and clang.exe --driver-mode=cl.
therefore, it doesn't belong to the settings and shouldn't be a part of default settings.yml (thus, I've removed it from this PR).
this should be something that our build helpers should consider when apply various flags (like -m32 and so on), but that's another topic for another PR.

[SSE4]

the important thing for package_id which matter is so called target.
given the simple source code:

int main()
{
#ifdef __GNUC__
#error "__GNUC__ is defined"
#else
#error "__GNUC__ is not defined"
#endif
    
#ifdef _MSC_VER
#error "_MSC_VER is defined"
#else
#error "_MSC_VER is not defined"
#endif
}

and clang from MSYS:

$ clang --version
clang version 3.8.0 (/repo/sources/clang 1d5b05f1ef9d1b9889ddb5ad946944f224a7ba88) (/repo/sources/llvm 2aebced35905eb3891eba484e4d1902cf7399558)
Target: x86_64-pc-msys
Thread model: posix
InstalledDir: /usr/bin

we get:

$ clang test.cpp
test.cpp:4:2: error: "__GNUC__ is defined"
#error "__GNUC__ is defined"
 ^
test.cpp:12:2: error: "_MSC_VER is not defined"
#error "_MSC_VER is not defined"
 ^
2 errors generated.

and with regular clang:

>"C:\Program Files\LLVM\bin\clang" --version
clang version 8.0.1 (tags/RELEASE_801/final)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

we get:

>"C:\Program Files\LLVM\bin\clang-cl" test.cpp
test.cpp(6,2): error: "__GNUC__ is not defined"
#error "__GNUC__ is not defined"
 ^
test.cpp(10,2): error: "_MSC_VER is defined"
#error "_MSC_VER is defined"
 ^
2 errors generated.

>"C:\Program Files\LLVM\bin\clang"  test.cpp
test.cpp:6:2: error: "__GNUC__ is not defined"
#error "__GNUC__ is not defined"
 ^
test.cpp:10:2: error: "_MSC_VER is defined"
#error "_MSC_VER is defined"
 ^
2 errors generated.

>"C:\Program Files\LLVM\bin\clang" --driver-mode=cl  test.cpp
test.cpp(6,2): error: "__GNUC__ is not defined"
#error "__GNUC__ is not defined"
 ^
test.cpp(10,2): error: "_MSC_VER is defined"
#error "_MSC_VER is defined"
 ^
2 errors generated.

related code is here

pls share what do you think/prefer about modelling target in settings.yml

maybe we can use the approach similar to the Intel (clang with compiler.base=Visual Studio)

[memsharded]

@SSE4 @jgsogo

What I am seeing here is basically that these clang compilers might be basically incompatible between them. More like they are different compilers? Should we treat them as such?

[SSE4]

@jgsogo if you can provide your opinion on this would be nice :) I don't think it's different compiler(s), more like sub-setting(s) to be modeled. WDYT?

[jgsogo]

I really find myself a bit confused about this. We agree that ABI incompatibilities should be encoded into settings, but here we have different compilers (different executables, maybe requiring an option) that generate identical binaries and I think it is useful to question if they should compute the same package-id unless an explicit sub-setting is given to differentiate between them... is this what you mean, @SSE4 ? ...but I'm afraid there is no way to do it right now in Conan: if a setting takes a different value, it will be encoded in a different package-id.

Maybe the relation 1-1 between a compiler entry in the settings and an executable is no longer true.

I feel like this PR being opened for so long (and the investigation we did about msvc compiler) is a symptom, there is something we are not modeling well enough in the settings.

IMO we need to have a look at the full picture before entering in detail, and probably it is something needed before Conan 2.0.

[SSE4]

@jgsogo yes, we have different executables (clang-cl.exe and clang.exe**) producing the identical binaries. still, in conan we have to distinguish them as they accept flags in the different format.
we also have clang.exe executables which produce incompatible binaries from the two mentioned previously. and these executables have to be differentiated with some (sub-)settings.

still, maybe we can go incremental and as a first iteration merge only proposed settings changes, which are quite simple, straightforward and safe? then, once you have some brilliant and smart idea on how to model the weirdness mentioned above, we can go with second PR.

[jgsogo]

Just as an example, I was struggling with the profile generation in CCI, trying to encode all the combinations in a YML file, trying to figure out how to forbid certain combinations (like here, not all toolchains can be combined with all the compilers)... and finally I came out with a evident solution: you can have repeated entries!

- os: ["Linux"]
  os_build: ["Linux"]
  arch: ["x86_64"]
  arch_build: ["x86_64"]
  compiler:
    - "gcc":
        compiler.version: ["4.9"]
        compiler.libcxx: ["libstdc++"]
        build_type: ["Release", "Debug"]
    - "gcc":
        compiler.version: ["5", "6", "7", "8", "9"]
        compiler.libcxx: ["libstdc++11", "libstdc++"]
        build_type: ["Release", "Debug"]
    - "clang":
        compiler.version: ["3.9", "4.0", "5.0", "6.0", "7.0", "8", "9"]
        compiler.libcxx: ["libstdc++", "libc++"]
        build_type: ["Release", "Debug"]

It will require changing how we read and interpret the settings from the settings.yml file, but it is easy to read and easy to avoid some incompatibilities.

---

I agree we should add clang to Windows, first step is to add it to the settings, then we need to work on all the helpers and tools to manage those values (if we want to), but these two things are not coupled: how to declare valid settings combinations and how to deal with the profiles once they arrive to the build_helpers/tools.

[memsharded]

Having a look at this for the N-th time 😅

I like the idea of better representation of incompatibilities in the settings.yml, but I also think it can be a non-obvious implementation, and a bit of a challenge to introduce it without breaking, so probably needs to be something to consider for 2.0.

Regarding the definition of clang, lets move it forward. It will be the responsibility of the users to not select a wrong combination of settings in their profiles. I still wonder if separating a clang-win compiler, would be a better way, but no way to be sure so lets merge.

[memsharded]

Please @SSE4 follow up with the related issues:

#1839
#6789
conan-community/community#242
conan-community/community#236
bincrafters/community#680

Thanks!