LLVM: add LLVM support on Windows #36753

chen-png · 2021-07-06T08:38:13Z

The compiler clang of llvm won't use built-in linker
when doing zephyr_compiler_check_flag, it will be failed
with "program not executable" due to it couldn't find
a valid linker, so use "-fuse-ld=lld" option to force
clang use built-in lld.
triple i686-pc-none-elf couldn't be recognized and
used to find a valid linker on windows, we need to use
i686-pc-linux-elf.

Fixes #32111

Signed-off-by: Chen, Peng1 [email protected]

stephanosio

The commit title should probably start with something like cmake: llvm and not just llvm:.

Also the commit message should be more specific about why this change is required to support Clang/LLVM on Windows.

LeiW000 · 2021-07-06T13:37:29Z

cmake/toolchain/llvm/generic.cmake

-    set(triple x86_64-pc-none-elf)
+    # The LLVM built-in linker lld is a generic driver, needs to use triple os filed
+    # to distinguish which linker will be used.
+    set(triple x86_64-pc-linux-elf)


Does it force lld to generate an elf binary on Windows?

yes, this is a target name for cross compiler, but on windows, the older name "i686-pc-none-elf" couldn't be recognized and used to find a valid linker, we need to use "i686-pc-linux-elf".

where is this coming from? Why does llvm on windows require a triple with linux in it? Is this in LLVM?

@nashif as I understood, we need to build a zephyr elf file which could be executed on linux, it's a cross compilation from windows to linux, and llvm on windows has four linker based on os type: ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly), so I added linux in triple name, then it will use this ld.lld.

i686-pc-none-elf refers to the x86 (i686) ELF bare metal target, while x86_64-pc-linux-elf refers to the x86 (i686) ELF Linux target.

I do not know how LLVM internally handles this; but, at least in terms of GNU toolchains, they are not the same thing and we must use i686-pc-none-elf to ensure correct behaviour.

If the Windows LLVM distribution does not support i686-pc-none-elf (i.e. bare metal target) while the Linux one does, we should be filing an LLVM-side bug instead to make it support that -- unless, of course, there is a good reason why it should not support bare metal targets on Windows hosts (I really cannot see any).

I installed LLVM 10.0.0 on Windows (same version as the one I have on my Linux machine) and tested this.

I was able to reproduce clang: error: unable to execute command: program not executable on Windows host with non-Linux targets, but that is only because Clang internally calls the GCC front-end (supposedly to invoke binutils in order to link the targets that it thinks are not supported by LLD) and I did not have GCC installed/available in the PATH. Note that the same behaviour is observed on Linux hosts as well, except that GCC is installed there, so no such error message is displayed.

In theory, if we have gcc/binutils that supports the x86_64-none-elf target installed on the Windows host, it will work just fine (note that MSYS2/MinGW-w64 GCC does not support x86_64-none-elf target, so using that is not an option; this will not be a problem if we distribute Windows version of Zephyr SDK though, since it supports *-none-elf targets).

Now the question is:

Does LLD not support bare metal targets (e.g. *-*-elf)? (i.e. should Clang be calling GCC front-end/binutils for these targets?)

If LLD supports bare metal targets, is there a way to force Clang to directly invoke ld.lld for these targets as it does for the Linux targets?

The main problem here is that, on both Linux and Windows, Clang is calling GCC front-end with -fuse-ld=lld, and the GCC front-end calls the binutils linker (collect2), supposedly ignoring -fuse-ld=lld -- at this point, we might as well be just using -fuse-ld=bfd.

cc @tejlmand
It looks like there is something funny going on with the invocation of LLD, or rather lack thereof.

@stephanosio yes, actually I used a cross compiler i486-elf to complete the link process on windows, rather than the llvm built-in linker.

Both i686-pc-none-elf and i686-none-elf as well as x86_64-pc-none-elf and x86_64-none-elf work for me:

it seems that it's related to gcc version, I found a comment "gcc 8.x doesn't support -fuse-ld=lld option at all, it has been only added in http://gcc.gnu.org/r265940 for gcc 9"
then I installed a gcc 9.4 on my fedora machine, it worked with x86_64-none-elf.

"/usr/lib64/ccache/gcc" -nostdlib -fuse-ld=lld -m32 -o dummy /tmp/dummy-675e45.o
ld.lld: warning: cannot find entry symbol _start; defaulting to 0x401120

I used gcc 8.3 before.

chen-png · 2021-07-06T15:04:58Z

@stephanosio thanks, I have updated the commit message.

yanzhouy · 2021-07-07T08:18:28Z

This patch helps with LLVM built on Windows, after adding the patch, error 'Assertion failed: The toolchain is unable to build a dummy C file.' will disappear.

LeiW000 · 2021-07-07T08:39:52Z

Is it better to change the commit log like below?

"The linker can't be found when zephyr_compiler_check_flag() is called to check the flags of the compiler. So the "-fuse-ld=lld" option is added to force to use lld, which is the default linker of clang."

nashif · 2021-07-08T13:08:50Z

cmake/compiler/clang/target.cmake

@@ -69,7 +69,9 @@ if(NOT "${ARCH}" STREQUAL "posix")
    list(APPEND TOOLCHAIN_LIBS gcc)
  endif()

-  set(CMAKE_REQUIRED_FLAGS -nostartfiles -nostdlib ${isystem_include_flags})
+  # CMake needs a valid linker to check compiler flag, but clang won't use LLVM built-in linker in default,


can you please make this line shorter.

but clang won't use LLVM built-in linker in default

... by default

Is it better to change the commit log like below?

thanks, have updated it.

The linker can't be found when zephyr_compiler_check_flag() is called to check the flags of clang compiler. So the "-fuse-ld=lld" option is added to force to use lld, which is the built-in linker of llvm. triple i686-pc-none-elf couldn't be recognized and used to find a valid linker on windows, we need to use i686-pc-linux-elf. Signed-off-by: Chen Peng1 <[email protected]>

stephanosio · 2021-07-12T09:43:04Z

cmake/toolchain/llvm/generic.cmake

-    set(triple x86_64-pc-none-elf)
+    # The LLVM built-in linker lld is a generic driver, needs to use triple os filed
+    # to distinguish which linker will be used.
+    set(triple x86_64-pc-linux-elf)


AFAICS, this change is not necessary.

See https://github.com/zephyrproject-rtos/zephyr/pull/36753/files#r666941908

yes, I agree this.

stephanosio · 2021-07-12T09:43:13Z

cmake/toolchain/llvm/generic.cmake

  else()
-    set(triple i686-pc-none-elf)
+    set(triple i686-pc-linux-elf)


stephanosio · 2021-07-12T09:53:43Z

cmake/compiler/clang/target.cmake

+  # CMake needs a valid linker to check compiler flag, but clang
+  # won't use LLVM built-in linker by default, so use this option
+  # to force clang to use LLVM built-in linker
+  set(CMAKE_REQUIRED_FLAGS -nostartfiles -nostdlib -fuse-ld=lld ${isystem_include_flags})


So the "-fuse-ld=lld" option is added to force to use lld,
which is the built-in linker of llvm.

It looks like Clang defaults to -fuse-ld=bfd (i.e. binutils linker) for x86_64-pc-none-elf and i686-pc-none-elf targets; so, if any, we should be specifying -fuse-ld=bfd here.

Note that -fuse-ld=bfd depends on the GNU binutils being available on the host and, to my knowledge, there is currently no readily available Windows binutils/gcc distribution supporting the x86_64-pc-none-elf and i686-pc-none-elf targets.

For now, users shall be responsible for providing their own Windows binutils/gcc build that supports x86_64-pc-none-elf and i686-pc-none-elf targets. In the future, the Windows Zephyr SDK distribution shall provide the binutils/gcc for use in this case.

As a proof of concept, you can try using the following Windows Zephyr SDK test build:
https://github.com/zephyrproject-rtos/crosstool-ng-old/releases/download/zephyr-crosstool-ng-1.24.0.4/zephyr-crosstool-ng-1.24.0.4_windows-x86_64_x86_64-zephyr-elf.zip

>path PATH = ...;D:\zephyr-crosstool-ng-1.24.0.4\x86_64-zephyr-elf\bin;C:\Program Files\LLVM\bin >clang dummy.c -o dummy -target x86_64-zephyr-elf -fuse-ld=bfd -nostdlib d:/zephyr-crosstool-ng-1.24.0.4/x86_64-zephyr-elf/bin/../lib/gcc/x86_64-zephyr-elf/9.2.0/../../../../x86_64-zephyr-elf/bin/ld.bfd.exe: warning: cannot find entry symbol _start; defaulting to 0000000000400078 >x86_64-zephyr-elf-objdump -x dummy dummy: file format elf64-x86-64 dummy architecture: i386:x86-64, flags 0x00000112: EXEC_P, HAS_SYMS, D_PAGED start address 0x0000000000400078 Program Header: STACK off 0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**4 filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw- Sections: Idx Name Size VMA LMA File off Algn 0 .comment 00000016 0000000000000000 0000000000000000 00000078 2**0 CONTENTS, READONLY SYMBOL TABLE: 0000000000000000 l d .comment 0000000000000000 .comment 0000000000000000 l df *ABS* 0000000000000000 dummy.c 0000000000000000 *UND* 0000000000000000 _start 0000000000601000 g *ABS* 0000000000000000 __bss_start 0000000000601000 g *ABS* 0000000000000000 _edata 0000000000601000 g *ABS* 0000000000000000 _end

if we don't use i686-pc-linux-elf, this option -fuse-ld=lld is not necessary too.
because this is more about environment issues, we need to ensure there is a gnu linker which supports i686-pc-none-elf, then it will invoke this linker by default and won't report error "program not executable".
I installed a cross compiler i486-elf on windows, it supports i686-pc-none-elf, just need to add it into system path, then it's ok to build zephyr binary.

@stephanosio I used -fuse-ld=lld because I wanted to use the llvm built-in linker to complete link process firstly, but I found that both on linux and windows, it reported the same error as below, then I used another i486-elf-ld on windows to build binary.

ld.lld: error: zephyr/linker_zephyr_prebuilt.cmd:29: malformed number: "__start"
epoint = (("__start") - ((0x0 + 0x100000) - (0x0 + 0x100000)));
^
collect2: error: ld returned 1 exit status

chen-png · 2021-07-19T02:13:44Z

closed this as this is more about environment issues, we need to ensure there is a gnu linker which supports i686-pc-none-elf on windows firstly, no need to change zephyr code base.

yanzhouy · 2022-01-10T03:11:12Z

@stephanosio Hi, I tried to use -fuse-ld=lld to force clang to use LLVM built-in linker on Windows, but if failed at step:
[178/191] Linking CXX executable zephyr/zephyr_prebuilt.elf
FAILED: zephyr/zephyr_prebuilt.elf zephyr/zephyr_prebuilt.map
with error:

And Peng @chen-png also occurred this issue on Linux.
Could you please give us some advises on this? Is it required to change zephyr code or is it a compiler issue?

yanzhouy · 2022-01-10T08:43:50Z

Regarding to https://www.mail-archive.com/[email protected]/msg54117.html,
I tried with updated LLD 14.0.0 (compatible with GNU linkers), build stopped with error:

chen-png requested review from nashif and tejlmand as code owners July 6, 2021 08:38

github-actions bot added the area: Build System label Jul 6, 2021

chen-png requested a review from LeiW000 July 6, 2021 08:47

stephanosio requested changes Jul 6, 2021

View reviewed changes

LeiW000 reviewed Jul 6, 2021

View reviewed changes

zephyrbot assigned tejlmand Jul 6, 2021

chen-png force-pushed the fix_issue_#32111 branch 2 times, most recently from 28d3782 to c62c20e Compare July 6, 2021 15:02

nashif requested a review from stephanosio July 7, 2021 12:54

nashif reviewed Jul 8, 2021

View reviewed changes

chen-png force-pushed the fix_issue_#32111 branch from c62c20e to 3f53fa0 Compare July 9, 2021 12:55

stephanosio requested changes Jul 12, 2021

View reviewed changes

chen-png closed this Jul 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LLVM: add LLVM support on Windows #36753

LLVM: add LLVM support on Windows #36753

chen-png commented Jul 6, 2021 •

edited by nashif

Loading

stephanosio left a comment •

edited

Loading

LeiW000 Jul 6, 2021

chen-png Jul 6, 2021

nashif Jul 8, 2021

chen-png Jul 9, 2021

stephanosio Jul 9, 2021

stephanosio Jul 12, 2021

stephanosio Jul 12, 2021

chen-png Jul 12, 2021

chen-png Jul 12, 2021

chen-png Jul 12, 2021

chen-png commented Jul 6, 2021

yanzhouy commented Jul 7, 2021

LeiW000 commented Jul 7, 2021

nashif Jul 8, 2021

nashif Jul 8, 2021

chen-png Jul 9, 2021

stephanosio Jul 12, 2021

chen-png Jul 12, 2021

stephanosio Jul 12, 2021

stephanosio Jul 12, 2021

chen-png Jul 12, 2021

chen-png Jul 12, 2021

chen-png commented Jul 19, 2021

yanzhouy commented Jan 10, 2022 •

edited

Loading

yanzhouy commented Jan 10, 2022 •

edited

Loading

LLVM: add LLVM support on Windows #36753

LLVM: add LLVM support on Windows #36753

Conversation

chen-png commented Jul 6, 2021 • edited by nashif Loading

stephanosio left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chen-png commented Jul 6, 2021

yanzhouy commented Jul 7, 2021

LeiW000 commented Jul 7, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chen-png commented Jul 19, 2021

yanzhouy commented Jan 10, 2022 • edited Loading

yanzhouy commented Jan 10, 2022 • edited Loading

chen-png commented Jul 6, 2021 •

edited by nashif

Loading

stephanosio left a comment •

edited

Loading

yanzhouy commented Jan 10, 2022 •

edited

Loading

yanzhouy commented Jan 10, 2022 •

edited

Loading