Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llvmPackages_16: init #223282

Merged
merged 2 commits into from
May 3, 2023
Merged

llvmPackages_16: init #223282

merged 2 commits into from
May 3, 2023

Conversation

RaitoBezarius
Copy link
Member

@RaitoBezarius RaitoBezarius commented Mar 26, 2023

Description of changes

For aarch64-darwin: some old workarounds should be re-introduced.
For other archs: help me!

DO NOT MERGE : this is for review and further work only, as @rrbutani is working on introducing a common function between LLVM versions to reduce the duplication.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@SuperSandro2000
Copy link
Member

The babel commit is on staging, so a rebase should get it in

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that file is empty

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I need to check why.

Copy link
Member

@SuperSandro2000 SuperSandro2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should stop copying the entire directory and make changes to it, thats not in the nix spirit.
probably out of scope for this but something that should eventually be improved.

@RaitoBezarius
Copy link
Member Author

we should stop copying the entire directory and make changes to it, thats not in the nix spirit. probably out of scope for this but something that should eventually be improved.

This is still being worked on, you can see what we are doing in the Compilers Team matrix. We will get to this eventually, no worries.

@RaitoBezarius
Copy link
Member Author

The babel commit is on staging, so a rebase should get it in

I fucked up my previous PR, but I will pull in the next time I try.

@primeos
Copy link
Member

primeos commented Mar 31, 2023

Thanks for the PR @RaitoBezarius! :)

IMO we should definitely cc the llvm team for completeness here (even though you already seem to be part of it 😄): @dtzWill @Ericson2314 @lovek323 @primeos @alyssais @RaitoBezarius @rrbutani @sternenseemann

Would be great if we could get this into a mergable state ASAP to test chromiumDev (M113 will become chromiumBeta in a few days and will be released on the 2nd of May so we don't have much time at all, unfortunately (given that we have to test Chromium, fix potential regressions, and backport LLVM 16 to NixOS 22.11, etc.)). IMO it'd be perfectly fine to mark a few "optional" packages as broken for now and fix them via additional PRs (I already used that approach in the past as I only had to add new versions for Chromium).

we should stop copying the entire directory and make changes to it, thats not in the nix spirit. probably out of scope for this but something that should eventually be improved.

Definitely out of scope for this PR! :D

I fucked up my previous PR, but I will pull in the next time I try.

This confuses me. What previous PR? You should be able to just rebase and force push without having to open a new PR.

PS: I like my old approach of having a dedicated commit to copy the files (e.g., llvmPackages_13: Copy from llvmPackages_git / #132643 / #162104) so that the other commits clearly show the relevant changes but IIRC that hasn't been done since then (and it also has its drawbacks so feel free to use a different approach).

@primeos primeos linked an issue Mar 31, 2023 that may be closed by this pull request
@RaitoBezarius
Copy link
Member Author

Thanks for the PR @RaitoBezarius! :)

IMO we should definitely cc the llvm team for completeness here (even though you already seem to be part of it smile): @dtzWill @Ericson2314 @lovek323 @primeos @alyssais @RaitoBezarius @rrbutani @sternenseemann

Would be great if we could get this into a mergable state ASAP to test chromiumDev (M113 will become chromiumBeta in a few days and will be released on the 2nd of May so we don't have much time at all, unfortunately (given that we have to test Chromium, fix potential regressions, and backport LLVM 16 to NixOS 22.11, etc.)). IMO it'd be perfectly fine to mark a few "optional" packages as broken for now and fix them via additional PRs (I already used that approach in the past as I only had to add new versions for Chromium).

Let's try to do this :). With @alyssais and @rrbutani we are trying to reduce the differences between llvmPackages_15 and llvmPackages_git.

we should stop copying the entire directory and make changes to it, thats not in the nix spirit. probably out of scope for this but something that should eventually be improved.

Definitely out of scope for this PR! :D

I fucked up my previous PR, but I will pull in the next time I try.

This confuses me. What previous PR? You should be able to just rebase and force push without having to open a new PR.

I created a previous PR which almost mass pinged a lot of people. :)

PS: I like my old approach of having a dedicated commit to copy the files (e.g., llvmPackages_13: Copy from llvmPackages_git / #132643 / #162104) so that the other commits clearly show the relevant changes but IIRC that hasn't been done since then (and it also has its drawbacks so feel free to use a different approach).

Got it. :)

@RaitoBezarius RaitoBezarius changed the base branch from staging to master April 4, 2023 11:53
@RaitoBezarius RaitoBezarius force-pushed the llvmPackages_16 branch 2 times, most recently from 28c7966 to 54abe0f Compare April 4, 2023 12:29
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Apr 4, 2023
@ofborg ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Apr 4, 2023
@3541
Copy link

3541 commented Apr 8, 2023

libunwind is broken on armv7l-linux too:

<......................................................>
libunwind> -- Build files have been written to: /build/libunwind-src-16.0.0/runtimes/build
libunwind> cmake: enabled parallel building
libunwind> cmake: enabled parallel installing
libunwind> building
libunwind> build flags: -j1
libunwind> [1/19] Building CXX object libunwind/src/CMakeFiles/unwind_shared_objects.dir/libunwind.cpp.o
libunwind> FAILED: libunwind/src/CMakeFiles/unwind_shared_objects.dir/libunwind.cpp.o
libunwind> /nix/store/b1imbh7gwvrbl18lk7sihj7a3smmz8j3-clang-wrapper-16.0.0/bin/clang++ -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LIBUNWIND_LINK_DL_LIB -D_LIBUNWIND_LINK_PTHREAD_LIB -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/build/libunwind-src-16.0.0/libunwind/include -fvisibility-inlines-hidden -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -fdiagnostics-color -O3 -DNDEBUG -fPIC -Werror=return-type -W -Wall -Wchar-subscripts -Wconversion -Wmismatched-tags -Wmissing-braces -Wnewline-eof -Wno-unused-function -Wshadow -Wshorten-64-to-32 -Wsign-compare -Wsign-conversion -Wstrict-aliasing=2 -Wstrict-overflow=4 -Wunused-parameter -Wunused-variable -Wwrite-strings -Wundef -Wno-suggest-override -Wno-error -pedantic -funwind-tables -nostdinc++ -D_DEBUG -UNDEBUG -D_LIBUNWIND_IS_NATIVE_ONLY -fno-rtti -std=c++11  -fstrict-aliasing -fno-exceptions -fno-rtti -MD -MT libunwind/src/CMakeFiles/unwind_shared_objects.dir/libunwind.cpp.o -MF libunwind/src/CMakeFiles/unwind_shared_objects.dir/libunwind.cpp.o.d -o libunwind/src/CMakeFiles/unwind_shared_objects.dir/libunwind.cpp.o -c /build/libunwind-src-16.0.0/libunwind/src/libunwind.cpp
libunwind> clang-16: warning: argument unused during compilation: '-rtlib=compiler-rt' [-Wunused-command-line-argument]
libunwind> In file included from /build/libunwind-src-16.0.0/libunwind/src/libunwind.cpp:12:
libunwind> /build/libunwind-src-16.0.0/libunwind/include/libunwind.h:19:10: fatal error: 'stddef.h' file not found
libunwind> #include <stddef.h>
libunwind>          ^~~~~~~~~~
libunwind> 1 error generated.
libunwind> ninja: build stopped: subcommand failed.
error: builder for '/nix/store/n6r57sdwkz06i52yxfhk78baf6y6wyj4-libunwind-16.0.0.drv' failed with exit code 1;

Full log

I think it's just because the clang-lib include directory is now at lib/clang/16/include, not lib/clang/16.0.0/include, meaning the symlink in resource-root no longer points to the right place. I was able to get it to build (on amd64) with this:

diff --git a/pkgs/development/compilers/llvm/16/default.nix b/pkgs/development/compilers/llvm/16/default.nix
index 17074a37b2b..a7b6c18982e 100644
--- a/pkgs/development/compilers/llvm/16/default.nix
+++ b/pkgs/development/compilers/llvm/16/default.nix
@@ -102,10 +102,11 @@ in let

   tools = lib.makeExtensible (tools: let
     callPackage = newScope (tools // { inherit stdenv cmake ninja libxml2 python3 release_version version monorepoSrc buildLlvmTools; });
+    major = lib.versions.major release_version;
     mkExtraBuildCommands0 = cc: ''
       rsrc="$out/resource-root"
       mkdir "$rsrc"
-      ln -s "${cc.lib}/lib/clang/${release_version}/include" "$rsrc"
+      ln -s "${cc.lib}/lib/clang/${major}/include" "$rsrc"
       echo "-resource-dir=$rsrc" >> $out/nix-support/cc-cflags
     '';
     mkExtraBuildCommands = cc: mkExtraBuildCommands0 cc + ''

But I'm not sure whether that's actually the right solution.

@rrbutani
Copy link
Contributor

rrbutani commented Apr 8, 2023

I think it's just because the clang-lib include directory is now at lib/clang/16/include, not lib/clang/16.0.0/include,

But I'm not sure whether that's actually the right solution.

Yup, this is exactly right; this is a documented change in LLVM16.

@RaitoBezarius
Copy link
Member Author

Thank you everyone for your help, with this, LLVM16 is compiling fine on x86_64-linux.
Please by all means, test the latest revision now.

For aarch64-darwin, I see 2 test failures:

FAIL: LLVM :: ExecutionEngine/Interpreter/intrinsics.ll (24905 of 47137)
******************** TEST 'LLVM :: ExecutionEngine/Interpreter/intrinsics.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/bin/lli -jit-kind=mcjit -O0 -force-interpreter < /private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/test/ExecutionEngine/Interpreter/intrinsics.ll
--
Exit Code: 134

Command Output (stderr):
--
LLVM ERROR: Tried to execute an unknown external function: roundevenf
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/bin/lli -jit-kind=mcjit -O0 -force-interpreter
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  libLLVM.dylib            0x000000010c4f56a0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 72
1  libLLVM.dylib            0x000000010c4f45a8 llvm::sys::RunSignalHandlers() + 128
2  libLLVM.dylib            0x000000010c4f5d24 SignalHandler(int) + 320
3  libsystem_platform.dylib 0x00000001ada002a4 _sigtramp + 56
4  libsystem_pthread.dylib  0x00000001ad9d1cec pthread_kill + 288
5  libsystem_c.dylib        0x00000001ad90b2c8 abort + 180
6  libLLVM.dylib            0x000000010c437f00 llvm::report_fatal_error(llvm::Twine const&, bool) + 468
7  libLLVM.dylib            0x000000010dea2d3c llvm::Interpreter::callExternalFunction(llvm::Function*, llvm::ArrayRef<llvm::GenericValue>) + 4264
8  libLLVM.dylib            0x000000010de96f68 llvm::Interpreter::callFunction(llvm::Function*, llvm::ArrayRef<llvm::GenericValue>) + 192
9  libLLVM.dylib            0x000000010de96e2c llvm::Interpreter::visitCallBase(llvm::CallBase&) + 396
10 libLLVM.dylib            0x000000010dea0c20 llvm::Interpreter::run() + 64
11 libLLVM.dylib            0x000000010dea4a7c llvm::Interpreter::runFunction(llvm::Function*, llvm::ArrayRef<llvm::GenericValue>) + 60
12 libLLVM.dylib            0x000000010de7ebfc llvm::ExecutionEngine::runFunctionAsMain(llvm::Function*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, char const* const*) + 1044
13 lli                      0x0000000104f80fc4 main + 9284
14 dyld                     0x00000001ad6a7e50 start + 2544
/private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/test/ExecutionEngine/Interpreter/Output/intrinsics.ll.script: line 2: 51722 Abort trap: 6           /private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/bin/lli -jit-kind=mcjit -O0 -force-interpreter < /private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/test/ExecutionEngine/Interpreter/intrinsics.ll

--

********************
FAIL: LLVM-Unit :: TargetParser/./TargetParserTests/11/12 (45775 of 47137)
******************** TEST 'LLVM-Unit :: TargetParser/./TargetParserTests/11/12' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/unittests/TargetParser/./TargetParserTests-LLVM-Unit-29877-11-12.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=12 GTEST_SHARD_INDEX=11 /private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/unittests/TargetParser/./TargetParserTests
--

Script:
--
/private/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/build/unittests/TargetParser/./TargetParserTests --gtest_filter=HostTest.getMacOSHostVersion
--
/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/unittests/TargetParser/Host.cpp:397: Failure
Expected equality of these values:
  0
  RetCode
    Which is: -1
/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/unittests/TargetParser/Host.cpp:430: Failure
Expected equality of these values:
  runAndGetCommandOutput(SwVersPath, argv, Buffer, Size)
    Which is: false
  true

/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/unittests/TargetParser/Host.cpp:397
Expected equality of these values:
  0
  RetCode
    Which is: -1
/tmp/nix-build-llvm-16.0.1.drv-0/llvm-src-16.0.1/llvm/unittests/TargetParser/Host.cpp:430
Expected equality of these values:
  runAndGetCommandOutput(SwVersPath, argv, Buffer, Size)
    Which is: false
  true


********************
********************
Failed Tests (2):
  LLVM :: ExecutionEngine/Interpreter/intrinsics.ll
  LLVM-Unit :: TargetParser/./TargetParserTests/HostTest/getMacOSHostVersion

@RaitoBezarius
Copy link
Member Author

@primeos With this, we have a working LLVM16 for x86_64-linux if you want to run tests for Chromium M113.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2090

@primeos
Copy link
Member

primeos commented Apr 19, 2023

@RaitoBezarius thanks for the heads-up (and sorry for the delay - I was sick). I can built chromiumBeta (M113) with llvmPackages_16 from this PR and the VM test passes (which is quite a relief 😄) so functionally it LGTM. Would be great if we could get this merged soon. Are there any blockers left?

@RaitoBezarius
Copy link
Member Author

@RaitoBezarius thanks for the heads-up (and sorry for the delay - I was sick). I can built chromiumBeta (M113) with llvmPackages_16 from this PR and the VM test passes (which is quite a relief smile) so functionally it LGTM. Would be great if we could get this merged soon. Are there any blockers left?

The idea was to have some common "helper" function to be involved once we finished ensuring that llvmPackages_15 and llvmPackages_git have almost zero delta in patches, etc.

cc @rrbutani @alyssais

I think we are almost there for the delta in patches. llvmPackages_git was bumped to llvmPackages_15.
Once we confirm that delta is zero, we can introduce the helper I guess, to reduce code duplication and onboard llvmPackages_16 as part of this helper with updated patches.

@rrbutani rrbutani mentioned this pull request May 2, 2023
12 tasks
Kept separate from 2c627d9 to skip CI for this trivial change.
@primeos
Copy link
Member

primeos commented May 3, 2023

DO NOT MERGE : this is for review and further work only, as @rrbutani is working on introducing a common function between LLVM versions to reduce the duplication.

Chromium M113 got released yesterday and contains important security fixes.
LLVM 16 is required now so I'll merge this now. Sorry not sorry :P
We really need to have new LLVM versions in Nixpkgs much sooner... (https://github.com/llvm/llvm-project/releases/tag/llvmorg-16.0.0 is from 2023-03-18)

@primeos primeos marked this pull request as ready for review May 3, 2023 21:56
@primeos primeos requested a review from matthewbauer as a code owner May 3, 2023 21:56
@primeos primeos merged commit b8960cd into NixOS:master May 3, 2023
@RaitoBezarius
Copy link
Member Author

DO NOT MERGE : this is for review and further work only, as @rrbutani is working on introducing a common function between LLVM versions to reduce the duplication.

Chromium M113 got released yesterday and contains important security fixes. LLVM 16 is required now so I'll merge this now. Sorry not sorry :P We really need to have new LLVM versions in Nixpkgs much sooner... (https://github.com/llvm/llvm-project/releases/tag/llvmorg-16.0.0 is from 2023-03-18)

Obviously, for security measures, I definitely understand (and that's why I left it this way.), we wanted to race you before.

So, it's definitely okay IMHO.

@willcohen
Copy link
Contributor

When trying to update emscripten to use LLVM 16 rather than a patched LLVM 15 (see #229718), the build fails with the following error (and succeeds on 15) on darwin-aarch64 at least:

ache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libcompiler_rt.a... (this will be cached in "/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libcompiler_rt.a" for subsequent builds)
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c:21:33: error: unknown type name 'atomic_flag'
void atomic_flag_clear(volatile atomic_flag *object) {
                                ^
1 error generated.
emcc: error: '/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration --sysroot=/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot -resource-dir=/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/lib/clang/16.0.1/ -idirafter/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g3 -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_flag_clear.o' failed (returned 1)
emcc: error: Subprocess 16/175 failed (returned 1)! (cmdline: /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/bin/emcc -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g -sSTRICT -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_flag_clear.o)
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_test_and_set.c:21:41: error: unknown type name 'atomic_flag'
_Bool atomic_flag_test_and_set(volatile atomic_flag *object) {
                                        ^
1 error generated.
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear_explicit.c:21:42: error: unknown type name 'atomic_flag'
void atomic_flag_clear_explicit(volatile atomic_flag *object,
                                         ^
emcc: error: '/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration --sysroot=/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot -resource-dir=/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/lib/clang/16.0.1/ -idirafter/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g3 -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_test_and_set.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_flag_test_and_set.o' failed (returned 1)
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear_explicit.c:22:33: error: unknown type name 'memory_order'
                                memory_order order) {
                                ^
2 errors generated.
emcc: error: '/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration --sysroot=/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot -resource-dir=/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/lib/clang/16.0.1/ -idirafter/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g3 -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear_explicit.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_flag_clear_explicit.o' failed (returned 1)
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_signal_fence.c:21:26: error: unknown type name 'memory_order'
void atomic_signal_fence(memory_order order) {
                         ^
1 error generated.
emcc: error: '/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration --sysroot=/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot -resource-dir=/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/lib/clang/16.0.1/ -idirafter/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g3 -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_signal_fence.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_signal_fence.o' failed (returned 1)
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_test_and_set_explicit.c:21:50: error: unknown type name 'atomic_flag'
_Bool atomic_flag_test_and_set_explicit(volatile atomic_flag *object,
                                                 ^
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_test_and_set_explicit.c:22:41: error: unknown type name 'memory_order'
                                        memory_order order) {
                                        ^
2 errors generated.
/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_thread_fence.c:21:26: error: unknown type name 'memory_order'
void atomic_thread_fence(memory_order order) {
                         ^
emcc: error: '/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration --sysroot=/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot -resource-dir=/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/lib/clang/16.0.1/ -idirafter/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g3 -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_test_and_set_explicit.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_flag_test_and_set_explicit.o' failed (returned 1)
1 error generated.
emcc: error: '/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration --sysroot=/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot -resource-dir=/nix/store/2qb7w9ch3qlxjkp16syga3j4bgbsclvl-emscripten-llvm-3.1.36/lib/clang/16.0.1/ -idirafter/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -O2 -Wall -Werror -fno-unroll-loops -fno-builtin -g3 -I/nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/libc -c /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/system/lib/compiler-rt/lib/builtins/atomic_thread_fence.c -o /nix/store/wbpjz395ik0ml56gp2cc8xyv9jis6bql-emscripten-3.1.36/share/emscripten/cache/build/libcompiler_rt-tmp/atomic_thread_fence.o' failed (returned 1)

@willcohen
Copy link
Contributor

Based on emscripten-core/emscripten#18302, it may be that libc++ may have some kind of mismatch. Hopefully there is a way to get that to align for emscripten alone.

@nedwill
Copy link

nedwill commented May 4, 2023

I also hit this error today for a C codebase using atomics. The header that I expect to be included in my toolchain is located at /nix/store/[hash]-clang-16.0.1-lib/lib/clang/16/include/stdatomic.h on my system. I'm also debugging this under the assumption that I have a config problem (using nixpkgs_cc_configure with Bazel), but just sharing since I was surprised to see someone else run into this.

@fleimgruber
Copy link
Contributor

@primeos @RaitoBezarius thanks for your efforts! Is this planned to be backported to 22.11 or do you aim for 23.05 already?

@primeos
Copy link
Member

primeos commented May 6, 2023

@RaitoBezarius thanks for understanding :) And of course thanks for this awesome PR that saved me a lot of work! <3

I wanted to write another comment last week but it looks like I somehow forgot to... :o At least I already mentioned the hard deadline in #223282 (comment). But I also wanted to mention that it would be best to merge this as it is anyway since I need to backport LLVM 16 to 22.11 for Chromium (which would likely be way too risky with that common "helper" function). And merging this as it is shouldn't interfere with that common "helper" function as it basically only adds new files and we can simply delete all of pkgs/development/compilers/llvm/16/ once that abstraction is ready (might just have to consider a few additional patches that got commited in the meantime).

https://hydra.nixos.org/eval/1794582?filter=llvmPackages_16&compare=1794561&full= looks really good (no failures - I only tested Chromium and expected that some LLVM packages might still fail) so I'll try to give backporting a try today (if someone else wants to handle it or help that's of course welcome as well - it should just be as simple as possible / a straightforward backport to get it merged ASAP).

@primeos @RaitoBezarius thanks for your efforts! Is this planned to be backported to 22.11 or do you aim for 23.05 already?

Yes, I already mentioned it in #223282 (comment) ;)

@primeos primeos mentioned this pull request May 6, 2023
12 tasks
@primeos
Copy link
Member

primeos commented May 6, 2023

I had a bit of time just now so I already got around to preparing a backport: #230327
Hopefully I didn't mess anything up. If CI passes and it doesn't cause (mass)rebuilds it should be good to go.

@willcohen
Copy link
Contributor

@nedwill
The problem for me was the use of the llvm release_version -- the derivation was trying to include a directory at /16.0.1 while the actual location built by llvm 16 is just /16.

@nedwill
Copy link

nedwill commented May 20, 2023

@willcohen Thanks for letting me know. I was having some trouble getting one of the documentation tests to pass but was able to work around it with a derivation that skipped checking the tests. Now that this is merged the binary package the original derivation is working great for me.

@willcohen willcohen mentioned this pull request Jan 2, 2024
13 tasks
@rrbutani rrbutani added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 11.by: package-maintainer This PR was created by the maintainer of the package it changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

request: add llvmPackages_16