Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llvmPackages_13.compiler-rt-libc: fix python3 setuptools dependency #327038

Merged
merged 1 commit into from
Jul 31, 2024

Conversation

numinit
Copy link
Contributor

@numinit numinit commented Jul 14, 2024

Description of changes

Fixing LLVM 13 breakage due to setuptools removal in #320924.

Partially resolves #326927

Build of llvmPackages_13.compiler-rt-libc fails at the tail end with:

Traceback (most recent call last):
  File "/build/source/libcxx/utils/merge_archives.py", line 12, in <module>
    import distutils.spawn
ModuleNotFoundError: No module named 'distutils'
make[5]: *** [cxx/src/CMakeFiles/cxx_static.dir/build.make:739: /build/source/compiler-rt/build/lib/fuzzer/libcxx_fuzzer_x86_64/lib/libc++.a] Error 1
make[5]: *** Deleting file '/build/source/compiler-rt/build/lib/fuzzer/libcxx_fuzzer_x86_64/lib/libc++.a'
make[4]: *** [CMakeFiles/Makefile2:444: cxx/src/CMakeFiles/cxx_static.dir/all] Error 2
make[3]: *** [Makefile:136: all] Error 2
make[2]: *** [lib/fuzzer/CMakeFiles/libcxx_fuzzer_x86_64-build.dir/build.make:73: lib/fuzzer/libcxx_fuzzer_x86_64-stamps/libcxx_fuzzer_x86_64-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:8967: lib/fuzzer/CMakeFiles/libcxx_fuzzer_x86_64-build.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 96%] Built target RTAsan_dynamic.x86_64
[ 96%] Built target RTAsan.x86_64
make: *** [Makefile:136: all] Error 2

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@numinit numinit requested a review from RossComputerGuy as a code owner July 14, 2024 07:03
@github-actions github-actions bot added 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related labels Jul 14, 2024
@numinit numinit added the 0.kind: build failure A package fails to build label Jul 14, 2024
@numinit
Copy link
Contributor Author

numinit commented Jul 14, 2024

Uh oh, infinite recursion...

@numinit
Copy link
Contributor Author

numinit commented Jul 14, 2024

llvm/llvm-project#54337

@numinit
Copy link
Contributor Author

numinit commented Jul 14, 2024

Since the PR's patch won't work, I'm trying to figure out where to put this. merge_archives.py was dropped in v15, seems mostly unused in v14, and needs the following patch there and in v13: edit: fixed.

+ lib.optionalString (lib.versionOlder release_version "15") ''
    # https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/libcxx/utils/merge_archives.py dropped in v15
    substituteInPlace libcxx/utils/merge_archives.py \
      --replace-fail 'import distutils.spawn' 'from shutil import which as find_executable' \
      --replace-fail distutils.spawn. ""
  '' +

Original source: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/libcxx/utils/merge_archives.py

  • pkgs/development/compilers/llvm/common/libcxx/default.nix: doesn't appear to be the right place
  • pkgs/development/compilers/llvm/common/compiler-rt/default.nix: doesn't appear to have libcxx

nix-build -A llvmPackages_13.compiler-rt-libc is how I'm testing.

cc @RossComputerGuy

@numinit numinit marked this pull request as draft July 14, 2024 08:47
@numinit numinit changed the title llvm/compiler-rt-libc: fix python3 setuptools dependency llvmPackages_{13,14}: compiler-rt-libc: fix python3 setuptools dependency Jul 14, 2024
@numinit numinit force-pushed the llvm-compiler-rt-python3 branch 2 times, most recently from 84c41e7 to dc57cdd Compare July 15, 2024 04:13
@numinit numinit marked this pull request as ready for review July 15, 2024 04:13
@numinit numinit force-pushed the llvm-compiler-rt-python3 branch from dc57cdd to bda34eb Compare July 15, 2024 04:23
@numinit numinit changed the title llvmPackages_{13,14}: compiler-rt-libc: fix python3 setuptools dependency llvmPackages_13: compiler-rt-libc: fix python3 setuptools dependency Jul 15, 2024
@numinit
Copy link
Contributor Author

numinit commented Jul 15, 2024

Got the PR down to just the fix for 13. 12 and 14 are all right.

Merging in #327037 to test:

nix-build -E 'with (import ./. {}); klee.override { llvmPackages = llvmPackages_12; }'
/nix/store/9ljqkj2c1y8p3vl52fmx9xysw09fvmlg-klee-3.1
nix-build -E 'with (import ./. {}); klee.override { llvmPackages = llvmPackages_13; }'
/nix/store/8n7d7gl6vpv1lab18yj4gghgnpmj3815-klee-3.1
nix-build -E 'with (import ./. {}); klee.override { llvmPackages = llvmPackages_14; }'
/nix/store/07lp006mbzghbg8pm9livgh12za7isza-klee-3.1

@numinit numinit force-pushed the llvm-compiler-rt-python3 branch 2 times, most recently from 4ce47c4 to 502b500 Compare July 15, 2024 04:32
@numinit
Copy link
Contributor Author

numinit commented Jul 15, 2024

Updated comment with:

Seems to only be used in v13 though it's present in v12 and v14, and dropped in v15.

@RossComputerGuy RossComputerGuy changed the title llvmPackages_13: compiler-rt-libc: fix python3 setuptools dependency llvmPackages_13.compiler-rt-libc: fix python3 setuptools dependency Jul 15, 2024
@numinit numinit force-pushed the llvm-compiler-rt-python3 branch from 502b500 to 4c1d982 Compare July 15, 2024 04:37
@hissssst
Copy link

hissssst commented Jul 19, 2024

@RossComputerGuy, hi, this PR is blocking almost any upgrade of nixpkgs-unstable systems. Maybe you could help me out and give it a quick 5 minute review please, I'd heavily appreciate it

@RossComputerGuy
Copy link
Member

Maybe you could help me out and give it a quick 5 minute review please, I'd heavily appreciate it

My review is the same as @SuperSandro2000's. @numinit just needs to add the change and this will be gtg.

@numinit
Copy link
Contributor Author

numinit commented Jul 19, 2024

Thanks, I'll get to the change tonight. Busy week 😅

@numinit numinit force-pushed the llvm-compiler-rt-python3 branch from 4c1d982 to f486fe9 Compare July 19, 2024 08:51
@numinit
Copy link
Contributor Author

numinit commented Jul 19, 2024

Done.

This function was removed in Python 3.12 and seems to only be used in
LLVM 13, despite the Python file this is used in being present in
LLVM 12 and 14. The merge_archives.py file was removed in LLVM 15.
@numinit numinit force-pushed the llvm-compiler-rt-python3 branch from f486fe9 to 37ba897 Compare July 19, 2024 08:51
@hissssst
Copy link

@numinit @RossComputerGuy what else is required to have it merged into master and thenunstable? Maybe I can ask around in some chats to help move this patch?

@numinit
Copy link
Contributor Author

numinit commented Jul 31, 2024

Good question. As far as I know, it works and fixes KLEE, at least.

@RossComputerGuy RossComputerGuy merged commit 928c778 into NixOS:master Jul 31, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: build failure A package fails to build 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Development

Successfully merging this pull request may close these issues.

klee error: No module named 'distutils'
4 participants