Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue for llvmPackages_15 changes #213033

Closed
29 of 48 tasks
rrbutani opened this issue Jan 27, 2023 · 8 comments
Closed
29 of 48 tasks

Tracking Issue for llvmPackages_15 changes #213033

rrbutani opened this issue Jan 27, 2023 · 8 comments
Assignees
Labels
5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related

Comments

@rrbutani
Copy link
Contributor

rrbutani commented Jan 27, 2023

This issue tracks the work left over from #194634.

Fixes/missed patches for LLVM 15

In-flight

Unifying llvmPackages_git with llvmPackages_15

Tracking the current state of llvmPackages_git (i.e. all the PRs listed above applied) here: rrbutani/tracking/llvm-git-parity

LLVM Package Set Updates


Tests and additional coverage:

Issues/bugs:

  • fix use as a cross-compiler targeting FreeBSD (libcxx currently fails to build)
  • fix pkgsStatic.llvmPackages_15
  • follow up on the armv7l build (didn't finish building before llvmPackages_15: init at 15.0.7 #194634 was merged)
  • the three failing x86_64-darwin LLVM tests
    • one: roundevenf isn't present on x86_64-darwin; unclear if this is expected or not
    • two: sw_vers on aarch64-darwin links against frameworks (CoreFoundation) outside the sandbox
    • three: crash in a dsymutil test

Future Work/Wishlist

(not tracked by this issue; will point to issues here as we begin working on these things)

  • common LLVM function across the package sets
  • LLVM 16
  • updating llvmPackages_git to tip-of-tree
  • bumping llvmPackages to newer LLVM
  • MLIR
  • BOLT
  • auto updater
  • splitting useLLVM
  • expanding test coverage (i.e. enabling doCheck on more packages/platforms)
  • adding regression tests for the stdenv(s)
  • building LTO/PGO/bolt optimized binaries in the llvmPackages sets
  • reducing closure size of LLVM by splitting out the .a files in lib outputs? (but in a way that doesn't break users of LLVM static libs..)
@rrbutani rrbutani self-assigned this Jan 27, 2023
@rrbutani rrbutani added this to pending Jan 28, 2023
@Et7f3
Copy link
Contributor

Et7f3 commented Jan 28, 2023

Cool the PR is merged is their a Darwin task you are not working on ?

@rrbutani
Copy link
Contributor Author

rrbutani commented Jan 28, 2023

@Et7f3 I think, for now, I'm planning to focus on doing what I can to help bump llvmPackages_latest and then to update llvmPackages_git.

I'm still gathering my notes to update the OP with but off the top of my head, for darwin:

  • I didn't really get to the bottom of the three test failures, detailed in this commit
    • one, two, three
    • for the first one it probably makes sense to dig deeper into if/why the JIT test that links against roundevenf succeeds on aarch64-darwin (is the test actually running on aarch64-darwin? if so, where does that symbol come from, libSystem? is this an artifact of us seemingly using system frameworks – the second issue – on aarch64-darwin? etc)
    • the second one is pretty spooky (and the commit message linked above has more details about how to reproduce it) and probably involves digging into the aarch64-darwin stdenv and the build steps for DarwinTools to figure out why the system framework is getting linked
    • I haven't looked into the third one at all but verifying that that test case fails outside the sandbox too is probably a good place to start
  • Actually using the llvmPackages_15 stdenv (as part of this test script – note that aarch64-darwin and x86_64-darwin are commented out there in systems) leads to cycle errors on darwin
    • I don't have a good understanding of why; it seems like our stdenv gets used for some bootstrapping things, creating a cycle but it's not apparent to me why that doesn't affect the default (LLVM based) darwin stdenv
  • I wanted to maybe try enabling the tests for other packages in the set (libcxx and clang to start with)
    • this isn't strictly a darwin task but I'm sure we'll end up needing to spend time figuring out darwin/linux/etc specific fails as part of this 😃

If any of the above things sound interesting to you, let me know; I can update the OP accordingly and try to provide some more context.

None of the above are high priority or anything though; if there's something else you're interested in working on (i.e. getting cross-compiling LLVM to work in more configurations, experimenting with ways to share package definitions between the llvmPackages_* sets, etc) definitely feel free to pursue that instead!

@Et7f3
Copy link
Contributor

Et7f3 commented Jan 30, 2023

In the other PR this technique was used for newer requirements #209536 (comment) it use gcc on linux and not clang which might not be the default. This patch avoid mixing compiler, What do you think ?

I think I will try to enable test for libcxx on x86_64-darwin. I thought on enabling the test on all platforms and disabling those failing like

# !!! Note: these tests are removed in LLVM 16.
#
# See here for context: https://github.com/NixOS/nixpkgs/pull/194634#discussion_r999790443
rm test/CodeGen/RISCV/rv32zbp.ll
rm test/CodeGen/RISCV/rv64zbp.ll

So other works doesn't add regression and we can still keep progress.

@Et7f3
Copy link
Contributor

Et7f3 commented Jan 30, 2023

Also linking a PR linked to these #213497

@rrbutani
Copy link
Contributor Author

@Et7f3 Apologies for the late response.

In the other PR this technique was used for newer requirements #209536 (comment) it use gcc on linux and not clang which might not be the default. This patch avoid mixing compiler, What do you think ?

I believe it's okay for us to mix and match compilers and that it's mixing C++ stdlib/libc impls that's cause for concern. For example, AIUI, llvmPackages.stdenv exists and defaults to the platform's C++ stdlib instead of always using libc++ specifically so that some packages -- like chromium -- can use clang but can still link against other libraries compiled against libstdc++ (default on linux targets).

In the case of libc++/libc++-abi, we aren't linking against a C++ stdlib impl anyways (and the stdenv we use to build these packages is careful to use the platform's libc) so I think this isn't a concern for us (those with more experience in this area should definitely feel free to weigh in or clarify).

Given that, I think using the clang that's built alongside libcxx is preferable; it mirrors how libc++ is built in upstream LLVM and it lets us avoid having to maintain logic to select a new-enough gcc version, per-platform.

I think I will try to enable test for libcxx on x86_64-darwin.

That sounds great! I'll add an item for this in the OP; let me know if you open any issues or PRs and I can add them too.

@misuzu

This comment was marked as resolved.

@Artturin
Copy link
Member

Artturin commented May 6, 2023

should be applied to LLVM 15: #207863

almost fix in #230387 , will have to be actually fixed in a staging PR

@pca006132
Copy link
Contributor

I managed to get Bolt built in #176536 (comment), not yet tested (will do in a few days). A lot of patching has to be done, should I open a PR or try to get something upstreamed first?

@samueldr samueldr added the 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems label Apr 23, 2024
@rrbutani rrbutani closed this as completed May 8, 2024
@github-project-automation github-project-automation bot moved this to Done in pending May 8, 2024
@rrbutani rrbutani added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related
Projects
None yet
Development

No branches or pull requests

6 participants