Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zig: commonify & bootstrap #331011

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

RossComputerGuy
Copy link
Member

Description of changes

This is the first step towards my goal of being able to bootstrap LLVM in Nix with Zig. A couple things this PR does is commonify Zig to make handling future updates simpler, I've been meaning to do this for a while. This PR also implements the bootstrap derivations for Zig.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

hash = "sha256-dDOmSRBKl/gs7I3kmLXIyQk3zsOdlaYov72pPSel4+I=";
})
];
./linker-support-zig-cc.patch;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we patching this in Nixpkgs at all instead of fixing it properly upstream? My read of the upstream PR was that upstream was responsive and interested, and that there was just the small issue of getting the test suite fixed left to do, and upstream had even provided a hint for how to do that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because there's already an upstream PR. This is just a vendored copy after reading the feedback left after #318034 was merged. I don't see anything about a test suite fix and I'm not familiar with how meson works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that this is just vendoring the patch, but I'm not thrilled that it was added in the first place ­— the original author of the upstream PR doesn't seem to be continuing with it, which means that we have to keep this patch around (with the implied maintenance burden on Nixpkgs' Meson maintainers) until somebody else picks it up. We can add patches to Nixpkgs when they're urgent, when it's not feasible to upstream them, or when it's doing something Nixpkgs-specific, but allowing patches like this, where none of those things are true, means that, for non-maintainers, there's never any incentive to upstream anything, resulting in packages getting more and more complicated until they collapse under their own weight. For that reason, I think this patch needs to be sorted out upstream, and that we should not keep it around in Nixpkgs in the meantime. The path to getting it upstream is clear, and doesn't sound like it would be a lot of work.

I don't see anything about a test suite fix

mesonbuild/meson#12293 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, @tomberek and I talked yesterday about this patch. I only added it so meson projects would at least configure in pkgsZig. I understand the patch isn't exactly stable but there's not really any other way around the error which occurs without this patch. The problem looks like tests in meson fail because Zig doesn't support a flag which means we likely would have to implement the -rpath-link flag.

Copy link
Member

@alyssais alyssais Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's not really any other way around the error which occurs without this patch.

Well, there is, and it's to help this get fixed upstream, which as far as I can see means just extending an existing check to also check for Zig.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I've made a patch mesonbuild/meson#13493

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks. For testing on NixOS I just use a nix-shell with python3 and whatever else the test suite needs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it looks like I'll need to pull in the CC wrapper for the tests to perform correctly.

@Aleksanaa
Copy link
Member

The CI is considered not correct on unused arguments in this case, as noted in #331085.

@RossComputerGuy RossComputerGuy force-pushed the feat/zig-bootstrap branch 7 times, most recently from a0c11c2 to 5af85b0 Compare August 1, 2024 04:47
@RossComputerGuy
Copy link
Member Author

zig.passthru.stdenv works better now.

''
mkdir -p $out/bin
for tool in ar objcopy; do
makeWrapper "$zig/bin/zig" "$out/bin/${targetPrefix}-$tool" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be ${targetPrefix}$tool (without the -)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was a typo.

}
''
mkdir -p $out/bin
for tool in cc c++; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for tool in cc c++; do
for tool in cc c++ ld.lld; do

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 35 to 36
mv $out/bin/c++ $out/bin/clang++
mv $out/bin/cc $out/bin/clang
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mv $out/bin/c++ $out/bin/clang++
mv $out/bin/cc $out/bin/clang
mv $out/bin/cc $out/bin/clang
mv $out/bin/c++ $out/bin/clang++
mv $out/bin/ld.lld $out/bin/ld

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

}
''
mkdir -p $out/bin
for tool in ar objcopy; do
Copy link
Contributor

@m-bdf m-bdf Nov 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for tool in ar objcopy; do
for tool in ar objcopy ranlib; do

Those are the only 3 bintools available as zig subcommands though, so some packages fail to build with zigStdenv because they rely on other tools like strip.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@kirillrdy
Copy link
Member

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 331011


x86_64-linux

❌ 15 packages failed to build:
  • backlight-auto
  • blackshades
  • colorstorm
  • cyber
  • dt
  • findup
  • linuxwave
  • pkgsZig.stdenv
  • zig_0_10
  • zig_0_10.doc
  • zig_0_11
  • zig_0_11.doc
  • zig_0_9
  • zig_0_9.doc
  • zon2nix
✅ 28 packages built:
  • arocc (aroccPackages.latest)
  • aroccPackages.latest-unwrapped
  • aroccStdenv
  • cargo-lambda
  • cargo-zigbuild
  • clipbuzz
  • emacsPackages.zig-mode
  • glsl_analyzer
  • ly
  • mepo
  • minizign
  • ncdu
  • pkgsArocc.stdenv
  • poop
  • river
  • river.man
  • rivercarro
  • superhtml
  • waylock
  • wayprompt
  • zf
  • zig (zig_0_13)
  • zig.doc (zig_0_13.doc)
  • zigStdenv
  • zig_0_12
  • zig_0_12.doc
  • zls
  • ztags

@RossComputerGuy
Copy link
Member Author

I've figured out that Zig 0.9 and 0.10 have the postBuild hook wrong. Will fix tonight and move from staging to master.

@kirillrdy
Copy link
Member

we can soon remove 0.9 benbusby/colorstorm#15

@kirillrdy
Copy link
Member

zig 0.10 is also unused after #365116

@RossComputerGuy RossComputerGuy marked this pull request as draft December 23, 2024 02:12
@RossComputerGuy RossComputerGuy changed the base branch from staging to master December 23, 2024 03:48
@RossComputerGuy RossComputerGuy marked this pull request as ready for review December 23, 2024 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions 6.topic: zig 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 11.by: package-maintainer This PR was created by the maintainer of the package it changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.