rustc with musl target #71195
Comments
nmattia
added
the
0.kind: packaging request
|
If you are looking at cross-compilating with Although, one major blocker with compiling musl binaries is that |
veprbl
added
6.topic: rust
0.kind: enhancement
|
FWIW, I just ran into this when investigating moving the It'd be convenient if I could just give |
|
I got triggered into working on this, or at least a relevant part of it. I'm making I managed to build static GUI apps with GTK (nh2/static-haskell-nix#50 (comment)), and next want to build static Rust GUI apps. Of course I'll publish/upstream what I currently have in my git tree, but for the impatient and my own record, I needed got get around the I also had to add the bootstrap compiler download to nixpkgs/pkgs/development/compilers/rust/bootstrap.nix Lines 7 to 8 in e0907e2 I'm currently at |
|
Also super relevant are the Alpine package description and its patches: https://git.alpinelinux.org/aports/tree/community/rust?id=351e828ddb447e649acfc38775596d290f92cb05 |
|
Full output of my current blocker: |
|
I've given up on Only in rust/: 1_38_0.nix
Only in rust/: 1_39_0.nix
diff /Users/nicolas/nixpkgs/pkgs/development/compilers/rust/bootstrap.nix rust/bootstrap.nix
1c1
< { stdenv, fetchurl, callPackage, version, hashes }:
---
> { stdenv, fetchurl, callPackage }:
3a4,17
> # Note: the version MUST be one version prior to the version we're
> # building
> version = "1.38.0";
>
> # fetch hashes by running `print-hashes.sh 1.38.0`
> hashes = {
> i686-unknown-linux-gnu = "41aed8a350e24a0cac1444ed99b3dd24a90bc581dd88cb420c6e547d6b5f57af";
> x86_64-unknown-linux-gnu = "adda26b3f0609dbfbdc2019da4a20101879b9db2134fae322a4e863a069ec221";
> armv7-unknown-linux-gnueabihf = "8b1bf1680a61a643d6b5c7a3b1a1ce88448652756395e20ba5846739cbd085c4";
> aarch64-unknown-linux-gnu = "06afd6d525326cea95c3aa658aaa8542eab26f44235565bb16913ac9d12b7bda";
> i686-apple-darwin = "cdbf2807774bed350a3af6f41d7f7dd7ceff28777cde310c3ba90033188eb2f8";
> x86_64-apple-darwin = "bd301b78ddcd5d4553962b115e1dca5436dd3755ed323f86f4485769286a8a5a";
> };
>
diff /Users/nicolas/nixpkgs/pkgs/development/compilers/rust/default.nix rust/default.nix
1,6d0
< { rustcVersion
< , rustcSha256
< , bootstrapVersion
< , bootstrapHashes
< , selectRustPackage
< }:
12c6
< , pkgsBuildTarget, pkgsBuildBuild
---
> , path
19c13,14
< buildRustPackage = callPackage ../../../build-support/rust {
---
> # XXX: we should probably replace this with naersk
> buildRustPackage = callPackage (path + /pkgs/build-support/rust) {
22c17
< fetchcargo = buildPackages.callPackage ../../../build-support/rust/fetchcargo.nix {
---
> fetchcargo = buildPackages.callPackage (path + /pkgs/build-support/rust/fetchcargo.nix) {
45,48c40
< prebuilt = callPackage ./bootstrap.nix {
< version = bootstrapVersion;
< hashes = bootstrapHashes;
< };
---
> prebuilt = callPackage ./bootstrap.nix {};
55c47
< (selectRustPackage buildPackages).packages.prebuilt);
---
> buildPackages.rust.packages.prebuilt);
59c51
< buildRustPackages = (selectRustPackage buildPackages).packages.stable;
---
> buildRustPackages = buildPackages.rust.packages.stable;
63,65d54
< version = rustcVersion;
< sha256 = rustcSha256;
<
70,72c59,61
< pkgsBuildBuild = pkgsBuildBuild // { targetPackages.stdenv = llvmPackages_5.stdenv; };
< pkgsBuildHost = pkgsBuildBuild // { targetPackages.stdenv = llvmPackages_5.stdenv; };
< pkgsBuildTarget = pkgsBuildTarget // { targetPackages.stdenv = llvmPackages_5.stdenv; };
---
> #pkgsBuildBuild = pkgsBuildBuild // { targetPackages.stdenv = llvmPackages_5.stdenv; };
> #pkgsBuildHost = pkgsBuildBuild // { targetPackages.stdenv = llvmPackages_5.stdenv; };
> #pkgsBuildTarget = pkgsBuildTarget // { targetPackages.stdenv = llvmPackages_5.stdenv; };
Common subdirectories: /Users/nicolas/nixpkgs/pkgs/development/compilers/rust/rls and rust/rls
diff /Users/nicolas/nixpkgs/pkgs/development/compilers/rust/rust-src.nix rust/rust-src.nix
9c9
< rm -rf $out/{ci,doc,etc,grammar,llvm-project,llvm-emscripten,rtstartup,rustllvm,test,tools,vendor,stdarch}
---
> rm -rf $out/{ci,doc,driver,etc,grammar,llvm,rt,rtstartup,rustllvm,test,tools,vendor}
diff /Users/nicolas/nixpkgs/pkgs/development/compilers/rust/rustc.nix rust/rustc.nix
1c1,2
< { stdenv, removeReferencesTo, pkgsBuildBuild, pkgsBuildHost, pkgsBuildTarget
---
> { stdenv, removeReferencesTo
> , lib
3c4
< , llvm_9, darwin, git, cmake, rustPlatform
---
> , llvm_8, llvmPackages_8, darwin, git, cmake, rustPlatform
7,8c8,12
< , version
< , sha256
---
> , pkgsStatic
> , pkgs
> , runCommand
> , musl
> , makeWrapper
10a15,23
> # We make some assumptions about the platform below, this is just a sanity
> # check
> assert
> if stdenv.isLinux then
> stdenv.buildPlatform.config == "x86_64-unknown-linux-gnu"
> else
> stdenv.buildPlatform.config == "x86_64-apple-darwin"
> ;
>
11a25,33
> targets =
> [
> "${stdenv.buildPlatform.config}"
> "wasm32-unknown-unknown"
> ]
> # on Linux we also build the musl target for fully static executables
> ++ lib.optionals stdenv.isLinux [ "x86_64-unknown-linux-musl" ]
> ;
> host = stdenv.buildPlatform.config;
15,18d36
< llvmSharedForBuild = pkgsBuildBuild.llvm_9.override { enableSharedLibraries = true; };
< llvmSharedForHost = pkgsBuildHost.llvm_9.override { enableSharedLibraries = true; };
< llvmSharedForTarget = pkgsBuildTarget.llvm_9.override { enableSharedLibraries = true; };
<
20c38
< llvmShared = llvm_9.override { enableSharedLibraries = true; };
---
> llvmShared = llvm_8.override { enableSharedLibraries = true; };
23c41
< inherit version;
---
> version = "1.39.0";
27c45
< inherit sha256;
---
> sha256 = "0mwkc1bnil2cfyf6nglpvbn2y0zfbv44zfhsd5qg4c9rm6vgd8dl";
48c66
< ++ optional stdenv.isDarwin "-rpath ${llvmSharedForHost}/lib";
---
> ++ optional stdenv.isDarwin "-rpath ${llvmShared}/lib";
56,64c74,94
< setBuild = "--set=target.${stdenv.buildPlatform.config}";
< setHost = "--set=target.${stdenv.hostPlatform.config}";
< setTarget = "--set=target.${stdenv.targetPlatform.config}";
< ccForBuild = "${pkgsBuildBuild.targetPackages.stdenv.cc}/bin/${pkgsBuildBuild.targetPackages.stdenv.cc.targetPrefix}cc";
< cxxForBuild = "${pkgsBuildBuild.targetPackages.stdenv.cc}/bin/${pkgsBuildBuild.targetPackages.stdenv.cc.targetPrefix}c++";
< ccForHost = "${pkgsBuildHost.targetPackages.stdenv.cc}/bin/${pkgsBuildHost.targetPackages.stdenv.cc.targetPrefix}cc";
< cxxForHost = "${pkgsBuildHost.targetPackages.stdenv.cc}/bin/${pkgsBuildHost.targetPackages.stdenv.cc.targetPrefix}c++";
< ccForTarget = "${pkgsBuildTarget.targetPackages.stdenv.cc}/bin/${pkgsBuildTarget.targetPackages.stdenv.cc.targetPrefix}cc";
< cxxForTarget = "${pkgsBuildTarget.targetPackages.stdenv.cc}/bin/${pkgsBuildTarget.targetPackages.stdenv.cc.targetPrefix}c++";
---
> ccForBuild = "${stdenv.cc}/bin/${stdenv.cc.targetPrefix}cc";
> cxxForBuild = "${stdenv.cc}/bin/${stdenv.cc.targetPrefix}c++";
>
> # musl-enabled cc, used in linux builds
> ccMusl = "${pkgsStatic.stdenv.cc}/bin/${pkgsStatic.stdenv.cc.targetPrefix}cc";
> cxxMusl = "${pkgsStatic.stdenv.cc}/bin/${pkgsStatic.stdenv.cc.targetPrefix}c++";
> muslRoot =
> # the musl-root requires a libunwind.a, so we provide one from llvm
> # XXX: create ticket about libunwind.a from LLVM breaking because rustc
> # doesn't link-in libstdc++
> let libunwind = pkgsStatic.callPackage ../packages/libunwind.nix
> { enableShared = false; };
> in
> runCommand "musl-root" {}
> ''
> mkdir -p $out
> cp -r ${musl}/* $out
> chmod +w $out/lib
> cp ${libunwind}/lib/libunwind.a $out/lib/libunwind.a
> '';
>
73,86c103
< "--target=${stdenv.targetPlatform.config}"
<
< "${setBuild}.cc=${ccForBuild}"
< "${setHost}.cc=${ccForHost}"
< "${setTarget}.cc=${ccForTarget}"
<
< "${setBuild}.linker=${ccForBuild}"
< "${setHost}.linker=${ccForHost}"
< "${setTarget}.linker=${ccForTarget}"
<
< "${setBuild}.cxx=${cxxForBuild}"
< "${setHost}.cxx=${cxxForHost}"
< "${setTarget}.cxx=${cxxForTarget}"
< ] ++ optional (!withBundledLLVM) [
---
> "--target=${lib.concatStringsSep "," targets}"
88,91c105,117
< "${setBuild}.llvm-config=${llvmSharedForBuild}/bin/llvm-config"
< "${setHost}.llvm-config=${llvmSharedForHost}/bin/llvm-config"
< "${setTarget}.llvm-config=${llvmSharedForTarget}/bin/llvm-config"
< ] ++ optional stdenv.isLinux [
---
> "--set=target.${stdenv.buildPlatform.config}.llvm-config=${llvmShared}/bin/llvm-config"
> ] ++ lib.optionals stdenv.isLinux
> [
> "--set=target.x86_64-unknown-linux-gnu.cc=${ccForBuild}"
> "--set=target.x86_64-unknown-linux-gnu.linker=${ccForBuild}"
> "--set=target.x86_64-unknown-linux-gnu.cxx=${cxxForBuild}"
>
> "--set=target.x86_64-unknown-linux-musl.cc=${ccMusl}"
> "--set=target.x86_64-unknown-linux-musl.linker=${ccMusl}"
> "--set=target.x86_64-unknown-linux-musl.cxx=${cxxMusl}"
> "--set=target.x86_64-unknown-linux-musl.musl-root=${muslRoot}"
>
>
93c119,124
< ];
---
> ] ++ lib.optionals stdenv.isDarwin
> [
> "--set=target.x86_64-apple-darwin.cc=${ccForBuild}"
> "--set=target.x86_64-apple-darwin.linker=${ccForBuild}"
> "--set=target.x86_64-apple-darwin.cxx=${cxxForBuild}"
> ];
125c156
< which libffi removeReferencesTo pkgconfig
---
> which libffi removeReferencesTo pkgconfig makeWrapper
136a168
> # XXX: change path wrapping
138a171,176
>
> # rustdoc needs a cc
> # we wrap it with a new PATH until 1.39:
> # https://github.com/rust-lang/cargo/issues/7529
> wrapProgram $out/bin/rustdoc \
> --prefix PATH ":" "${stdenv.cc}/bin"
diff /Users/nicolas/nixpkgs/pkgs/development/compilers/rust/setup-hook.sh rust/setup-hook.sh
2c2
< if [[ -z ${IN_NIX_SHELL-} && -z ${CARGO_HOME-} ]]; then
---
> if [[ -z $IN_NIX_SHELL && -z $CARGO_HOME ]]; then
with libunwind: # We need libunwind-9.0.0 because earlier versions link c++ symbols we don't
# have to have to link to
{ fetchurl, stdenv, cmake, enableShared ? true }:
let
version = "9.0.0";
fetch = sha256: fetchurl {
url = "https://releases.llvm.org/${version}/libunwind-${version}.src.tar.xz";
inherit sha256;
};
in
stdenv.mkDerivation rec {
pname = "libunwind";
inherit version;
src = fetch "1chd1nz4bscrs6qa7p8sqgk5df86ll0frv0f451vhks2w44qsslp";
nativeBuildInputs = [ cmake ];
enableParallelBuilding = true;
cmakeFlags = stdenv.lib.optional (!enableShared) "-DLIBUNWIND_ENABLE_SHARED=OFF";
}Hope that helps, let me know if you have any questions. |
|
Still struggling with the includ error. |
|
I can't figure out how to add stuff to the end of the search path. Even my |
|
I got it to build.
My mistake was to add I've pushed my WIP to nh2@70e7b40 (part of my hacky |
|
Next obstacle (when building Which is probably rust-lang/rust#40174. |
|
@nh2 that's because you're cross compiling rustc itself, which means that the proc macro crates (TH) need to be compiled for that target as well, though unfortunately (AFAIR) they must be dynamically linked (like GHC by default only loading |
@nmattia But I'm not cross compiling (using none of the cross infrastructure), I'm just compiling with musl as my libc. (I've also clarified with an edit that I have finished building My understanding of the situation from the linked issue rust-lang/rust#40174 is that when the musl I think the Alpine patches in https://git.alpinelinux.org/aports/tree/community/rust?id=351e828ddb447e649acfc38775596d290f92cb05 handle this, also by introducing a custom targets (e.g. |
|
I have commented upstream: rust-lang/rust#58575 (comment) |
Apparently this is fixed in 1.44.0-beta: rust-lang/rust#40174 (comment) |
|
Hello, I'm a bot and I thank you in the name of the community for opening this issue. To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human. The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it. If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them. Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel. |
stale
bot
added
the
2.status: stale
|
Still important to me - would love to get this in nixpkgs! |
stale
bot
removed
the
2.status: stale
|
I was able to build rustc 1.47 with musl target without any modifications. I tried to build fd and got a result, but it instantly segfaults on start: |
|
Oh! It works fine when I use rustc 1.45, because it the issue described above is #94228. So we already have a rustc compiler capable of building for musl in nixpkgs, and it is even in the binary cache. Maybe we should add a line to the static overlay to set the default rust version to 1.45 when building pkgsStatic.*. |
|
This works (using Rust 1.45): And this works as well (using Rust 1.47, but using a different, I guess unwrapped, linker): |
|
I would rather not pin to 1.45. You're on the right track I think, using |
|
@petabyteboy do you have any code example on how to use that to make |
|
Yes. The examples above actually use buildRustPackage since the And yes, rustc for the musl target is not in the binary cache, so you'll have to build that locally. By the way: Under the hood pkgsStatic evaluates nixpkgs with a crossSystem parameter. This should be mostly equivalent: |
|
@petabyteboy thank you very much!! I've tried with your setup, I get this error: I think I am at As for nixpkgs-mozilla, I'll try open issue there. |
|
There's no need to override the rust compiler or to set the target/TARGET variable in the build environment when you have crossSystem. Also no need to explicitly use prev.pkgsStatic anywhere. All buildInputs will be automatically cross-compiled as static libraries as well. If you still need to change something about libraries to make them compile statically, you can add this to a crossOverlay. |
|
The issue you are seeing is with the new Rust 1.48 that was merged with staging-next this week. |
|
@petabyteboy thank you! so I have use Now I'm still building the package again (compiling I'll let you know the progress. |
|
Try Rust 1.47 from nixpkgs 24eb3f8 |
|
You can find another example here: https://github.com/petabyteboy/chaostomato/blob/main/default.nix |
@petabyteboy using rust 1.47 works, thanks. Though I haven't used it to build |
Problem: Rust doesn't work well with musl on nix, see NixOS/nixpkgs#71195. Solution: Build dynamic binaries instead. It's fine since these binaries aren't used in the releases and used only in our internal infrastructure.
Problem: Rust doesn't work well with musl on nix, see NixOS/nixpkgs#71195. Solution: Build dynamic binaries instead. It's fine since these binaries aren't used in the releases and used only in our internal infrastructure.
Problem: Rust doesn't work well with musl, see NixOS/nixpkgs#71195. Solution: Provide dynamically linked binaries instead. This should be fine since we don't use them in the releases and they're mostly used in our internal infrastructure.
Problem: Rust doesn't work well with musl, see NixOS/nixpkgs#71195. Solution: Provide dynamically linked binaries instead. This should be fine since we don't use them in the releases and they're mostly used in our internal infrastructure.
Problem: Rust doesn't work well with musl, see NixOS/nixpkgs#71195. Solution: Provide dynamically linked binaries instead. This should be fine since we don't use them in the releases and they're mostly used in our internal infrastructure.
Problem: Rust doesn't work well with musl, see NixOS/nixpkgs#71195. Solution: Provide dynamically linked binaries instead. This should be fine since we don't use them in the releases and they're mostly used in our internal infrastructure.
Problem: Rust doesn't work well with musl, see NixOS/nixpkgs#71195. Solution: Provide dynamically linked binaries instead. This should be fine since we don't use them in the releases and they're mostly used in our internal infrastructure.
Problem: Rust doesn't work well with musl, see NixOS/nixpkgs#71195. Solution: Provide dynamically linked binaries instead. This should be fine since we don't use them in the releases and they're mostly used in our internal infrastructure.
|
It’s unclear from the comments if this now simply works, and could be closed? |
|
(Whoops, commented on the non-rust issue first. Deleting that in favor of this) I don't think this works, at this time. I would love to see this just work, but for others needing a workaround, this is cribbed together from https://www.reddit.com/r/NixOS/comments/f0yi3b/how_to_build_a_simple_static_rust_binary_using/ and here and works for me on NixOS 20.09. I have a feeling the mozilla overlay is not needed, but I don't know how to do the |
|
I haven't read all the thread, but I sucessfully cross-compiled a x86_64 linux musl executable with crate2nix. |
Project description
I would love to have a rustc version that compiles static binaries on Linux.
I've hacked one together:
The idea is that I really, really don't want to go through the
pkgsCrossinfrastructure as this involves using a regular rustc (pkgsBuild) but then build two static rustcs (pkgsHost + pkgsTarget) when using pkgsStatic (not even sure this is possible). Instead I'd rather just instruct the rustc build system to build an extra target (i.e. stdlib) targetting musl. The above works well (modulo thewithBundledLLVMwhich I've always set totrueso far).Would love to hear your thoughts @dtzWill @matthewbauer .
The text was updated successfully, but these errors were encountered: