aarch64-multiplatform-musl: llc' failed in phase LLVM Compiler'

[KtorZ]

Hi there,

I am trying to cross-compile and statically build my project from linux/amd64 to linux/arm64, with musl-lib and GHC-8.10.7. If that is even a thing. A bit by chance, I did found and try the aarch64-multiplatform-musl platform target from pkgsCross which sounded like what I want (see default.nix below).

The build started and went through for a while -- seemed like I've been trying to rebuild GHC and eventually failed:

$ nix-build -A platform.arm64
[...]
`llc' failed in phase `LLVM Compiler'. (Exit code: -9)
make[1]: *** [compiler/ghc.mk:308: compiler/stage2/build/GHC/Hs/Instances.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:128: all] Error 2
error: builder for '/nix/store/im11v299bp8pjdnpywgkcix1fvfjn5pr-aarch64-unknown-linux-musl-ghc-8.10.7.drv' failed with exit code 2
error: 1 dependencies of derivation '/nix/store/nhcm5c63xzbk7l2bzrxnwddp2armdx62-dummy-data-aarch64-unknown-linux-musl-ghc-8.10.7.drv' failed to build
error: 1 dependencies of derivation '/nix/store/6fmqnjwpfc670w5fbg1r7yw32m5g78a6-dummy-aarch64-unknown-linux-musl-ghc-8.10.7.drv' failed to build
error: 1 dependencies of derivation '/nix/store/9dg2lxn7iac28p62cbcnghs5y0shvj5j-haskell-project-plan-to-nix-pkgs.drv' failed to build

Why would this attempt to build GHC with musl -- and not only my project and its dependencies? What am I doing wrong / missing 🤔 ? Should I abandon all hope to cross-compile (w/ static linking) to aarch64?

default.nix

{ compiler ? "ghc8107"
, system ? builtins.currentSystem
, haskellNix ? import
    (builtins.fetchTarball
      "https://github.com/input-output-hk/haskell.nix/archive/974a61451bb1d41b32090eb51efd7ada026d16d9.tar.gz")
    { }
, iohkNix ? import
    (builtins.fetchTarball
      "https://github.com/input-output-hk/iohk-nix/archive/edb2d2df2ebe42bbdf03a0711115cf6213c9d366.tar.gz")
    { }
, nixpkgsSrc ? haskellNix.sources.nixpkgs-unstable
, nixpkgsArgs ? haskellNix.nixpkgsArgs
}:
let
  pkgs = import nixpkgsSrc (nixpkgsArgs // {
    overlays = haskellNix.overlays ++ iohkNix.overlays.crypto;
  });

  mkProject = arch:
    ( arch.haskell-nix.project {
        compiler-nix-name = compiler;
        projectFileName = "cabal.project";
        src = arch.haskell-nix.haskellLib.cleanSourceWith {
          name = "ogmios-src";
          src = ./.;
          subDir = "server";
          filter = path: type:
            builtins.all (x: x) [
              (baseNameOf path != "package.yaml")
            ];
        };
      }
    ).ogmios.components.exes.ogmios;
in {
  platform = {
    arm64 = (mkProject pkgs.pkgsCross.aarch64-multiplatform-musl);
    amd64 = (mkProject pkgs.pkgsCross.musl64);
  };
}

[L-as]

Maybe you want to use GHC 9.2.4 if you want to compile for aarch64-linux.

[KtorZ]

Well, I am not sure that the project and dependencies will compile with GHC 9+; Haven't tried though. How will that help? Was there any work being done regarding arm support since 9.2.4?

[L-as]

There are PRs for getting stuff to work on 9.2.4. See IntersectMBO/cardano-node#4256

[KtorZ]

Thanks for the link. Though coming back to the original question, is there any reason to believe that 9.2.4 would work better for cross-compilation to arm64?

[reqlez]

I know 9.2.4 has better aarch64 support.

Also, ARM support in GHC is not first class citizen yet, you need LLVM to build for example, where you don't need LLVM on AMD64.

"LLVM Compiler" error here i see from your above output, do you have LLVM-12 in the build process?

[KtorZ]

@reqlez what do you mean by "in the build process"?

Is it too naive to think that haskell.nix's project utility would take care of bringing in the right dependencies in scope of the Nix build, including llvm?

I have llvm installed on that system, the exact version I'd have to check. Though it's installed through homebrew, not Nix and both don't usually interoperate with one another so I wouldn't expect Nix to pick up any llvm installation already present on the system.

[reqlez]

Well, i don't know nix ... but i do know, that GHC NEEDS llvm under aarch64 to build, and does not need it under amd64.

Our Ogmios FreeBSD port builds under aarch64 for example, but it pulls llvm-12 for the process.

[KtorZ]

@reqlez note that I have no problem building natively on aarch64 (which indeed requires LLVM for a few Cardano tools); however, I was hoping to give a try to the cross-compilation framework from Haskell.nix to avoid having to build on aarch64 machines.

[reqlez]

@reqlez note that I have no problem building natively on aarch64 (which indeed requires LLVM for a few Cardano tools); however, I was hoping to give a try to the cross-compilation framework from Haskell.nix to avoid having to build on aarch64 machines.

Ah, I see! Well, cannot help with that, sorry! NIX is not my cup of tea.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.