build Setup.hs with local compiler when cross-compiling

[rwbarton]

I have a script that invokes cabal with --with-ghc=..., --with-ghc-pkg=..., --with-gcc=..., etc. for cross-compiling to Android. It works great for most packages, but fails for a package with Build-type: Custom, because it builds Setup.hs with the cross-compiler specified on the command-line, and then obviously it can't run dist/setup/setup because it's an ARM executable.

I can work around the issue by doing an ordinary cabal configure with no --with-ghc=... options first, then doing a cabal install with my cross-compiler script, because the second invocation uses the cached dist/setup/setup from the first one. But that's a hack, obviously, and annoying to do when the package I'm installing is a dependency of the one I actually want.

What I would like is a separate set of options for specifying the program configuration to use to build Setup.hs; or at least an option to tell cabal to just use the ones on my path. I found that using defaultSetupScriptOptions for setupScriptOptions in configure and performInstallations seemed to do the trick for me, to give an idea of what I want to happen, though I haven't tested the resulting libraries. I couldn't determine whether there is already a way to achieve the same effect through command-line options.

[23Skidoo]

What would be an appropriate name for such an option? --host-compiler?

[rwbarton]

I was imagining a series of options like --with-host-compiler, --with-host-PROGRAM to specify the compiler and program configuration to use for building Setup.hs. (And possibly in the future for building modules to be loaded by Template Haskell?)

[rwbarton]

Apparently the Haskell-iPhone people are working around this by patching packages to use build-type: Simple. See the thread http://www.haskell.org/pipermail/iphone/2012-December/000180.html and some of the patches at https://github.com/kaoskorobase/ghc-ios-cabal-scripts/tree/master/patches, like lifted-base and syb.

[singpolyma]

Yeah, this is also a pain point for me. I work around it by just running ghc Setup.hs and copying the result over the one produced by cabal and re-running, but it's a pain.

[acw]

Another possibility, which seems to be sufficient for most cases, would be to allow "build-type" to exist within "if" branches, so that, for example:

if os(HaLVM) || os(iOS)
Build-Type: Simple
else
Build-Type: Custom

I was considering looking into a patch to do this, which seemed like it might be simpler, as long as the if cases were restricted to non-computed things (like os()).

[23Skidoo]

if os(HaLVM) || os(iOS)
Build-Type: Simple
else
Build-Type: Custom

This is a bad idea IMO, since it requires all package authors to modify their .cabal files.

[sheyll]

I think that rwbarton is right with his idea.

What about adding flags along the line of '--with-build-ghc' --with-target-ghc?
We could let '--with-ghc' be a shorthand for passing both '--with-build-ghc' and '--with-target-ghc'
with the same value.
We could even make it illegal to pass a combination of old and new flags without breaking compatibility.

Additionally we could allow for additional config file entries.

Is anyone working on this? Otherwise I will go ahead and try to implement a proposal...

[23Skidoo]

@sheyll
To my knowledge, no-one is working on this.

I think we only need a --with-build-PROGRAM flag for all built-in programs (maybe only GHC initially). By default, Cabal will use the GHC specified with -w for everything, but if --with-build-ghc is provided, it will compile Setup.hs with it. Ditto for other programs.

I imagine that we would have a new section in the config file for this stuff, akin to program-locations:

cross-compilation
    build-ghc-location: /path/to/build-ghc
    build-gcc-location: /path/to/build-gcc
    [...]

[sheyll]

@23Skidoo Ok it seems to be consensus that additional flags are useful. I will simply start with "--with-build-PROGRAM" for ghc and ghc-pkg and we'll see how it flows from there.

[sheyll]

Ok .. I now have a draft for code review, located here:
sheyll@06c2501

[jms19]

Thanks sheyll. This is great stuff and just what I need right now to take the pain out of cross building packages (PowerPC as well as ARM). Though I don't feel qualified to comment on your changes in detail will take them for a spin soon.

[sheyll]

hey wait

this was only a prototype, the real thing is in the making.. I do not
guarantee it works. I am currently rewriting it using a different approach
and also with tests etc..

would you mind making me some kind docker recipe for your cross compiling
setup so I can test that locally?

wbr Sven
Am 14.11.2014 12:28 schrieb "jms19" [email protected]:

Thanks sheyll. This is great stuff and just what I need right now to take
the pain out of cross building packages (PowerPC as well as ARM). Though I
don't feel qualified to comment on your changes in detail will take them
for a spin soon.

—
Reply to this email directly or view it on GitHub
#1493 (comment).

[jms19]

I should have said I am very new to Haskell, even more so to cabal (a week or two) and the term docker has me reaching for Google. I am off the ground with cross compilers though and have gained some knowledge that might be used to improve https://ghc.haskell.org/trac/ghc/wiki/CrossCompilation and friends.

[sheyll]

I just wanted to let you know I continue to work on this now,

Do you think this issue could be assigned to me, to indicate that work is going on there?

I have done a fork of this repo a while ago, but I rebase it regularily, so it should not be too painful to merge when it is done.

I must admit I am surprised by the sophistication of the cabal internals, and coming from industry rather than academia I appreciate the pragmatic approaches chosen, although I have struggled with the fact that I had to touch so many different places for the intended changes; but this might as well have todo with me having the urge to make the changes as test-covered as possible and follow the boy-scout-rule.

I'd rather take some more time than leave a mess that just makes other maybe more important changes more expensive

[23Skidoo]

Do you think this issue could be assigned to me, to indicate that work is going on there?

It looks like you need to be a member of the Haskell organization on GitHub for me to be able to do that.

[jarlg]

@sheyll any progress lately? I would be willing to test it out if you have something working.

I want to cross-compile a cabal sandbox on a x86_64 server for use on an arm laptop, so I need Setup.hs compiled with the host compiler, then ./setup configure run with the (target) arm cross-compiler. Is the proposal to have this look like cabal install -w arm-ghc --with-build-ghc=ghc package? I feel cabal install --with-target-ghc=arm-ghc package would be more natural in my case.

I understand my setup isn't widely adopted, but I thought I'd mention it.

[sheyll]

I will make an experimental version on the fork I have in my github account, and notify you here.
Maybe what's there works, but I have not test-covered or implemented all affected parts yet, go ahead and try.
To answer your question: yes.
Oh and happy new year.

[acw]

Hey, what's the status on this? Is there anything we can do to accelerate this getting into mainline?

[sheyll]

hi
I will look into it in the next couple of days, I have a fork on github
that contains the current state if anyone is interested.
I remember that I added some unit test and todo comments to be able to pick
up the work where I had to stop.
Basically it was working, but the design approach taken did not feel too
good.

Wbr Sven
Am 09.05.2015 02:13 schrieb "Adam Wick" [email protected]:

Hey, what's the status on this? Is there anything we can do to accelerate
this getting into mainline?

—
Reply to this email directly or view it on GitHub
#1493 (comment).

[Ericson2314]

@ezyang heh I actually missed the link to that patch reading through, I was just commenting on the issue in principle. Your plan there looks very good to me: "mixed GHC" plans should hopefully just a be a further decoupling of setup depends on normal depends solving.

[acw]

Merged! Thanks!

[Ericson2314]

Ok, to do this the Right Way™, I think I want to change the type of qualifiers. Note the comment on setup about nesting. Well, this applies equally well to exe deps too, and because packages can constrain on exe version, when cross compiling or bootstrapping this matters.

[expansio ad absurdum: Let's say somebody wants to bootstrap across 5 platforms (and platform-specific compilers), oh my! A build-tool-depends or setup exe running on platform n is built on platform n-1. Let's also say a bunch of packages have various platform-conditional deps/flags/whatever. Well, we need to keep track of have many exe deps deep we are to know which compiler+platform to use. They must be configured separately!]

Thus, I propose instead

type Qualifier = [QualifyingEdge]

data QualifierEdge
  = QualBase PackageName
  | QualExe { from :: PackageName, to :: QualExeTarget }

-- | May just use `Maybe PackageName` in practice
data QualExeTarget = TargetExe PackageName | TargetOwnSetup

Now, there is also a good argument in that comment about the search space growing too large. Well, I propose that after we build the deps we "prune" all qualifier paths accordingly. There are two options I see for this pruning. Let n be the number of compilers/platforms passed in, and m be the number of exe edges in the qualifier path. Then the options are:

• filter all but the first n QualExes, leaving a shorter list
• take the last or first element (whatever matches today) along with min n m saying which compiler+platform to resolve against.

How does this sound?

[ezyang]

For reference, the commits that reverted infinite qualifiers are 37978a6 (fix) and 3d2ad8e (test). I am not entirely sure why @edsko decided to change the package path structure, as oppose to teach the solver how to prune things, but I think that if you are working on this patch, it is more likely to work if you keep the qualifiers finite, because it is not known what changes you would have to make to the solver to prune qualifier paths.

So, sure: in principle, someone might cross compile over five platforms. But are they actually going to in practice? How would they even say that they wanted to cross compile over five platforms?

[Ericson2314]

@ezyang well in practice there would probably only be two, but I think the code is clearer using a list and supporting arbitrary.

[Ericson2314]

Also BTW in https://github.com/Ericson2314/cabal/tree/cross-solve I have that change to qualifier, with prune just doing at most one exe/setup dep because there is no cross support yet.

CC @grayjay

[grayjay]

I just took a quick look, and I think it looks similar to the qualifiers that cabal had before #3220. So I would want to make sure that a new implementation handles the issues that led to #3220 before increasing the depth. Other than that, I think that the structure of the qualifiers looks good.

How would the solver know which compiler to use for each executable? Would it always be specified by the user?

[Ericson2314]

@grayjay the difference is the prune function---it limits the number of QualExes and there can already be only one QualBase, thus the length of the path must be finite. [I forgot to use prune too on user-specified constraints, but those are also finite and worst-case just wouldn't apply, so less harm there.]

How would the solver know which compiler to use for each executable? Would it always be specified by the user?

Specifying a single compiler/platform, or single cross and native compiler/platform, generalizes into specifying a list of compilers/platforms----whatever the user-facing interface would be, this is how things would be stored internally. Then, simply take the number of QualExes in the qualifier, and that is the index for compilers/platforms.

FWIW I am doing some very similar changes with nixpkgs atm, and documenting the new world in https://github.com/NixOS/nixpkgs/blob/master/doc/cross-compilation.xml . That may of interest, but probably not yet.

[grayjay]

@grayjay the difference is the prune function---it limits the number of QualExes and there can already be only one QualBase, thus the length of the path must be finite. [I forgot to use prune too on user-specified constraints, but those are also finite and worst-case just wouldn't apply, so less harm there.]

I meant that we should make sure that the solver continues to handle cycles well before increasing the pruning depth beyond 0. I don't know if this will be an issue, but increasing the depth might cause the solver to search a large tree of possibilities every time it encounters a cycle through setup dependencies, when it could backtrack quickly with a depth of 0.

Specifying a single compiler/platform, or single cross and native compiler/platform, generalizes into specifying a list of compilers/platforms----whatever the user-facing interface would be, this is how things would be stored internally. Then, simply take the number of QualExes in the qualifier, and that is the index for compilers/platforms.

So the solver never has to choose the compiler. Does this feature mean that the solver also needs to keep track of multiple compilers' package databases?

FWIW I am doing some very similar changes with nixpkgs atm, and documenting the new world in https://github.com/NixOS/nixpkgs/blob/master/doc/cross-compilation.xml . That may of interest, but probably not yet.

👍

[Ericson2314]

@grayjay Granted, I haven't looked to much at the solver code, but as I understand it, cycles would be handled the same, and qualifiers essentially would unroll the cycle up to the length of the qualifier, with the longest qualifier in the series handling the remaining cycle. I hope that makes sense :).

So the solver never has to choose the compiler. Does this feature mean that the solver also needs to keep track of multiple compilers' package databases?

In the new-build world, "improvement" i.e. dropping already-built packages/components from the plan takes place after solving. I think this makes stuff not so bad. But yes the compiler/platform, rather than being an input, is now a function of the number of QualExes in the qualifier.

[Ericson2314]

@grayjay Would it be useful for me to bother rebasing that prior to your upcoming solver overhaul?

[grayjay]

@Ericson2314 Do you mean that you would rebase your cross-solve branch so that it could be merged as part of #4087? I think that the current design for #4087 doesn't affect package qualifiers, so you could change the qualifier types independently. #4087 would just add more information about components that we could use to make the qualifier edges more specific.

[Ericson2314]

@grayjay makes sense. Thanks for the feedback.

[acw]

Hi Folks -

Any news on this one? In progress? Blocked on something? (And if so what?) I find myself working on a HaLVM issue, and am wondering if I should choose the Right Way, which relies on this issue being solved in the near future, or if I should do something fairly evil, under the assumption that this issue isn't going to be solved for awhile.

I'm happy to help, if it's just a matter of someone piping data through to the right places, but it's been a long time since I've looked at Cabal internals. Alternatively, I do have a test cross compiler all ready to go, if you want me to run some experiments. 😃

[Ericson2314]

I was sidetracked with other Cabal work (sub library refactors), and then Nixpkgs cross compilation work. I'd probably be best unwinding that stack before I got back to this. Not sure what the status of @grayjay's per-component solving is---it will be easiest to work when that's finished but there's things to do in the meantime.

My plan is more expensive, but solves this the Really Right Way. If you want to write nice cabal files but get by with a hacky Cabal implementation, the patch earlier in the thread is probably still similar to what would be needed for a stop-gap.

[x1ddos]

Instead of adding or extending existing options, why not make cabal just ignore --with-xxx options during configure phase, when compiling custom setup? It has to always run on host either way, right.

At least as a quick short term solution. It isn't usable as is at the moment anyway.

[x1ddos]

Actually, that won't work either. Please, ignore.

[howtonotwin]

Has there been any further progress on this?

[Ericson2314]

It's still on my radar, but no direct work yet.