Reject solver solutions involving un-buildable libraries.

[ezyang]

Now you get something like this:

Resolving dependencies...
cabal: Could not resolve dependencies:
trying: p-0.1 (user goal)
unknown package: unbuildable (dependency of p-0.1)
fail (backjumping, conflict set: p, unbuildable)
Dependency tree exhaustively searched.

The error message is not the best.

Fixes #3978

Signed-off-by: Edward Z. Yang [email protected]

[mention-bot]

@ezyang, thanks for your PR! By analyzing the history of the files in this pull request, we identified @grayjay, @kosmikus and @BardurArantsson to be potential reviewers.

[ezyang]

Solver tests fail but I think that is just a matter of the semantics having changed. I'd appreciate opinions on whether or not this semantics change is right, though.

[grayjay]

I don't think this is the best solution to #3978. The error message isn't accurate, and it won't work with build-tools or top-level executables that have unbuildable libraries.

I can see why we need to solve for components in the dependency solver now. That would fix #3978, and then we would probably want to treat an unbuildable component like a failure, not something to ignore.

If we need to work around the problem with unix before the solver can handle individual components, I have another idea. We could change convGPD to create a failure node if the package has no buildable libraries or executables with the given values of os, arch, etc. Then we could add a new FailReason with a better error message, similar to Broken or Shadowed. It wouldn't work if Buildable is under a flag, though.

unix probably shouldn't use buildable: False, if its only purpose is to prevent cabal from building any part of the package on Windows. build-depends: unbuildable<0 alone would work with cabal before and after 1.24.

I agree that the tests should change. They should test disabling an unbuildable executable instead of a library. I can update them.

[ezyang]

Your suggested alternate fix sounds reasonable. Maybe I can also just fix the assert failure directly in cabal-install's code by handling the case when a package has no components.

[dcoutts]

@grayjay It's not clear to me that solving for components would help here (not to say it's not, I just don't see it).

Another approach that I was thinking of was to keep using the syntactic transformation for executables (so the core solver never sees it, it just gets transformed away), whereas for libraries we would treat it directly. Here's the idea. The buildable: False thing is a special case of a more general user-defined failure. Imagine:

if this(..) & !that(..)
  build-depends: ...
if !this(..) & that(..)
  build-depends: ...
if !this(..) & !that(..)
  build-depends: ...

if this(..) & that(..)
  fail: cannot select this and that, one or the other but not both

Or for the unix example

if os(windows)
  fail: The unix package cannot be built on windows

And then buildable: False (for libraries) is just a special case, with no error supplied.

So this failure might be added directly like other kinds of dependency (pkg, extension, pkg-config etc), except that it's a dependency that is never satisfied, and carries a failure reason (or a built-in failure reason in the case of buildable: False).

So concretely my suggestion was to add this fail dep kind (alongside pkg, extensions etc) and in the index conversion, convert buildable: False in libs into this. Then at some point after the new parser is merged and we probably add the surface syntax for explicit failure, then it'll be easy to integrate with the solver.

[kosmikus]

@dcoutts If I understand @grayjay correctly, then it would be natural for the solver to reject non-buildable components if we'd solve on a per-component basis. But because we solve for packages instead, we currently just disable the non-buildable components yet still consider the package choice valid.

The idea to remove packages that have no buildable components left sounds reasonable to me.

[dcoutts]

disable the non-buildable components yet still consider the package choice valid.

Ok, but this is the correct behaviour for exes (according to the slightly weird behaviour of buildable: False). It's just for libs where it has to be treated as failure. And yeah I guess if a package contained only exes and all were not buildable then that's probably a failure too.

[grayjay]

Maybe I can also just fix the assert failure directly in cabal-install's code by handling the case when a package has no components.

I'm not sure I understand.

@dcoutts If I understand @grayjay correctly, then it would be natural for the solver to reject non-buildable components if we'd solve on a per-component basis. But because we solve for packages instead, we currently just disable the non-buildable components yet still consider the package choice valid.

Exactly.

As I see it, when cabal solves for whole packages, Buildable: False means that a component should be ignored without a failure, and build-depends: unbuildable means that the whole package should fail. If we changed the solver so that packages depended on components instead of packages, then we would know exactly which components of each package need to be build. Buildable: False and build-depends: unbuildable (or user-defined failure) would then have the same meaning. Both could be implemented as an unsatisfiable dependency in the solver.

Ok, but this is the correct behaviour for exes (according to the slightly weird behaviour of buildable: False). It's just for libs where it has to be treated as failure.

I thought that the behavior should be similar for libraries and exes, now that packages can depend on build tools. The solver should try to avoid disabling the exe if another package depends on it.

[grayjay]

I started updating the tests (https://github.com/grayjay/cabal/tree/buildable-solver-tests), but I realized that the solver DSL doesn't handle languages and extensions in executables. I'll have to look into that later.

[dcoutts]

@grayjay ok yes it's possible to fix by looking at things on a component level, but it's also possible to fix at the package level still. I say this because I assume that going to a component level is not going to be a trivial change we can make quickly (as well as likely having performance implications).

As I see it, when cabal solves for whole packages, Buildable: False means that a component should be ignored without a failure

We just have to adjust that logic slightly. We say instead that Buildable: False means an executable (or testsuite or benchmark) component should be ignored without a failure, but for a library it means that the whole package should fail. We can achieve that by transforming Buildable: False away as we do now during the index conversion phase, but only doing that transformation for non-lib components. For the lib components we'd translate Buildable: False into a failure, in much the same way as build-depends: unbuildable generates a failure. My suggestion above about failure dep kinds is just one plausible approach for doing that.

[grayjay]

@dcoutts What about a package that has an unbuildable library that is only needed for an executable because it provides an executable? Shouldn't the package not fail in that case?

[dcoutts]

@grayjay historically the Cabal spec has not supported that, since it works at the package level. The whole unbuildable thing was an ill-thought-out convenience hack that we now have to interpret in vaguely sane ways. I don't think we need to care too much about corner cases of a bad feature that are not in practice used.

[grayjay]

Okay. Do you think we should work towards deprecating the field? I'm not sure it's a corner case now, though.

I just checked hackage, and there are 74 packages and 521 package versions that have at least one buildable: False in the library. I looked at a few of them. Some seem to use the field to cause failure, such as https://hackage.haskell.org/package/happstack-server-7.4.6.2/happstack-server.cabal. However, there are others that seem to use it as an optimization or a way to use flags to control the building of components:
https://hackage.haskell.org/package/xmonad-0.12/xmonad.cabal
https://hackage.haskell.org/package/hledger-web-0.18.2/hledger-web.cabal
https://hackage.haskell.org/package/pandoc-1.9.4.5/pandoc.cabal.

[ezyang]

I don't think it makes sense to deprecate the field until we have per-component solving.

[grayjay]

I was wondering if we might still need Buildable for other things, even if we had per-component solving, such as allowing authors to create flags that control the set of executables that is built when a user installs a package.

[grayjay]

Closing, since #5337 fixed the same issue.