custom-setup stanza ignored when building with cabal-install-2.0.0.1 and ghc-8.2.2

[erikd]

Trivial project that demonstrates the issue is here: https://github.com/erikd-ambiata/cabal-with-ghc-8.2-error . The project doesn't need any libraries and should just build with cabal build.

The project contains a custom Setup.hs file that requires version 1.24 of the Cabal library and the cabal file includes:

custom-setup
  setup-depends:       base >= 4 && < 5
                     , Cabal == 1.24.*

This project builds fine with cabal-install-2.0.0.1 and ghc-8.0.2, but with ghc-8.2.2, it fails to build because it seems to ignore the custom-setup stanza. It also fails with ghc-8.2.1.

This may of course be a bug in GHC 8.2.

[hvr]

I can't seem to reproduce this, I cloned your repro-repo and did

$ /opt/cabal/bin/cabal-2.0 --version
cabal-install version 2.0.0.1
compiled using version 2.0.1.1 of the Cabal library

$ /opt/cabal/bin/cabal-2.0  install -w ghc-8.2.2 --dry
Resolving dependencies...
In order, the following would be installed (use -v for more details):
process-1.4.3.0 (latest: 1.6.2.0)
Cabal-1.24.2.0 (latest: 2.0.1.1)
cabal-with-ghc82-error-0.1.0.0

$ /opt/cabal/bin/cabal-2.0  new-build -w ghc-8.2.2 --dry
Resolving dependencies...
In order, the following would be built (use -v for more details):
 - process-1.4.3.0 (lib) (requires build)
 - Cabal-1.24.2.0 (lib) (requires build)
 - cabal-with-ghc82-error-0.1.0.0 (lib:cabal-with-ghc82-error) (first run)

IOW both, old-build and new-build, would honor the custom-setup stanza....

PS: I see you mention that you want to used cabal build directly; that however has a fallback in place if you don't have Cabal-1.24 in your package db (because e.g. you didn't run cabal install --dep beforehand to make sure all pre-reqs were preinstalled):

$ /opt/cabal/bin/cabal-2.0  configure -w ghc-8.2.2 -vnormal+nowrap
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
trying: cabal-with-ghc82-error-0.1.0.0 (user goal)
next goal: cabal-with-ghc82-error:setup.Cabal (dependency of cabal-with-ghc82-error-0.1.0.0)
rejecting: cabal-with-ghc82-error:setup.Cabal-2.0.1.0/installed-2.0... (conflict: cabal-with-ghc82-error => cabal-with-ghc82-error:setup.Cabal==1.24.*)
fail (backjumping, conflict set: cabal-with-ghc82-error, cabal-with-ghc82-error:setup.Cabal)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: cabal-with-ghc82-error, cabal-with-ghc82-error:setup.Cabal
Trying configure anyway.

[1 of 1] Compiling Main             ( dist/setup/setup.hs, dist/setup/Main.o )

dist/setup/setup.hs:45:8: error:
    Not in scope: data constructor ‘PackageName’
...

[23Skidoo]

Can confirm that the following works:

$ cabal sandbox init
$ cabal install
Resolving dependencies...
Notice: installing into a sandbox located at
/home/refold/test/cabal-with-ghc-8.2-error/.cabal-sandbox
Configuring process-1.4.3.0...
Building process-1.4.3.0...
Installed process-1.4.3.0
Configuring Cabal-1.24.2.0...
Building Cabal-1.24.2.0...
Installed Cabal-1.24.2.0
Configuring cabal-with-ghc82-error-0.1.0.0...
Building cabal-with-ghc82-error-0.1.0.0...
Installed cabal-with-ghc82-error-0.1.0.0

The issue is most likely the configure hack @hvr mentioned in his P.S. that we still have in order to make the containers test suite work. See #1575 for more context.

[hvr]

@23Skidoo minor braindump: fwiw, I'm starting to think that cycles in the test-suite framework leading back to the package under development are questionable anyway; i.e. if you're hacking on containers, and your testsuite/benchmark suite framework's core logic depends/relies upon the same containers version you're hacking on (possibly introducing bugs, or other weird things), you can't really trust your test/benchmark framework imho. Hence, I'm personally trying to avoid such cycles, and rather go for separate .cabal files (c.f. https://github.com/haskell/text/blob/master/benchmarks/text-benchmarks.cabal) which build the implementation-under-test (IUT) as a distinct package, which also gives the potential benefit that you can compare your IUT to a previous reference version more easily, as you have access to both, the old and the new implementation from within the same program.

[23Skidoo]

@hvr The conclusion in the summary in #1575 is basically similar, the proposed solution there is to treat test suite dependencies similarly to how custom-setup deps are treated, so that the test framework would be compiled against a different version of containers. Though including the IUT modules in other-modules will still be required because a component can't depend on two versions of the same library (without private dependencies). So your solution (cabal.project and a test suite in a separate package) produces almost the same end result.

BTW, containers does stuff like

benchmark lookupge-map
  type: exitcode-stdio-1.0
  hs-source-dirs: benchmarks/LookupGE, .
  [...]
  build-depends:
    base >= 4.2 && < 5,
    containers,
    criterion >= 0.4.0 && < 1.2,
    [...]

which looks a bit confused to me due to the inclusion of both hs-source-dirs: . and build-depends: containers.

[erikd]

Ok, this is weird. This is what I am doing:

> git clone https://github.com/erikd-ambiata/cabal-with-ghc-8.2-error
Cloning into 'cabal-with-ghc-8.2-error'...
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 7 (delta 0), reused 7 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.
 
> cd cabal-with-ghc-8.2-error/

> ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.2.2

> cabal --version
cabal-install version 2.0.0.1
compiled using version 2.0.1.1 of the Cabal library 

> cabal build
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
trying: cabal-with-ghc82-error-0.1.0.0 (user goal)
next goal: cabal-with-ghc82-error:setup.Cabal (dependency of
cabal-with-ghc82-error-0.1.0.0)
rejecting: cabal-with-ghc82-error:setup.Cabal-2.0.1.0/installed-2.0...
(conflict: cabal-with-ghc82-error =>
cabal-with-ghc82-error:setup.Cabal==1.24.*)
fail (backjumping, conflict set: cabal-with-ghc82-error,
cabal-with-ghc82-error:setup.Cabal)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: cabal-with-ghc82-error,
cabal-with-ghc82-error:setup.Cabal
Trying configure anyway.
[1 of 1] Compiling Main             ( dist/setup/setup.hs, dist/setup/Main.o )

dist/setup/setup.hs:45:8: error:
    Not in scope: data constructor ‘PackageName’
    Perhaps you meant one of these:
      variable ‘packageName’ (imported from Distribution.Simple),
      variable ‘unPackageName’ (imported from Distribution.Simple),
      variable ‘mkPackageName’ (imported from Distribution.Simple)

I can't see what is different.

[ezyang]

So, I haven't actually looked at your setup, but the error message you are seeing is generally indicative a Setup.hs that is not ported to work with Cabal 2.0 (which made PackageName abstract so that we could swap it to a packed representation).

[erikd]

@ezyang Yes, but the cabal file has:

custom-setup
  setup-depends:       base >= 4 && < 5
                     , Cabal == 1.24.*

which should force the Setup.hs file to be compiled against version 1.24 of the Cabal library.

The aim here is to have a Setup.hs file which can be built with cabal-install 1.24 and 2.0 with any ghc from 7.10 to 8.2 inclusive.

[23Skidoo]

@erikd The issue you're bumping into is the aforementioned hack in cabal configure where it tries to configure even though the solver couldn't find a solution. I think we can make the warning more prominent and print it after the setup script fails to compile.

[erikd]

@23Skidoo Thanks for the response, but I don't really understand it. Is there something I can do to fix this?

[23Skidoo]

@erikd Try running install --only-dependencies before cabal build.

[erikd]

@23Skidoo Thanks! Yes, that works!

[erikd]

After much difficulty, I finally managed to come up with a version of Setup.hs which works with three versions of GHC (7.10, 8.0 and 8.2) as well as two versions of cabal-install (1.24 and 2.0) and that will work on a clean version of my test repo without running cabal install --only-dependencies.

However, it seems that ghc-8.,4 is going to break this once again. Thanks to @hvr ' s PPA, I can test with ghc-8.4 in Travis now. Unfortunately it failed with both cabal-install-1.24 and cabal-install-2.0.

Is the Cabal library supposed to break working Setup.hs files with every new GHC and cabal-install release?

[ezyang]

No, it's not, but Cabal is afflicted by the same problem as GHC, in that it exposes such a large surface API intended for internal use that was never designed for external use (slash, there was never a process put in place to keep Cabal from continually breaking BC), that seemingly innocuous refactors break Custom setups.

Case in point, the BC-breakage here was caused by me ten months ago in 0aa73b7 because I was trying to get better call stacks when IO actions in Cabal failed (to avoid those mysterious errors where Cabal will just say "permission denied" without saying anything else about what was denied, etc.)

Would you like me to rename the Verbosity-ified version of rewriteFile and restore the one with the old signature? That should fix all of your build failures.

[gbaz]

Should we integrate some similar thing to the test repo into the cabal test suite -- i.e. ensure that there's an example working setup.hs that can be written to be compat with three versions?

[erikd]

@ezyang wrote:

Would you like me to rename the Verbosity-ified version of rewriteFile and restore the one with the old signature? That should fix all of your build failures.

Yes, please!!!!

@gbaz wrote:

Should we integrate some similar thing to the test repo into the cabal test suite

I think that would help. What I have here may be the start of that test, but I'm pretty sure what I'm using here is only a small subset of what others may be using.

Please let me know how I can help.

[dalaing]

I'm currently hacking on something for #4492 that would put Verbosity into a Reader-like monad, which might alleviate some of these kinds of issues in the future. That might have to wait until we're clear to do some crazily big breakage, since it's kind of nuke as far as BC goes.