stack 2.1.3 fails to build happy for (at least) lts-3 and lts-8

[phlummox]

General summary/comments

This seems to be a regression, as happy builds for both these LTSs using stack 1.9.3. I note that similar issues have been reported previously (e.g. #5023 for stack 2.1.3, #1427 and #157 for older versions).

I encountered the issue while building haskell-src-meta (which requires haskell-src-exts, which requires happy).

I have tested LTSs 9, 10 and 12 with stack 2.1.3, and happy builds successfully for those. The build might well fail for LTSs 4 through 7, but I have not tested those.

Steps to reproduce

Install stack 2.1.3. (In my case, on Ubuntu 16.04.6.) For lts-3:

1. stack --resolver=lts-3 setup
2. stack --resolver=lts-3 build happy

And similarly for lts-8.

Expected

The build should complete successfully.

Actual

The build fails - stack output (for lts-3) ends with

happy> Linking /tmp/stack1704/happy-1.19.5/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/setup ...
happy> Warning: happy.cabal: Ignoring unknown section type: custom-setup
happy> Configuring happy-1.19.5...
happy> build
happy> Building happy-1.19.5...
happy> Preprocessing executable 'happy' for happy-1.19.5...
happy> setup: The program 'happy' is required but it could not be found
Completed 3 action(s).

--  While building package happy-1.19.5 using:
      /tmp/stack1704/happy-1.19.5/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/setup --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.4.0 build --ghc-options ""
    Process exited with code: ExitFailure 1

Here are Gists with the verbose output from running the "build" step from lts-3 and
lts-8.

Stack version

$ stack --version
Version 2.1.3, Git revision 0fa51b9925decd937e4a993ad90cb686f88fa282 (7739 commits) x86_64 hpack-0.31.2

Method of installation

• Official binary, downloaded from stackage.org or fpcomplete's package repository

[Calvin-L]

This also affects lts-7.x: #5242

[qrilka]

I filed an issue about the general problem with build-tools in Stack as #5245

[phlummox]

I don't think that's a "more general problem", though? A user might want to build and use the happy tool, independent of whether it happens to be in the build-tools stanza of some other package. And currently, despite happy appearing in every stackage LTS, it seems it now can't be built in any LTS prior to 9.

[qrilka]

@phlummox yes, it look you're right and it has not so much to do with the build-tools problem in this case, the line

happy> Warning: happy.cabal: Ignoring unknown section type: custom-setup

looks like it could have something to do with outdated Cabal version and maybe because of that happy doesn't get into the path

[qrilka]

BTW I couldn't reproduce this - stack --resolver=lts-3 build happy went fine on my machine and:

$ stack --resolver=lts-3 exec -- happy --version
Selected resolver: lts-3.22
Happy Version 1.19.5 Copyright (c) 1993-1996 Andy Gill, Simon Marlow (c) 1997-2005 Simon Marlow

Happy is a Yacc for Haskell, and comes with ABSOLUTELY NO WARRANTY.
This program is free software; you can redistribute it and/or modify
it under the terms given in the file 'LICENSE' distributed with
the Happy sources.

[phlummox]

BTW I couldn't reproduce this - stack --resolver=lts-3 build happy went fine on my machine and:

<snipped>

Hi Kirill, unfortunately I don't know the details of the machine you're using, so I'm not able to reproduce your failure to reproduce. Perhaps you already had happy 1.19.5 installed on your machine? Since the command you typed, stack --resolver=lts-3 exec -- happy --version, will not actually attempt to build happy: it will simply attempt to execute an already-existing happy binary. To reproduce the bug I've described, you will probably need to make sure that you don't already have any versions of happy already built and somewhere on the $PATH where stack looks for them.

Perhaps this GitHub repo might help you - it contains a .travis.yml config file which does nothing more than carry out the steps I gave above, using two versions of stack (1.9.3 and 2.1.3) and two LTSs (3 and 9), on a fresh Ubuntu 16.04.6 machine. A look at the Travis build seems to suggest that in such an environment, a build of happy from LTS 3.22 does indeed build using stack 1.9.3, but fails using stack 2.1.3. (And if LTS 9.21 is used, then happy builds successfully using either version of stack.)

[Calvin-L]

@phlummox I think that explanation makes sense. When I run stack --resolver=lts-3 exec -- happy --version I get:

Selected resolver: lts-3.22
Preparing to install GHC to an isolated location.
[...]
Installed GHC.    
Executable named happy not found on path: ["/Users/loncaric/.stack/global-project/.stack-work/install/x86_64-osx/91403fae1a43f7fb55b617f25edeb24e1c528cd1f302642966eb448a0a6fe5a7/7.10.2/bin","/Users/loncaric/.stack/snapshots/x86_64-osx/91403fae1a43f7fb55b617f25edeb24e1c528cd1f302642966eb448a0a6fe5a7/7.10.2/bin","/Users/loncaric/.stack/compiler-tools/x86_64-osx/ghc-7.10.2/bin","/Users/loncaric/.stack/programs/x86_64-osx/ghc-7.10.2/bin","/Users/loncaric/.nix-profile/bin","/opt/nix/bin","/Users/loncaric/.opam/default/bin","/Users/loncaric/bin","/Users/loncaric/.cabal/bin","/usr/local/bin","/usr/bin","/bin","/usr/sbin","/sbin","/Library/TeX/texbin","/opt/X11/bin","/usr/local/sbin"]

That suggests you are right: Stack isn't building happy at all, it is only trying to find it somewhere.

The command stack --resolver=lts-3 build happy fails with the error you described.

[phlummox]

@phlummox yes, it look you're right and it has not so much to do with the build-tools problem in this case, the line

happy> Warning: happy.cabal: Ignoring unknown section type: custom-setup

looks like it could have something to do with outdated Cabal version and maybe because of that happy doesn't get into the path

That is the cabal version contained in the LTS, so I assumed stack would be able to use it: it was able to in version 1.9.3, and nothing in the ChangeLog says that LTS 3 is no longer buildable. The ChangeLog entry for stack 2.1.1 does say, under "Behavior changes", that stack "No longer supports Cabal versions older than 1.19.2. This means projects using snapshots earlier than lts-3.0 or nightly-2015-05-05 will no longer build". Since LTS 3 uses cabal 1.22.6.0, I assumed stack would still be able to use it. However, if it can't, and if stack can no longer build all the packages in LTSs 3-8, either, perhaps that needs to be noted as another "behaviour change".

[qrilka]

BTW I couldn't reproduce this - stack --resolver=lts-3 build happy went fine on my machine and:

Hi Kirill, unfortunately I don't know the details of the machine you're using, so I'm not able to reproduce your failure to reproduce. Perhaps you already had happy 1.19.5 installed on your machine? Since the command you typed, stack --resolver=lts-3 exec -- happy --version, will not actually attempt to build happy: it will simply attempt to execute an already-existing happy binary.

@phlummox I was just showing you the end result after successful build here is the full build log - https://gist.github.com/qrilka/f308eb26e90ef0125e89365b6453803a

Probably verbose build log of your failure could give some hint, could you share it?

[phlummox]

The verbose output is in the Gist, linked to in the original report.

[qrilka]

It looks like https://github.com/simonmar/happy/blob/1.19.5/Setup.lhs for some reason wants to have happy already in the $PATH when building the executable but it's not clear why it doesn't happen with LTS-9...

[phlummox]

I encountered this again while trying to build the lts-3 snapshot version of Agda: verbose output here.

[phlummox]

Interestingly, a Travis build of that version of Agda on MacOS, again using stack 2.1.3 and lts-3 as a resolver, seemed to complete with no problem. Verbose output here.

[qrilka]

a possible explanation could be that Travis had some global version of happy installed

[phlummox]

Indeed. However, in this case I know that wasn't possible; hence my mentioning it. But for the removal of any doubt, here is the output from a new Travis build, with which happy and stack --resolver lts-3 --no-terminal --install-ghc exec which -- happy run beforehand, to show that happy can't be found on the path.

Start of output is:

++: check whether happy is already installed
++which happy
++stack --resolver lts-3 --no-terminal --install-ghc exec which -- happy
�[0mSelected resolver: lts-3.22�[0m
++stack --resolver lts-3 --no-terminal --install-ghc build Agda-2.4.2.4
�[0mSelected resolver: lts-3.22�[0m
�[0mdata-hash           > configure�[0m
�[0mdata-hash           > Configuring data-hash-0.2.0.1...�[0m
�[0mdata-hash           > build�[0m

End of output is:

�[0mAgda                > Installing library in�[0m
�[0mAgda                > /Users/travis/.stack/snapshots/x86_64-osx/06120eb9d31e57f15f48108558480a7c218ed52bb26918f3ef97d5de32c04ce3/7.10.2/lib/x86_64-osx-ghc-7.10.2/Agda-2.4.2.4-1MbAH76zxozCAjaUT7D1kn�[0m
�[0mAgda                > Installing executable(s) in�[0m
�[0mAgda                > /Users/travis/.stack/snapshots/x86_64-osx/06120eb9d31e57f15f48108558480a7c218ed52bb26918f3ef97d5de32c04ce3/7.10.2/bin�[0m
�[0mAgda                > Registering Agda-2.4.2.4...�[0m
�[0mCompleted 32 action(s).�[0m

And in between, happy gets successfully built (refer to the gist).

Stack "--version" output is:

++stack --version
Version 2.1.3, Git revision 0fa51b9925decd937e4a993ad90cb686f88fa282 (7739 commits) x86_64 hpack-0.31.2

Stack was installed with:

curl --insecure -L "https://github.com/commercialhaskell/stack/releases/download/v2.1.3/stack-2.1.3-osx-x86_64.tar.gz" | tar xz --strip-components=1 --include '*/stack' -C ~/.local/bin

*edited to add: I see I didn't pass the flag for verbose output. I'll do that now and give a Gist with the verbose output.

[mpilgrem]

I am closing given the passage of time. The master branch version of Stack no longer supports LTS < 12.0 and (on Windows):

stack --snapshot lts-12.0 build happy

works as expected.