stack build should check whether git is available

[jwaldmann]

I was running stack build on a (virtual) machine, with some github locations in stack.yaml. I got errors like

 .stack-work/downloaded/Lcy0nQgWISfb/: getDirectoryContents: does not exist (No such file or directory)

The reason is that my machine did not have git installed.

Looks like stack called git, but did not check the exit (error) code.

[sjakobi]

I'm under the impression that there were more problems here:

1. Distro packages of stack ought to have a package dependency on git. Did you install stack through a package manager, @jwaldmann, and what operating system were you running on that VM?
2. stack ought to have failed already when the git clone command failed.

[jwaldmann]

This is on a gitlab builder, usind a docker image for haskell-7.10.3. Here is a typical build that works, but still produces a lot of noise: https://gitlab.imn.htwk-leipzig.de/waldmann/pure-matchbox/builds/931

[Blaisorblade]

We should probably move checkDependency stuff from src/Stack/Setup.hs and use it for this.

[Blaisorblade]

Alternatively, there's isGitInstalled in Stack.PackageIndex, some consolidation might be in order.

[Blaisorblade]

So... at least on OS X, this seems to also relate to System.Process.Read and its different ways of launching processes and handling exceptions. Also, this is not about the exit code—that's handled correctly. The problem is specific to git not existing. There's some unclear overlap with #2489.

BTW, using isGitInstalled would probably hide the bug, but I'd rather learn where the failure is being ignored and fix that.

Experiment:

$ cat > ~/.local/bin/git << EOF
> #!/bin/sh
> exit 127
> EOF
$ chmod +x ~/.local/bin/git
$ stack build --dry-run
Running /Users/pgiarrusso/.local/bin/git clone --recursive https://github.com/agda/agda /Users/pgiarrusso/AeroFS/Repos/ilc-agda-bluevelvet/.stack-work/downloaded/kZ1ACjtmNMB9 in directory /Users/pgiarrusso/AeroFS/Repos/ilc-agda-bluevelvet/.stack-work/downloaded/ exited with ExitFailure 127

A problem is, in some functions different exceptions are thrown (unlike documented) for when the executable is missing and for when launching it fails.

[Blaisorblade]

After lots of confusing investigation on master sources, this appears already fixed on master. Below are some logs showing this on Linux (where I could still move away git easily).

$ stack build --dry-run
Executable named git not found on path: ["/home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin","/home/pgiarrusso/.local/bin","/home/pgiarrusso/bin","/home/pgiarrusso/bin","/usr/local/sbin","/usr/local/bin","/usr/sbin","/usr/bin","/sbin","/bin","/usr/games","/usr/local/games","/usr/lib/jvm/java-8-oracle/bin","/usr/lib/jvm/java-8-oracle/db/bin","/usr/lib/jvm/java-8-oracle/jre/bin","/usr/lib/jvm/java-8-oracle/bin","/usr/lib/jvm/java-8-oracle/db/bin","/usr/lib/jvm/java-8-oracle/jre/bin"]

pgiarrusso@kamino:~/git/ilc-agda$ stack build --dry-run -v
Version 1.1.3 x86_64 hpack-0.14.0
2016-08-14 13:56:15.574738: [debug] Checking for project config at: /home/pgiarrusso/git/ilc-agda/stack.yaml
@(Stack/Config.hs:828:9)
2016-08-14 13:56:15.575023: [debug] Loading project config file stack.yaml
@(Stack/Config.hs:846:13)
2016-08-14 13:56:15.576570: [debug] Checking whether stack was built with libgmp4
@(Stack/Config.hs:332:5)
2016-08-14 13:56:15.598448: [debug] Run process: /usr/bin/ldd /home/pgiarrusso/.local/bin/stack                                                                                 [0/175]
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.607448: [debug] Process finished in 8 ms: /usr/bin/ldd /home/pgiarrusso/.local/bin/stack
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.607564: [debug] Stack was not built with libgmp4
@(Stack/Config.hs:336:14)
2016-08-14 13:56:15.607663: [debug] Trying to decode /home/pgiarrusso/.stack/build-plan-cache/x86_64-linux/lts-6.6.cache
@(Data/Store/VersionTagged.hs:68:5)
2016-08-14 13:56:15.618493: [debug] Success decoding /home/pgiarrusso/.stack/build-plan-cache/x86_64-linux/lts-6.6.cache
@(Data/Store/VersionTagged.hs:72:13)
2016-08-14 13:56:15.622432: [debug] Getting system compiler version
@(Stack/Setup.hs:350:17)
2016-08-14 13:56:15.622890: [debug] Run process: /usr/bin/ghc --info
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.639144: [debug] Process finished in 16 ms: /usr/bin/ghc --info
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.641713: [debug] Getting global package database location
@(Stack/GhcPkg.hs:53:5)
2016-08-14 13:56:15.641921: [debug] Run process: /home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.642462: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:101:13)
2016-08-14 13:56:15.642742: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:170:5)
2016-08-14 13:56:15.642834: [debug] Run process: /home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.643157: [debug] Run process: /home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.662072: [debug] Process finished in 18 ms: /home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.662299: [debug] Process finished in 20 ms: /home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.671723: [debug] Process finished in 28 ms: /home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:271:3)
2016-08-14 13:56:15.671911: [debug] Resolving package entries
@(Stack/Setup.hs:232:5)
Executable named git not found on path: ["/home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin","/home/pgiarrusso/.local/bin","/home/pgiarrusso/bin","/home/pgiarrusso/bin","/usr/local/sbin","/usr/local/bin","/usr/sbin","/usr/bin","/sbin","/bin","/usr/games","/usr/local/games","/usr/lib/jvm/java-8-oracle/bin","/usr/lib/jvm/java-8-oracle/db/bin","/usr/lib/jvm/java-8-oracle/jre/bin","/usr/lib/jvm/java-8-oracle/bin","/usr/lib/jvm/java-8-oracle/db/bin","/usr/lib/jvm/java-8-oracle/jre/bin"]
pgiarrusso@kamino:~/git/ilc-agda$ stack build --dry-run
Executable named git not found on path: ["/home/pgiarrusso/.stack/programs/x86_64-linux/ghc-7.10.3/bin","/home/pgiarrusso/.local/bin","/home/pgiarrusso/bin","/home/pgiarrusso/bin","/usr/local/sbin","/usr/local/bin","/usr/sbin","/usr/bin","/sbin","/bin","/usr/games","/usr/local/games","/usr/lib/jvm/java-8-oracle/bin","/usr/lib/jvm/java-8-oracle/db/bin","/usr/lib/jvm/java-8-oracle/jre/bin","/usr/lib/jvm/java-8-oracle/bin","/usr/lib/jvm/java-8-oracle/db/bin","/usr/lib/jvm/java-8-oracle/jre/bin"]
pgiarrusso@kamino:~/git/ilc-agda$ stack build --dry-run
/usr/bin/git: createProcess: runInteractiveProcess: exec: permission denied (Permission denied)

[bergus]

@jwaldmann since you mentioned that you're on Gitlab-CI with the haskell-7.10.3 image - I experienced exactly the same issue. One fix that worked was adding

apt-get update -qq && apt-get install -y -qq git-all

before the stack install invocation, but changing the image to

image: fpco/stack-build

is much easier :-) See https://docs.haskellstack.org/en/stable/docker_integration/#image-repositories for other available images.