Following installation instructions leads to building GHC when entering nix-shell

[ghost]

Describe the bug

• I have development VM setup with a correctly configured nix cache (see https://github.com/abailly-iohk/nix-haskell-dev-vm/blob/main/packer/build-env.sh#L50)
• I git clone [email protected]:tweag/plutus-libs then direnv allow
• Nix rebuilds GHC

Expected behavior

• I expect nix to download whatever it can from the IOHK's caches and build the rest, but not GHC

Environment

• Linux haskell-dev-vm-1 5.13.0-1012-gcp #15-Ubuntu SMP Wed Jan 12 19:18:58 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
• Version of the code: 12f6915

Additional context

I am not a nix expert (actually, I hate nix :) ) but use it on a daily basis on the project I am working on without this kind of issue.

[VictorCMiraldo]

Thanks for submitting the issue @abailly-iohk. That's weird. It really should not compile GHC. None of us had this problem either nor are nix-experts. It could be a pinning that is now different from what IOHK provides. I eventually want to switch to using flakes instead of niv, but I don't have the time to address this immediately, unfortunately.

[VictorCMiraldo]

I just checked the logs from our CI and found GHC is being properly pulled from hydra.iohk.io, on this line:

copying path '/nix/store/ldh1r8ny0914fn8931a26ki4p3mky9bw-ghc-8.10.4.20210212' from 'https://hydra.iohk.io'...

[ghost]

Yeah, really I am at a loss. I will try again later on

[ghost]

Maybe or maybe not related, but I see the log messages repeated twice:

[ 1 of 51] Compiling Data.IxMap       ( src/Data/IxMap.hs, dist/build/Data/IxMap.o, dist/build/Data/IxMap.dyn_o )
[ 1 of 51] Compiling Data.IxMap       ( src/Data/IxMap.hs, dist/build/Data/IxMap.o, dist/build/Data/IxMap.dyn_o )
[ 2 of 13] Compiling Data.Conduit.Binary ( Data/Conduit/Binary.hs, dist/build/Data/Conduit/Binary.o, dist/build/Data/Conduit/Binary.dyn_o )
[ 2 of 13] Compiling Data.Conduit.Binary ( Data/Conduit/Binary.hs, dist/build/Data/Conduit/Binary.o, dist/build/Data/Conduit/Binary.dyn_o )
[ 3 of 13] Compiling Data.Conduit.ByteString.Builder ( Data/Conduit/ByteString/Builder.hs, dist/build/Data/Conduit/ByteString/Builder.o, dist/build/Data/Conduit/ByteString/Builder.dyn_o )
[ 3 of 13] Compiling Data.Conduit.ByteString.Builder ( Data/Conduit/ByteString/Builder.hs, dist/build/Data/Conduit/ByteString/Builder.o, dist/build/Data/Conduit/ByteString/Builder.dyn_o )

[ghost]

And it's definitely trying to build GHC:

[137 of 228] Compiling GHC.Hs.Expr[boot] ( compiler/GHC/Hs/Expr.hs-boot, dist/build/GHC/Hs/Expr.o-boot, dist/build/GHC/Hs/Expr.dyn_o )
[137 of 228] Compiling GHC.Hs.Expr[boot] ( compiler/GHC/Hs/Expr.hs-boot, dist/build/GHC/Hs/Expr.o-boot, dist/build/GHC/Hs/Expr.dyn_o )
[138 of 228] Compiling DataCon[boot]    ( compiler/basicTypes/DataCon.hs-boot, dist/build/DataCon.o-boot, dist/build/DataCon.dyn_o )
[138 of 228] Compiling DataCon[boot]    ( compiler/basicTypes/DataCon.hs-boot, dist/build/DataCon.o-boot, dist/build/DataCon.dyn_o )

[ghost]

copying path '/nix/store/ldh1r8ny0914fn8931a26ki4p3mky9bw-ghc-8.10.4.20210212' from 'https://hydra.iohk.io'...

This path exists in my local nix sstore:

$ ls -l /nix/store/ldh1r8ny0914fn8931a26ki4p3mky9bw-ghc-8.10.4.20210212
total 24
dr-xr-xr-x  2 root root 4096 Jan  1  1970 bin
dr-xr-xr-x  2 root root 4096 Jan  1  1970 evalDeps
dr-xr-xr-x 36 root root 4096 Jan  1  1970 exactDeps
dr-xr-xr-x  3 root root 4096 Jan  1  1970 lib
dr-xr-xr-x  2 root root 4096 Jan  1  1970 nix-support
dr-xr-xr-x  4 root root 4096 Jan  1  1970 share

Why is it compiling some GHC then?

[VictorCMiraldo]

That is really interesting indeed. I'd try commenting lines 53 and 58 from nix/packages.nix and see if it still wants to build GHC.

[ghost]

I ended up waiting for the build to finish, and was able to build plutus-libs but of course entering the shell took ages 😬
It seems that both 8.10.4 and 8.10.7 are pulled in the shell. When running

nix-store -qR --include-outputs $(nix-instantiate shell.nix)

I see the following derivations (among many others of course)

/nix/store/11r2j7xlgm0sffrkky3194w7crxykmbd-ghc-lib-parser-8.10.7.20210828.tar.gz
/nix/store/lznhlagid2ni8dvxpzibmn959ijpcj3s-ghc-8.10.7-src.tar.xz.drv
/nix/store/r98ksf5bjzgh1lw9g8pazm1dbcgx5yj5-ghc-8.10.7-configured-src.drv
/nix/store/c1qwj542h43fc04qs10k1i59cv1g6wbh-ghc-8.10.7.drv
...
/nix/store/wj5vlr3dm2kgfdqjwv769whg55qv8m6k-ghc-8.10.4-src.tar.xz.drv
/nix/store/dqx5fk7b6qrw1289hnkkba8z6c0q8gk2-ghc-8.10.4.20210212-configured-src.drv
/nix/store/dr12s6x8rck3v1xa2warfwg88xn1mjjm-ghc-8.10.4.20210212.drv
/nix/store/ldh1r8ny0914fn8931a26ki4p3mky9bw-ghc-8.10.4.20210212

[ghost]

Don't know if that helps but here is a list of referrers on ghc-8.10.7

nix-store -q --referrers-closure --include-outputs /nix/store/c1qwj542h43fc04qs10k1i59cv1g6wbh-ghc-8.10.7.drv

I can see that cabal 3.6.2.0 is there, could it be that the package is requiring a too recent cabal?

[VictorCMiraldo]

Are you having this issue on a pure nix-shell? I can't reproduce this issue and my nix knowledge is close to zero, hence I'm afraid I can't help much at this point. As I mentioned, I want to switch to using flakes to pin our dependencies but we have other higher priority issues at hand.

Also, it is worth mentioning solving #69 will trigger a much slimmer build process. I see that most of the upstream PRs needed to close that have been addressed, which means we might be able to tackle that soon!

[VictorCMiraldo]

@abailly-iohk, I just had to reinstall nix on my machine and I tried pulling cabal-install from nixpkgs instead of haskell-nix (which pulls cabal-install 3.4.0.0); GHC 8.10.7 was still brought in and I have no idea why.

I think this could be the haskell-language-server; which our CI does not require...

[ghost]

I have not tried it in a while, will retry on a fresh VM. Have you made any change?

[VictorCMiraldo]

@abailly-iohk , it didn't change anything, so I didn't make any changes. I'm clueless as to why its bringing two GHCs but given that the environment works, I won't be looking into that.

Also, I'm almost entirely certain that it did not build GHC since entering the nix-shell from a clean nix install took about 30 minutes, which is way too short to compile GHC.

[ghost]

I am clueless on how to fix this, and it does not prevent building the plutus-libs so let's close.