'cabal build' fails to build package from head.hackage (due to "Downloading" being missing)

[Mikolaj]

Describe the bug

'cabal build' fails with error code 1 when building a package from https://ghc.gitlab.haskell.org/head.hackage

To Reproduce
Steps to reproduce the behavior as reported by @phadej, after setting up head.hackage, as in the link above:

cabal v2-update
mkdir -p ~/.cabal/packages/head.hackage.ghc.haskell.org/primitive/0.7.1.0
cp ~/.cabal/packages/hackage.haskell.org/primitive/0.7.1.0/primitive-0.7.1.0.tar.gz ~/.cabal/packages/head.hackage.ghc.haskell.org/primitive/0.7.1.0/primitive-0.7.1.0.tar.gz 
cabal build primitive -w ghc-9.2.0.20210422 --allow-newer                                                                                                     

Cabal file:

cabal-version:       3.4
name:                test
version:             0.0.0

executable tst
  main-is:             Main.hs
  default-language:    Haskell2010
  build-depends:       base, primitive

Log:

Build profile: -w ghc-9.2.0.20210422 -O1
In order, the following will be built (use -v for more details):
 - primitive-0.7.1.0 (lib) (requires build)
Starting     primitive-0.7.1.0 (lib)
Building     primitive-0.7.1.0 (lib)

Return code: 1.

We expected to see "Downloading" in the log and then the building would have a chance to succeed.

[Mikolaj]

This is high priority if we want to advertise head.hackage more widely, which we probably want to.

[Mikolaj]

I'm not sure if #5952 is related. I have yet to try and reproduce with normal cabal update of head.hackage vs the workaround from this ticket.

Edit: it's potentially related, but it fails already at cabal update from what I read. Also, cabal install may be somehow related, so once this is fixed for cabal build, it would be great to test with cabal install.

[gbaz]

So as far as I can tell, the issue is that cabal doesn't verify that a foo-01.tar.gz it already has downloaded from a repo has not "changed out" from under it?

[gbaz]

Here's the logic, but I don't know the right way to fix it:

cabal/cabal-install/src/Distribution/Client/FetchUtils.hs

Line 113 in 67e58ca

checkRepoTarballFetched :: Repo -> PackageId -> IO (Maybe FilePath)

If we know we need a package, we generate the path we expect the local copy of the tar.gz to be in, and check if the file exists. If so, we return the tarball and make use of it.

I.e. we assume a repo does not actually change the tarball it is giving us out from under us.

I think that we would want to change the packageFile function see if the repo is secure, and if so to checksum the file and match it against the hash provided by the repo, redownloading if they don't match.

Does hackage-security provide any convenience functions for this? @phadej @edsko

Pointers to elsewhere in our code where we do this very welcome.