haddock test related failures with GHC 9.0.2 and 9.2.{1,2,3}

[andreabedini]

From #7952 (comment)

The following tests fail on GHC 9.0.2 (but not GHC 9.0.1) on both Linux and Mac

• PackageTests/Backpack/Includes2/setup-external.test.hs
• PackageTests/Backpack/Includes3/setup-external-ok.test.hs
• PackageTests/Backpack/Includes2/setup-per-component.test.hs

A failed run is https://github.com/haskell/cabal/runs/5178990935?check_suite_focus=true#step:10:1374

In all cases, the error happens while dealing with haddock. The relevant part of the log is:

Warning: The documentation for the following packages are not installed. No links will be generated to these packages: base-4.15.1.0, ghc-bignum-1.1, ghc-prim-0.7.0, Includes2-0.1.0.0
creating setup-per-component.dist/work/dist/doc/html/Includes2
/usr/local/.ghcup/ghc/9.0.2/bin/haddock-ghc-9.0.2 '@setup-per-component.dist/work/dist/doc/html/Includes2/haddock-response24859-1.txt'
   0% (  0 /  3) in 'Mine'
  Missing documentation for:
    Module header
    Mine (mylib/Mine.hs:3)
    mine (mylib/Mine.hs:4)
haddock: internal error: haddock:iface
CallStack (from HasCallStack):
  error, called at utils/haddock/haddock-api/src/Haddock/Interface.hs:178:16 in main:Haddock.Interface

[Mikolaj]

@Kleidukos, @wz1000: any cheap shot at why this fails with 9.0.2 and not 9.0.1?

[wz1000]

Might be related to https://gitlab.haskell.org/ghc/ghc/-/issues/20903

[Kleidukos]

No idea on my end :(

[Mikolaj]

Thanks.

@hasufell: I guess you had no reports of 9.0.2 having missing/broken haddocks in the Linux package, as distributed by ghcup? Probably not the case, but just checking.

[wz1000]

Some 9.0.2 distributions (all the hadrian generated ones in fact - which would be the fedora one IIRC) also don't have haddocks.

[Mikolaj]

@wz1000: thank you. So nothing to worry about. It'll get fixed in 9.0.3, if it gets released.

@andreabedini: how would you mark the failures as expected so that we remember to unmark then when 9.0.3 is out? Perhaps we should leave this ticket open to remember? Change the ticket title mentioning 9.0.3 and adding lots of exclamation marks?

[hasufell]

@hasufell: I guess you had no reports of 9.0.2 having missing/broken haddocks in the Linux package, as distributed by ghcup? Probably not the case, but just checking.

No idea

[andreabedini]

@andreabedini: how would you mark the failures as expected so that we remember to unmark then when 9.0.3 is out? Perhaps we should leave this ticket open to remember? Change the ticket title mentioning 9.0.3 and adding lots of exclamation marks?

@Mikolaj Given all three failures seem to be related to haddock, I'd say we can use this ticket, either as it as it is or changing the title. The ticket is also mentioned in the workflow so we can be reminded from there too.

[andreabedini]

This seems to appen on GHC 9.2 too https://github.com/haskell/cabal/runs/5298365693?check_suite_focus=true#step:11:1995

[ulysses4ever]

Unfortunately, hasn't got fixed in 9.2.2 or 9.2.3.

[ulysses4ever]

One more observation: after successfully checking Haddocks in base (via --show-iface, as suggested in https://gitlab.haskell.org/ghc/ghc/-/issues/20903), I can confirm that this failure still reproduces on GHC 9.2.3 (and 9.0.2, as delivered by GHCup in Linux x86_64). So we need another guess at what happens here...

[roconnor-blockstream]

Is there a workaround to avoid this crash, beyond simply not building documentation?

[Mikolaj]

@roconnor-blockstream: I don't think we know of any. What's the scenario in which you get an error of this kind?

[roconnor-blockstream]

Building my project under GHC 9.2 (and 9.4). More specifically the runhaskell Setup.lhs haddock command fails in the end with the error

haddock: internal error: haddock:iface
CallStack (from HasCallStack):
  error, called at utils/haddock/haddock-api/src/Haddock/Interface.hs:179:16 in main:Haddock.Interface

My project makes use of Backpack.

[Mikolaj]

Thank you for the details. Where is that Setup.lhs file? I can only see Setup.hs. Does it fail like that with cabal haddock as well?

[roconnor-blockstream]

Sorry. I should have written Setup.hs.

cabal haddock does appear to work.

[Mikolaj]

OK, that's an important hint for somebody willing to dig into cabal and haddock code. And the Setup.hs is not anything custom, so the problem is probably not related to the dreaded Custom Setups. I guess the place to start would be to run with verbosity -v2 and compare the arguments passed to haddock with runhaskell Setup.lh haddock and with cabal haddock in your project.

[roconnor-blockstream]

Using runhaskell Setup.hs haddock executes

/nix/store/4zpfpnwflsrpf1lmnnx9jkdypzpz2b1j-ghc-9.2.3-with-packages/bin/haddock '@dist/doc/html/Simplicity/haddock-response287381-7.txt'

while using cabal haddock

/nix/store/4zpfpnwflsrpf1lmnnx9jkdypzpz2b1j-ghc-9.2.3-with-packages/bin/haddock '@/home/roconnor/Simplicity/dist-newstyle/build/x86_64-linux/ghc-9.2.3/Simplicity-0.0.0/l/Simplicity-Indef/doc/html/Simplicity/haddock-response286953-1.txt'

But these response files are cleaned up right away so I cannot view them. Is there a flag to keep them?

[Mikolaj]

Ouch, that's not as helpful as I imagined. I can't find any such option either in haddock or in cabal haddock (in case the file is deleted by cabal). I suppose hacks such as copyfs may do the trick: https://unix.stackexchange.com/questions/71067/view-temporary-files-which-exist-for-milliseconds

[jneira]

such flag (--keep-temp-files?) would be really useful to trace problems like this

[roconnor-blockstream]

the --keep-temp-files flag works with runhaskell Setup.hs, however I haven't figured out where to place haddock-keep-temp-files: True to make it work with cabal haddock.

One thing to observe is that runhaskell Setup.hs builds the haddock for each different library in the same place, clobbering the previous library's documentation. This has always been a problem for my backpack builds. The cabal haddock command seems to properly build each component in a separate directory.

[Mikolaj]

One thing to observe is that runhaskell Setup.hs builds the haddock for each different library in the same place

Oh, that rings a bell. There was a realated (partial?) fix recently: #7827. Also related: haskell/haddock#1363 amd haskell/haddock#1379. But OTOH, I seem to remember support for multiple (public?) libraries was supposed to be added only at the cabal haddock level, not Setup level. Do I remember right @fgaz?

[Mikolaj]

BTW, is haskell/haddock#1363 fixed? @roconnor-blockstream seems to have a reasonable experience with cabal haddock and a backpack project. Is anything else missing? @Kleidukos, @coot, @fgaz? I guess Hackage may not yet be ready to display such haddocks?

[roconnor-blockstream]

I wouldn't say I have a reasonable experience. I make due with making a manual index.html, and even then, all the problems in haskell/haddock#1363 persist (plus now it is simply crashing). I just assumed they would be fixed one day, but maybe the right answer is to use cabal haddock. I haven't really examined the output yet.

[roconnor-blockstream]

FWIW, I no longer get these haddock: internal error: haddock:iface errors when using GHC 9.8.2 at least.

[Mikolaj]

Wonderful.

Anything else left to be done in this ticket? Do the tests from the ticket description pass now?