Cross-compilation broken by custom Setup.hs used for doctests

[vmchale]

Currently, the custom Setup.hs used for doctests causes cross-compilation to fail due to a loingstanding cabal bug.

I set write-ghc-environment-files to always in my cabal.project and then just run doctest from the command-line as a workaround.

[georgefst]

Hmm, yeah, I would like to get away from using a custom setup if it's at all possible. There are a lot of issues around them, and this isn't the kind of library that justifies it. The pretty-simple tests quite often fail to run for me on GHC 8.10 anyway.

I'm sure there was talk at some point of a proper cabal doctest command? I'll have a look for that discussion.

[georgefst]

Running doctest from the project route is giving me cannot satisfy -package-id pretty-simple-3.3.0.0-inplace, even when I have an environment file which contains precisely that (generated by cabal repl --write-ghc-environment-files=always.

@vmchale How are you getting this to work?

[vmchale]

I put them in my cabal.project file as a line like

write-ghc-environment-files: always

(perhaps you've run into another cabal bug 😬 )

[georgefst]

It doesn't seem to make a difference how I specify the flag (file vs. command line), which to be honest is a good thing...

I'm not very familiar with doctest though. To be clear, you're just running doctest, with no args, from the pretty-simple directory? If so then I guess I am hitting a bug.

FWIW, I'm using doctest 0.17 on pretty-simple's HEAD (763d769), with GHC 8.10.2, on Linux.

[vmchale]

My workflow is

cabal build --write-ghc-environment-files always
doctest src

For whatever reason doctest by itself doesn't work. And I set build-type: Simple in the .cabal file

(I think cabal repl doesn't quite do what one wants; it doesn't build pretty-simple itself)

[georgefst]

(I think cabal repl doesn't quite do what one wants; it doesn't build pretty-simple itself)

Aha, you're right! I'd missed that subtlety.

[cdepillabout]

The problem with doctest is that it doesn't know how to find the required package database when running with the various Haskell build tools (stack, cabal-v1, cabal-v2, haskell environment from nixpkgs, haskell.nix).

cabal-doctest is the only reliable solution right now (although it still doesn't work with haskell.nix).

I don't want to move away from cabal-doctest until there is some other way to reliably run doctests.

However, as you've seen, the custom Setup.hs is only for the doctests, so as a workaround you should be able to delete the Setup.hs, change the build-type to Simple, and disable the doctests.

If you wanted to make this process a little easier, please feel free to send a PR putting the doctests under a flag that you can easily disable.

[georgefst]

Worth keeping an eye on: https://mail.haskell.org/pipermail/haskell-cafe/2020-August/132570.html

[dpwiz]

BTW, lens did a similar update recently.

(An older version of lens and this package breaks with global -split-sections ghc flag.)

[georgefst]

Thanks @dpwiz, that link is very handy.

Looking at the cabal-docspec manual, I think that could be the perfect solution.

[georgefst]

If you wanted to make this process a little easier, please feel free to send a PR putting the doctests under a flag that you can easily disable.

Having just tried this, simply changing the one line to build-type: Simple is enough to get cross-compilation working. But it isn't something that can be put behind a flag AFAICT.

[cdepillabout]

There's a long thread on the Cabal issue tracker about doctests: haskell/cabal#4500

Recently sol commented about using cabal repl directly with doctest to run doctests: haskell/cabal#4500 (comment)

I'd be interested to see a comparison between cabal repl --with-ghc=doctest, cabal-docspec and cabal-doctests (including whether they easily work with stack, Nixpkgs haskell stuff, and haskell.nix).

[cdepillabout]

Looks like there is one more doctest contender in the mix, doctest-parallel:

• https://hackage.haskell.org/package/doctest-parallel
• https://github.com/martijnbastiaan/doctest-parallel
• https://www.reddit.com/r/haskell/comments/rnkof1/ann_doctestparallel_isolate_and_speed_up_your/

One feature of doctest-parallel is that it apparently parses cabal files to discover modules, so you don't need a custom Setup.hs.

It would be great add a comparison to doctest-parallel to the list above

[cdepillabout]

There has apparently recently been a PR to cabal that adds a new doctest-related feature:

• PR: code-generators field in test stanza. haskell/cabal#7688
• repo showing example of using this with doctests: https://github.com/gbaz/cabal-doctest-demo

I haven't looked at this thoroughly, but this might be the best option so far, since it will officially be supported by cabal.

edit: This has been released in Cabal-3.8.1.0: https://discourse.haskell.org/t/just-released-cabal-3-8-1-0/4920

[cdepillabout]

In #129 (comment), @andreabedini gave a link to haskell/cabal#9551, which may provide a better way to run doctests.

[georgefst]

In #129 (comment), @andreabedini gave a link to haskell/cabal#9551, which may provide a better way to run doctests.

Although it might be years before we're willing to drop support for all Cabal versions which don't support it!

[Kleidukos]

@cdepillabout doctest does not require a Custom setup anymore, would you be open to remove it?

[georgefst]

@cdepillabout doctest does not require a Custom setup anymore, would you be open to remove it?

I'd love to see it removed, especially with the new JS/WASM backends being about to make cross-compilation a lot more common. I believe @cdepillabout is keen in principle, modulo the concerns in #82 (comment).

To be clear, are you talking about this new experimental approach? I've found it difficult to keep up with progress on doctest support, especially as someone whose interest is limited entirely to being a co-maintainer of this particular library.

[Kleidukos]

Yes I am talking about it. It's labelled as experimental because it is fresh, but I have not found bugs when using it.

[cdepillabout]

@Kleidukos I wrote a little more about what I'd like in #129 (comment).

Is there some easy way to have the doctests run when you run cabal test? It seems annoying and error prone to have to run a separate command. (For instance, Haskell tests are automatically run for packages built with the Nixpkgs Haskell infrastructure, but doctests wouldn't be automatically run.)

On the other other hand, I realize that cabal-doctest has a bunch of downsides and is no longer really being maintained, so we'll have to do something eventually...

[Kleidukos]

@cdepillabout I understand what you mean.

If you can't move right away from the Custom setup, could you then perhaps bump the version bound of the Cabal library used in pretty-simple, so that users may use the package alongside --enable-multi-repl, which requires Cabal 3.12?

[georgefst]

If you can't move right away from the Custom setup, could you then perhaps bump the version bound of the Cabal library used in pretty-simple, so that users may use the package alongside --enable-multi-repl, which requires Cabal 3.12?

I'm not quite sure what you mean by this. pretty-simple doesn't have an upper bound on Cabal.

[Kleidukos]

Okay then it's my fault, it must be the cabal bound determined by my GHC version. :)