Better support for multiple packages

[coot]

Description

This PR allows to create documentation for multiple packages. It is useful if one has multiple packages in a single repository.

Workflow

Workflow to create documentation for multiple packages. It assumes that there are two packages package-1 and package-2.

Configure & create haddocks

Add the following lines to cabal.project.local file (for every package):

package package-1
  haddock-options: --use-index=../doc-index.html --use-contents=../index.html --base-url=..

package package-2
  haddock-options: --use-index=../doc-index.html --use-contents=../index.html --base-url=..

The newly added option --base-url configures the location from which static files will be loaded.

One can now run:

cabal haddock --haddock-html --haddock-quickjump all

To compile haddocks.

Copy documentation

Copy documentation to a common directory (docs in this example), e.g.

mkdir docs
cp -r ./dist-newstyle/build/x86_64-linux/ghc-8.10.2/pakcage-1-0.1.0.0/noopt/doc/html/package-1 docs
cp -r ./dist-newstyle/build/x86_64-linux/ghc-8.10.2/pakcage-2-0.1.0.0/noopt/doc/html/package-2 docs

Create index

The following command will create html index as well doc-index.json file by combining doc-index.json files of all included packages.

haddock \
    -o docs \
    --quickjump \
    --gen-index \
    --gen-contents \
    --read-interface=some-package-1,docs/package-1/some-package-1.haddock \
    --read-interface=some-package-2,docs/package-2/some-package-2.haddock

Serve docs directory and enjoy the documentation with a single html index and QuickSearch working across all projects.

Changes

• Using aeson for serialisation and deserialisation of doc-index.json files;
• doc-index.json file for the summary page is generated from doc-index.json files of all packages included with --read-interface option (-i);
• added --base-url flag (Flag_BaseURL). If given and not equal to . or ./ static files (css, js, json) will not be copied. All pages will load them from the given url.

Target branch

I am not sure which target branch I should pick for this PR, I choose ghc-8.10 as this is the current version of ghc.

[coot]

@Kleidukos anything, beside resolving conflicts ;), that I can help to make this PR get reviewed / accepted?

[Kleidukos]

Alright, a pretty good PR. I left a couple of remarks, mostly extraneous blank lines, and a question about the Json parser you're committing. cc @coot

[Kleidukos]

@coot Also, don't forget to open a PR to update the docs with the new option --base-url and the workflow. :)

[coot]

Which branch should I target, ghc-8.10 or ghc-9.0?

[coot]

I see that the ghc-9.0 is the default branch now, I'll target that branch then.

[coot]

@Kleidukos all your comments are addressed.

[Kleidukos]

@coot Wonderful, thank you very much for this contribution ✨

[coot]

I spottved a typo in the description of --base-url option, and I pushed one commit which documents it in the invoke.rst file. Could you proof read it. I sometimes make silly typos.

[mpickering]

This is certainly an improvement. Thanks @coot. The patch didn't do quite what I expected though, the resulting index has all the modules merged together from all different packages.

and clicking on links from one package to another local package result in a rendering error, as the links are not correctly relative to the other copied documentation.

[coot]

@Kleidukos maybe you can help me: how one can build haddock ghc-9.0 branch, which compiler version one should use?

I tried to use just released ghc-9.0.1 but I get:

cabal: Could not resolve dependencies:
[__0] trying: haddock-api-2.24.0 (user goal)
[__1] trying: ghc-paths-0.1.0.12 (dependency of haddock-api)
[__2] next goal: ghc-paths:setup.Cabal (dependency of ghc-paths)
[__2] rejecting: ghc-paths:setup.Cabal-3.4.0.0/installed-3.4.0.0 (conflict:
ghc-paths => ghc-paths:setup.Cabal>=1.6 && <3.3)
[__2] rejecting: ghc-paths:setup.Cabal-3.2.1.0, ghc-paths:setup.Cabal-3.2.0.0,
ghc-paths:setup.Cabal-3.0.2.0, ghc-paths:setup.Cabal-3.0.1.0,
ghc-paths:setup.Cabal-3.0.0.0, ghc-paths:setup.Cabal-2.4.1.0,
ghc-paths:setup.Cabal-2.4.0.1, ghc-paths:setup.Cabal-2.4.0.0,
ghc-paths:setup.Cabal-2.2.0.1, ghc-paths:setup.Cabal-2.2.0.0,
ghc-paths:setup.Cabal-2.0.1.1, ghc-paths:setup.Cabal-2.0.1.0,
ghc-paths:setup.Cabal-2.0.0.2, ghc-paths:setup.Cabal-1.24.2.0,
ghc-paths:setup.Cabal-1.24.0.0, ghc-paths:setup.Cabal-1.22.8.0,
ghc-paths:setup.Cabal-1.22.7.0, ghc-paths:setup.Cabal-1.22.6.0,
ghc-paths:setup.Cabal-1.22.5.0, ghc-paths:setup.Cabal-1.22.4.0,
ghc-paths:setup.Cabal-1.22.3.0, ghc-paths:setup.Cabal-1.22.2.0,
ghc-paths:setup.Cabal-1.22.1.1, ghc-paths:setup.Cabal-1.22.1.0,
ghc-paths:setup.Cabal-1.22.0.0, ghc-paths:setup.Cabal-1.20.0.4,
ghc-paths:setup.Cabal-1.20.0.3, ghc-paths:setup.Cabal-1.20.0.2,
ghc-paths:setup.Cabal-1.20.0.1, ghc-paths:setup.Cabal-1.20.0.0,
ghc-paths:setup.Cabal-1.18.1.7, ghc-paths:setup.Cabal-1.18.1.6,
ghc-paths:setup.Cabal-1.18.1.5, ghc-paths:setup.Cabal-1.18.1.4,
ghc-paths:setup.Cabal-1.18.1.3, ghc-paths:setup.Cabal-1.18.1.2,
ghc-paths:setup.Cabal-1.18.1.1, ghc-paths:setup.Cabal-1.18.1,
ghc-paths:setup.Cabal-1.18.0, ghc-paths:setup.Cabal-1.16.0.3,
ghc-paths:setup.Cabal-1.16.0.2, ghc-paths:setup.Cabal-1.16.0.1,
ghc-paths:setup.Cabal-1.16.0, ghc-paths:setup.Cabal-1.14.0,
ghc-paths:setup.Cabal-1.12.0, ghc-paths:setup.Cabal-1.10.2.0,
ghc-paths:setup.Cabal-1.10.1.0, ghc-paths:setup.Cabal-1.10.0.0,
ghc-paths:setup.Cabal-1.8.0.6, ghc-paths:setup.Cabal-1.8.0.4,
ghc-paths:setup.Cabal-1.8.0.2, ghc-paths:setup.Cabal-1.6.0.3,
ghc-paths:setup.Cabal-1.6.0.2, ghc-paths:setup.Cabal-1.6.0.1,
ghc-paths:setup.Cabal-1.4.0.2, ghc-paths:setup.Cabal-1.4.0.1,
ghc-paths:setup.Cabal-1.4.0.0, ghc-paths:setup.Cabal-1.2.4.0,
ghc-paths:setup.Cabal-1.2.3.0, ghc-paths:setup.Cabal-1.2.2.0,
ghc-paths:setup.Cabal-1.2.1, ghc-paths:setup.Cabal-1.1.6,
ghc-paths:setup.Cabal-1.24.1.0 (constraint from minimum version of Cabal used
by Setup.hs requires >=3.4)
[__2] fail (backjumping, conflict set: ghc-paths, ghc-paths:setup.Cabal)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: ghc-paths:setup.Cabal, haddock-api,
ghc-paths

Building with hadrian/build still does not work?

Wouldn't it be more convenient to support multiple versions of ghc by haddock?

[Kleidukos]

@coot Can you rebase on the current ghc-9.0? You should inherit the proper github action files for the CI

[coot]

Thanks @Kleidukos, it worked :).

[coot]

@mpickering did you tried to pass an absolute url to --base-url?

[coot]

Did you build haddock of sub-packages with cabal haddock --haddock-html --haddock-quickjump all and then copied all the produced haddocks to a common directory? Maybe I falsely assume that there should not be links to anything under dist-newstyle (at least for local packages).

[coot]

I force pushed, because I found that the json index was not generated when the option --quickjump option was specified. I think I introduced this when I was rebasing the other time.

[coot]

All tests now pass :).

[Kleidukos]

@alexbiehl This is a pretty meaty PR, I'll need your eyes on that one for the end ;)

[Mikolaj]

Great job! Am I right that this could be used (perhaps with some changes) as a workaround for haddock not being able to produce sensible Hackage documentation for packages with many internal libraries? If this is trivial enough, could the necessary invocations be listed in documentation as well? I'd gladly test them on my multi-lib package.

[coot]

Am I right that this could be used (perhaps with some changes) as a workaround for haddock not being able to produce sensible Hackage documentation for packages with many internal libraries?

Yes you're right :).

If this is trivial enough, could the necessary invocations be listed in documentation as well? I'd gladly test them on my multi-lib package.

Good point, I should add that; For now you can check here for instructions (the haddocs are served on github-pages). The ultimate goal is to have cabal haddock --directory docs all to produce haddocs for all local components.

[Kleidukos]

Alright it's squashed and merged. Thanks a lot for your patience @coot

[coot]

@Kleidukos my pleasure; since its merged to ghc-9.0 branch, does it mean it will be available in a next 9.0.x release?

[Kleidukos]

@coot I'll backport the patch to ghc-9.2 as well

[coot]

Let me know if you'll have any problems or need some help with testing. It's easy to get it wrong (or at least I got it wrong when I was rebasing).

[Kleidukos]

Okay sure, will ping you! :)