Migrate option documentation to Markdown #707
Conversation
emilazy
force-pushed
the
options-markdown-migration
branch
from
d37b966 to
b20e597
Compare
|
(Rebased to reorganize some commits in a more logical manner. No change to the resulting tree.) |
This is the last bump of the nixpkgs-unstable channel before the DocBookalypse.
The temporarily pinned nixpkgs version already has this version, and where we're going, we don't need backports. This reverts commit 737cfde.
Except for `manualEpub`, which is going away.
All exposed options should have documentation, and `nixosOptionDoc` will give an error if it's missing.
The only change to `options.json` are that the `declarations` fields now contain correct GitHub URLs. The changes to the HTML manual are all either strict improvements: * "Declared by:" links point to the correct URLs. * `lib.mdDoc` documentation is included. or else trivial: * Additional whitespace in the HTML source. * Some interchange of `<pre class="programlisting">` and `<code class="literal">`. * Fancy quotes used in some type descriptions. `optionsDocBook` is exported temporarily as it is used by `nix-doc-munge` for the Markdown conversion process. This drops support for building the manual on 22.11, which lacks a complete implementation of this processor, and provides stubbed-out manual packages with an explanation on that version.
These help `nix-munge-doc` automate more of the Markdown conversion process. See the following nixpkgs commits for explanations of many of these changes: * NixOS/nixpkgs@275a34e * NixOS/nixpkgs@694d5b1 * NixOS/nixpkgs@f1d39b6 * NixOS/nixpkgs@16102dc I couldn't think of any particularly good way to format the `system.defaults` breadcrumbs, so I just made them standalone paragraphs. They weren't rendering correctly in DocBook anyway.
The argument to `mkEnableOption` is automatically wrapped in a full sentence.
These all use DocBook markup too complex for `nix-doc-munge` to handle, have syntax that clashes with Markdown, or already contain Markdown syntax that currently isn't rendering correctly. Converting DocBook list syntax makes me think that maybe Markdown isn't so bad after all.
This process was automated by [my fork of `nix-doc-munge`]; thanks to @pennae for writing this tool! It automatically checks that the resulting documentation doesn't change, although my fork loosens this a little to ignore some irrelevant whitespace and typographical differences. As of this commit there is no DocBook remaining in the options documentation. You can play along at home if you want to reproduce this commit: $ NIX_PATH=nixpkgs=flake:nixpkgs/c1bca7fe84c646cfd4ebf3482c0e6317a0b13f22 \ nix shell nixpkgs#coreutils \ -c find . -name '*.nix' \ -exec nix run github:emilazy/nix-doc-munge/0a7190f600027bf7baf6cb7139e4d69ac2f51062 \ {} + [my fork of `nix-doc-munge`]: https://github.com/emilazy/nix-doc-munge
The Markdown transition is over and our compatibility with `nixpkgs-unstable` is restored.
Now that all the DocBook documentation is gone, we can switch to the new NixOS documentation generator. No meaningful change in HTML output, except that I removed the rather pointless preface and changed the title accordingly. Manual page output is greatly improved; it was kind of broken before. The `sed` hack is pretty gross but I have confirmed that nixpkgs will be happy to accept a PR to make things a little more customizable. This also drops the `manual` alias (deprecated in nixpkgs in 2018 and imported into nix-darwin), and `manualEpub` (because the NixOS documentation generator doesn't support it and also nobody wants this as an ebook).
22.11 is deprecated and can no longer generate the manual.
20.03 and 22.03 are both deprecated.
emilazy
force-pushed
the
options-markdown-migration
branch
from
b20e597 to
9d6702c
Compare
|
Rebased to fix conflicts and address reviews. I also did a few minor tweaks and, more importantly, completed the migration to I noticed that Would be nice to get this merged and done with, since it's a little bit of a pain to rebase, but I understand there's a good amount to review even in the non-automated commits. |
|
@emilazy Hello, I don't know anything about the new documentation generator. Looks quite interesting, though 🙂 I just have a few considerations that maybe you know the answer to. Mainly I'm curious whether the build closure size changes a lot? Any support for custom CSS styling of the generated HTML? Would it be difficult to add support for Asciidoc? |
|
Didn't check the build closure size but I assume it's comparable or went down as the new generator was specifically written for NixOS and doesn't bring in an XML toolchain; it's custom-written Python. The output is almost exactly identical to the previous one down to markup structure (you can compare the manuals on the NixOS site) and supports full CSS and JS customization. It's all-in on Markdown: current nixpkgs has completely switched over to its own flavour of Markdown and won't accept anything else. I like AsciiDoc more than Markdown in many ways myself, but I don't think there's much use in swimming against the tide here, now that the RFCs are done and the migration has been in progress for many months; all NixOS documentation has been converted, nixpkgs has Markdown documentation in options itself, all the tools take only one input format with DocBook support having been removed, so I don't think there would be any appetite to add additional ones now. It'd impede sharing code with NixOS too. (I was going to work on doing the same conversion for Home Manager based on nix-community/home-manager#4142 (comment), but if you have reservations then I'll hold off.) |
|
@emilazy Thanks for the response! Will have to do some experiments to see what would be the best course of action. For now, at least, I've updated nmd to handle the recent Nixpkgs change. |
|
Sure; either way the Home Manager NixOS/nix-darwin module needs to move over to Markdown documentation, as it's broken right now under latest nixpkgs. |
|
Thanks again for tackling this! |
NixOS has completed the the transition to using Markdown for option documentation, and
lib.mdDochas become a no-op. This poses a problem for us, as our creaky old copy of the NixOS option documentation generator can't even process Markdown. #704 was a stop-gap solution, but still doesn't give us a correct manual (we currently have in-tree modules usinglib.mdDocthat simply don't have their docs in the manual), and not moving to Markdown will make it increasingly more difficult to share code between NixOS and nix-darwin modules, as well as keeping it painful to write well-formatted documentation.This PR migrates nix-darwin to use the new NixOS documentation generator and converts all option documentation to Markdown. The process is:
nixosOptionsDocto process the option documentation.nix-doc-munge.nixos-render-docs.This removes a lot of complexity and duplication and produces a near-identical manual. It necessarily requires dropping support for building the manual with 22.11; its version of the documentation generator cannot handle the Markdown constructs the conversion produces. (23.05 works fine, of course.)
The number of commits might be a little daunting but I tried to factor them out for easy review :) The big automated conversion is e65131e and should be fully reproducible with the command in the commit message; the other commits are manually written or are simple find-and-replace jobs that I reviewed manually. I did not add a
.git-blame-ignore-revsfile for the mass change as NixOS doesn't seem to have for its equivalent migration, but I can add one if desired.I will try to send a PR to Home Manager to migrate their module documentation to Markdown after this is merged; if anyone knows of any other third-party nix-darwin modules using DocBook documentation I will look into fixing them too.
cc @LnL7 @domenkozar