Vision for MkDocs on Read the Docs

[davidfischer]

There was quiet a bit of discussion in #4556 about MkDocs on Read the Docs. I wanted to move some of that discussion here as that is an active PR that I am hoping to merge soonish.

I don't want to speak for people other than myself as the RTD maintainer who has worked on MkDocs more recently than anyone else at RTD.

Where RTD would like to go with MkDocs

• We have an MkDocs Milestone although much of it is bugfixes
• More documentation and a full "getting started with MkDocs on RTD" guide as well as a "customizing MkDocs" section
• Additional integrations with MKDocs via plugins. This part may be more of a community effort.

Things RTD would like from MkDocs

• Search Indexing RTD was using the json builder (removed in 0.17, see Remove MkdcocsJson builder #4205) in order to index for searching. It would be nice to have this or something like it as opposed to crawling/indexing the HTML.
• Format stability Stability in the mkdocs.yml format especially around themes. There were quite a few backward incompatible changes here from 0.15.x to 1.x. Because RTD supports multiple versions of MkDocs, this was an issue. This may be less of an issue once we stop overriding the theme.

Things MkDocs needs from RTD

I'm leaving this section a bit bare for now as I don't want to speak for the MkDocs maintainers. Some things mentioned in #4556 included:

• Honoring users wishes for themes and plugins. RTD supports plugins entirely although the process could be clearer. RTD had support for themes although bugs were introduced with the changes to mkdocs.yml in 0.17. After Updates and simplification for mkdocs #4556, it should support different themes better.

[waylan]

• Search Indexing RTD was using the json builder (removed in 0.17, see Remove MkdcocsJson builder #4205) in order to index for searching. It would be nice to have this or something like it as opposed to crawling/indexing the HTML.

MkDocs still provides a JSON file which lists all pages with their content as part of the search plugin. Obviously, RTD does not need lunr.js and supporting stuff, but the plugin is configurable to only provide the JSON data. That said, the configuration is intended to be defined from within a theme's config, so if RTD honor's users theme choices, it would be difficult to override that config. And the JSON data is actually broken up into header sections, not complete pages, which may or may not work with your search implementation.

However, it shouldn't be too difficult to clone the relevant parts of the plugin for RTD's purpose. As plugins can inject into themes, you can even use your plugin to override the versions.html template, or to add your own JavaScript, or whatever other theme mods you may need. In other words, I believe that, with a single plugin, RTD could easily work with any third party MkDocs theme as long as you document the hooks theme authors need to provide. See the docs for details.

The one tricky part is getting your plugin included in the config. By default an undefined (by the user) plugins setting results in the search plugin being included. However, if any user defined plugins are included, then only the user defined plugins are used (it is not additive). Therefore, you would need to be careful to not loose the user included plugins when adding your plugin. Except that if the user explicitly included search, you would also need to remove that one from the list.

• Format stability Stability in the mkdocs.yml format especially around themes. There were quite a few backward incompatible changes here from 0.15.x to 1.x. Because RTD supports multiple versions of MkDocs, this was an issue. This may be less of an issue once we stop overriding the theme.

Understood. However, those backward incompatible changes were all part of pre 1.0 development. Now that 1.0 is released, we intend to maintain stability throughout the 1.x series (2.0 is not yet on our roadmap). Future changes should be primarily additive and continue to work with the existing config format.

[waylan]

Things MkDocs needs from RTD

In addition to the item listed above, we also need:

• Clear documentation of what RTD depends on. Ideally, there would be one location which would list all of the hooks/APIs that RTD uses. For example, if such a list existed, we might not have removed the json builder, as we were of the impression that it was unused. Yes, this information is available by reading the code, but that's not practical every time we need to make a quick check. Note that the requested documentation is not for end users, but for MkDocs developers, Theme developers, and possibly plugin developers. It might not fit into your public documentation, so a wiki page or the like would be fine.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stsewd]

@waylan thanks, sorry for the so late reply. We are now focusing on improving the MkDocs experience on RTD, we are going to be indexing search data from the search/search_index.json file. So it would be nice if any change to the structure of that file would be kipped or versioned if there are changes. I'll be writing the document you mention in the next days/weeks.