Versioned Docs

[lhstrh]

As things change, we have a challenge to keep the documentation up-to-date. At the same time, users who are slower to adopt new versions may want to check out the docs of older versions.

These to things combined suggest to me that it might be better to keep the core docs (i.e., the handbook) closer tied to tags in this repo. One way to do that is to literally move the markdown for the handbook into this repo under a docs folder. We could then use something like the mechanism described here to let users chose what version of the docs they want to peruse on the website.

I'm not at all sure how difficult this would be to pull off, or whether there are obvious drawbacks to the approach I outlined, or whether there are better ways to approach this, but I'm curious to hear what others think.

Another approach that I can think of is that we dedicate special branches of the website repo to particular versions of the docs (and name them after the release they are documenting)...

[axmmisaka]

A temporary solution is to keep separate docs for each minor version, and add deprecation notice for older versions.

While TS docs doesn't have versioning built-in, it does have two versions of handbooks, located in different parts of the website: https://github.com/microsoft/TypeScript-Website/tree/v2/packages/documentation/copy/en/handbook-v2 (see the two directories)

If they can have 2, we can have 5 ;^)

We could simply maintain multiple directories, each for one minor version. It will eventually be a mess (I guess), but I think it is fine as of now as everything is unsettled and we are still developing rapidly, where backward compatibility could break in any nightly. Once that has became a huge issue for us, we will bump major version and only major version docs should be there.

Alternatively, we could nest a gatsby theme specifically made for docs with versioning and stuff, such as this. I don't like this idea, because it's too much work and might break a lot of things for something unnecessary as of now.

[edwardalee]

It seems to me that a much simpler solution is to update the website in a branch, say release-0.6, and just before merging it into main, copy the generated website, which is a bunch of static HTML pages, and create a static archival website, say, lf-lang.org/release-0.5. The release-0.6 version is available to developers by cloning the repo and running yarn locally.

[lhstrh]

That would be easy for us, but not very convenient for users. Ideally, there would just be a drop down to select the version of the docs...

[axmmisaka]

Personally, I don't see much of a difference, other than cosmetics (for ease of access, we can link them to corresponding articles).
Maintainability might be an issue in the long run. Yet, if we want to do versioning, we might be better of switch to a doc platform that supports it so we don't reinvent wheels.

Python documentation, for example, does this with a simple URL switching.

  function on_version_switch() {
    var selected_version = $(this).children('option:selected').attr('value') + '/';
    var url = window.location.href;
    var current_language = language_segment_from_url();
    var current_version = version_segment_from_url();
    var new_url = url.replace('/' + current_language + current_version,
                              '/' + current_language + selected_version);
    if (new_url != url) {
      navigate_to_first_existing([
        new_url,
        url.replace('/' + current_language + current_version,
                    '/' + selected_version),
        '/' + current_language + selected_version,
        '/' + selected_version,
        '/'
      ]);
    }
  }

[edwardalee]

I don't see a difference either. A drop-down vs a hyperlink? It's just a small amount of JavaScript

[cmnrd]

I think including the docs (plain markdown) with the lf repo is a good idea. This way versioning becomes trivial, and we can make doc updates part of PRs. Would there be a simple way for generating a viewable website just from the markdown files?

Generally, I think one problem is that updating the docs is just way too complicated and everything is hidden somewhere in an odd directory structure. I am not sure if we can simplify things using the current approach. Personally, I would be in favor of using a simpler and more common tool for doc generation, such as mdbook. This also increases the likelihood that new contributors already know how to work with it. However, I don't know how much effort it would be to convert our docs once again...

Yet, if we want to do versioning, we might be better of switch to a doc platform that supports it so we don't reinvent wheels.

I can only second this.

Personally, I like particularly well the documentation of hydra: https://hydra.cc/docs/intro/, but I don't know which tool they use. There is a dropdown that lets you choose between versions wherever you currently are. The docs for the current development version are also included under the label 'next'. I think it is also important for us to have a live version of the current development version of the docs. This is because we have nightly releases, but also to encourage developers to interact with the docs and keep them up to date.

[axmmisaka]

Personally, I like particularly well the documentation of hydra: https://hydra.cc/docs/intro/, but I don't know which tool they use.

They are using docusaurus, https://github.com/facebookresearch/hydra/tree/main/website.
Alternatively we use https://github.com/apollographql/docs . They are using Gatsby so we can nest them. Alternatively we use readthedocs.

Would there be a simple way for generating a viewable website just from the markdown files?

Yes, or at least with a very trivial CI task.

@lhstrh told me that the concern of migration is that in the current website, we have a hacky target language selection plugin, and moving to different platforms might prompt us to re-do it. Honestly, I think we should instead find a more clever way to do it, such as wrapping it around a i18n provider instead of writing our own, but that's a bit off-topic.

I think including the docs (plain markdown) with the lf repo is a good idea. This way versioning becomes trivial, and we can make doc updates part of PRs. Would there be a simple way for generating a viewable website just from the markdown files?

I think it depends, if we still don't update docs then versioning and git commit history will become a total mess. I think it's okay to do versioning separately currently, and we can at least mark what has been changed but the docs hasn't been updated.

[cmnrd]

Another aspect of versioning is also providing migration guides when there are breaking changes. This is also done pretty well in the hydra docs: https://hydra.cc/docs/upgrades/intro/.

[edwardalee]

A word of caution: Yes, the website infrastructure is complex and the markdown files are buried deep in subdirectory. There is a natural tendency, when encountering such complexity, that things can be simplified by redesigning from scratch using the next greatest infrastructure. Months later, we are still discovering and fixing new bugs, still haven't gotten back to the original functionality, and have an architecture that has become at least as complicated as the original. Do we really want to spend all this time rearchitecting the website? Don't we have other higher priorities?

Things I like about the current setup:

• The pages generated are static HTML with a small amount of embedded JavaScript to handle the language specificity. As a consequence, the website is extremely fast.
• Because the pages are static HTML, I believe it would be rather trivial to create snapshot archives of the pages. Those would not be (easily) updatable, but that is kind of what the word "archive" means.
• The pages are extremely easy to update online. Just go to the bottom of the page and open a pull request.

As for the complexity of the file layout, I run code in the root directory, click on the search button, and start typing the sentence I want to fix. It takes me right to the markdown file, I fix it, and commit. I don't even need to know where the markdown file is.

The one potentially serious problem with the website that probably has to be addressed is that GitHub says it has security risks. I'm not sure whether these are in the website itself (I find this unlikely given that it serves static pages) or in the GitHub repo. But someone probably needs to take a look.

[cmnrd]

We will have to pay a price either way if we want to integrate versioning in our documentation. If we keep our current infrastructure, we have to implement a solution for versioning, document it, and maintain it. If we switch to a more common tool that already solves the problem, we have to do the initial migration. What I would like to understand is how difficult it is to implement versioning in our current infrastructure (and maintaining that in the long run) and how difficult it would be to migrate to another html generation backend. If we reinvent the wheel, then there should also be a good reason for it. Such a reason could be the drop-down for target language support, which might be challenging to implement with other tools.

The points about static html and archiving are also true for any other documentation generation tool that I know of.