Implement versioned docs

[BClark09]

Hi all, I found no issue discussion on how the docs would be separated into versions, but it comes up in discussion from time. I've done some playing around based on this website:

The above site is built with no modifications, but with the following script: (For an actual server, you'd only need to build, not perform the initial serve.)

#!/bin/sh
git fetch --tags                                                            ## Currently, releases are kept as tags. Not stored in branches.
git checkout gh-pages                                                       ## Checkout the latest changes first.
jekyll serve --incremental --detach                                         ## A build will be fine, for an actual server.
tagsList="$(git tag --list [0-9].[0-9].[0-9])"                              ## Iterate through versions only.
echo "$tagsList" | while read version                                       ## For each version tag...
do
  git checkout "tags/$version"                                              ## Set contents to version.
  echo "baseurl : /$version" > _config.$version.yml                         ## Programatically make a new config
  jekyll build --config _config.yml,_config.$version.yml -d _site/$version/ ## Store the contents in [ip]:[port]/[version]/
done
git checkout gh-pages                                                       ## Continue with master branch.

Based on the above, I have the following suggestions:

• Make a branch for each version, instead of a Tag, so that changes can be backported, (e.g. adding a dropdown to each version).
• {% assign base = site.baseurl %} needs to be used in _includes/base.html.

Would this be the best way of doing it? Did anyone else have suggestions on how to implement it?

[ThomDietrich]

Hey @BClark09,
you should have asked before going through all that trouble. I've got something pretty similar in a test-setup of mine :) I wanted to finish #516 before continuing. Nonetheless, this looks pretty good!! 👏

You didn't find any discussion on the issue as this was so far only discussed in a non-public thread (which you are part of I believe). In this thread we were discussing different ideas with @kaikreuzer and @martinvw. The originally proposed solution was less elegant and I've since realized that it is not applicable.

I'm currently in the office and can't compare with my solution but the key ideas seem to be the same.
I came to the same conclusion regarding tags vs. branches, with the exception, that it would be interesting to discuss if these systematic backports need to be committed or can be performed on-the-fly on top of the tags idea. With the right set of actions that should be pretty easy to maintain in contrast to maintaining dozens of version branches in parallel. I've invested some additional thought into how we could propagate improvements to the master branch down the version branches conditionally. That would include version-independent files like the version dropdown or sections that should exist without versioning. I believe that's what you already planned for when you mentioned "backported changes".

Regarding next steps. I promised @kaikreuzer to look into the issue a few weeks ago, a few weeks too many. I'm currently preoccupied with a more pressing issue in my personal life and would be happy if we can work on this task together 😉🍻

[BClark09]

you should have asked before going through all that trouble.

There was no trouble really, I came across the website in my own research of Git usage and tested that script. I was pleasantly surprised to see that it just worked with the only input from me being the {{base}} variable change.

You didn't find any discussion on the issue as this was so far only discussed in a non-public thread (which you are part of I believe)

errrr 🙈

That would include version-independent files like the version dropdown or sections that should exist without versioning. I believe that's what you already planned for when you mentioned "backported changes".

The versioned files have their own directory in /_site/, when built and depending on which version you're looking at, the {{site.baseurl}} will be different. I'd take a good guess that having a {{sharedbase}} variable that points to "/" might actually be a good thing here". Therefore, there'd be no need to pull something like a version dropdown into separate branches, only in master.

and would be happy if we can work on this task together 😉🍻

Of course, I'm in a fairly busy period also but can always try to help where I can!

[ThomDietrich]

To add some more thought to this issue:

openHAB is constantly evolving and the documentation repo is keeping track with that. The official documentation page will soon contain a root documentation and a certain amount of sub-directories for certain versions. A user will normally want to stay on the root path.
@BClark09 I'd suggest to have the documentation of the latest stable version on the root level:

• https://docs.openhab.org/v2.0.0/... - 2.0.0 tag based
• https://docs.openhab.org/v2.1.0/... - 2.1.0 tag based
• https://docs.openhab.org/... - also 2.1.0 tag based
• https://docs.openhab.org/snapshot/... - latest (2.2.0-SNAPSHOT) gh-pages content

We can furthermore suppress search engines and users to end up in subdirectories via robots.txt.

I'd take a good guess

That sounds pretty good!

@kaikreuzer I should be able to start with this next week. The only real issue is, that we'll not be able to host this on GitHub... I can do first tests on a server of my own.

[BClark09]

Yeah I agree with all that. I noticed (but can't remember where) someone having problems with trying to implement a snapshot feature (a breaking change) because they were reading it from the docs.

I do think that we should branch off of the tags and use branches instead of tags though. WDYT?

[ThomDietrich]

it would be interesting to discuss if these systematic backports need to be committed or can be performed on-the-fly on top of the tags idea.

I'm not sure yet. Branches would not hurt but maybe tags are already sufficient. I fear that it might become tedious to manage changes to all existing old-version branches. All changes we need to apply to files of version-subdirectories will be systematic and can hence be scripted rather than stored in commits. Maybe I'm missing something? Wdyt?

[BClark09]

That's a good argument for it, but what would be done about small problems like broken links etc?

Also I think it would be necessary to have a 404 page for the old versions which point towards / or /snapshot/

[kaikreuzer]

I'd suggest to have the documentation of the latest stable version on the root level

In our private discussion, we said it might be good to have the snapshot (as a moving target) on the root and put "static" releases into sub-directories:

"While I agree that (hoping and assuming that we get to a more frequent release cycle) the normal user should see the latest stable documentation, I think that doing redirects for all pages can be avoided if we declare the "root" version to be the snapshot (and thus the one that is currently being worked on - just as it is now). Upon a release, this can then simply be copied to a 2.x sub-folder."

The only real issue is, that we'll not be able to host this on GitHub...

That would be a MAJOR drawback. Why is that so? From our discussion so far, we said that released versions are simply copied to a sub-folder (which can be automated to be copied from a tag/branch) in the master branch. Clearly not ideal from a git perspective, but the most efficient way for hosting.

[ThomDietrich]

At the Smart Home Day me and @kaikreuzer had a short discussion regarding a dedicated machine or container for the documentation page. His opinion was that this is currently not desirable. I agree that this will be extra work and an additional maintenance risk.

Let me try to discuss all options.

Dedicated System

The hosting aspects of GitHub pages are pretty restrictive. I'll shortly summarize the opportunities of a dedicated system:

• Jekyll Addons
• Multi-branch/tag deployment of docs versions
• Additional steps during deployment
• External resources retrieval during deployment (Good bye _addons folders and "Update Generated Content" commits!)
• Automated external resources retrieval (daily/...)
• Ability to host multiple subdomains (e.g. blog.openhab.org) (a prerequisite for you know what)

GitHub Pages

If we stay with GitHub pages, we are limited by the fact, that GitHub pages only deploys what is in the gh-pages repository branch. The _site folder is purged upon generation and the _addons folders can not be moved to subdirectories due to their jekyll collections nature.
In conclusion, I believe the only viable solution would be to commit (to gh-pages) the v2.x.0 page version as the resulting html pages to the subdirectory ./2.x.0 I believe these will then be copied over to _site untouched (I would need to do a few tests). Please propose a better option if you know one. The whole task would in the end probably be pretty easy to "implement" but honestly everything but cool.

Switching Control

The resulting page should of course present an option to switch between versions. I imagine a div above the content consisting of a text element and a combo box. The combo box will offer all available versions, selecting one will bring the user to the current article of the selected version. The text to the left will either say "You are currently browsing the documentation of an old openHAB version" or "...of the latest SNAPSHOT version".

Website Overhaul

With the long-term plans (I'll try to propose a first requirements list this weekend) to redesign the docs page to include the openHAB main page as well as a new blog page and possibly other subsection, deployment of multiple branches or even repositories might become interesting. The support of multiple subdomains will become a requirement. This will only be possible with the dedicated setup.

[ThomDietrich]

@kaikreuzer regarding the "moving target": I believe that the "default" non-version path of the documentation should point to the... well default. So if possible, shouldn't that be the stable release?

[kaikreuzer]

The support of multiple subdomains will become a requirement.

Not necessarily. Note that on www.openhab.org we have main menu links to community.openhab.org, github.org and myopenhab.org as well - these won't be served from the same server either. What we mainly need are common stylesheets and headers, but I do not see the need to merge the hosting. Especially wrt the versioning discussion in this issue, I very much think that we should keep docs.openhab.org maintained separately from other parts.

I believe that the "default" non-version path of the documentation should point to the... well default. So if possible, shouldn't that be the stable release?

I thought to remember that we once agreed that the entry "default" path (docs.openhab.org) should simply be used as an alias that will forward to docs.openhab.org/2.1 today and docs.openhab.org/2.2 tomorrow, while the latest snapshot is where it is today.

[ThomDietrich]

Regarding subdomains: As discussed privately, I was under the impression, that we are striving towards a project that contains not only the documentation (as of now) but also the main webpage, the blog and possibly other sub-areas in one repository. I'm open to discuss other approaches where we keep all of those separate. I'm not sure if a separation of all those and the synchronization of stylesheets etc will bring any benefits in the long run, they will for sure complicate development and outside contributions.

Regarding the default: I believe we are talking about the same thing. I was talking about the end result, you are talking about the repository structure.

• Yes, the root of the repository should contain the latest snapshot revision of the docs.
• Yes, the root of the web page should present the latest stable version of the docs.

Could you please also comment on the rest of the previous message?

[kaikreuzer]

Could you please also comment on the rest of the previous message?

Err, which parts exactly? You mean dedicated system vs. Github pages? Imho, we should try the approach you just now did with #552. I agree that it isn't nice to completely go against the history + versioning in git and just plainly copy all content, but if it helps us avoiding to run a dedicated server, it should still make our lives easier.

[BClark09]

@ThomDietrich, thanks to your work, this can be closed now right?

[Confectrician]

Correct. thanks for notifying. 😄