GitBook to Antora migration

[msavy]

I keep forgetting about this so writing it down here.

Looks like GitBook are archiving their legacy service on 30th April, meaning that it won't regenerate automatically for us any more (but will continue to be hosted).

Unfortunately, their new services does not appear to support Asciidoc for whatever reason, and we make extensive use of Asciidoc features throughout the documentation.

However, I think there is a clear solution: Antora.

It was written by Dan Allen (Asciidoctor creator & maintainer), as an alternative to GitBook that has first-class Asciidoc support.

The only downside is that we will need to host it ourselves, I think. That should not be a major hassle, really.

Here's a short blog-post from someone who migrated -- https://blog.yuzutech.fr/blog/from-gitbook-to-antora/index.html

Here are the Antora docs -- https://docs.antora.org/antora/2.2/

Here are some examples of Antora in other projects -- https://docs.couchbase.com/server/4.1/introduction/intro.html

I will do some further reading and tests this evening to progress things and see whether there are any edge cases.

[volkflo]

@msavy Just a quick thought about this, maybe it is easier to try to convert asciidoc to markdown like explained here: https://tinyapps.org/blog/201701240700_convert_asciidoc_to_markdown.html

[msavy]

Worth considering, but we are using many features that are in asciidoctor that are not in markdown.

[volkflo]

Do you have an example?
I just take a quick look at this https://apiman.gitbooks.io/apiman-installation-guide/ but did not saw any special things :)

[msavy]

User guide a better case, I suspect.

Off the top of my head:
Code references/annotations (aka callouts), tables with complex attributes (header, width specifications, etc), important/tip/info/warning blocks (aka admonitions), Footnotes.

Not saying it won't be possible, but will be difficult to change over.

Please do experiment, though.

[msavy]

BTW apologies for slow responses and short messages, but Apiman is extracurricular largely for me now. So just doing things as-and-when I can spare time!

[volkflo]

Totally understandable, I think apiman is also hard if you work fulltime on it :)

I just tried to convert the policies.adoc from the user guide to markdown.
The result is here: https://gist.github.com/volkflo/0e86cf735c7b03ba0f78de51ebe5af6b

I think it is not so bad for a quick convert :)

The broken code blocks should be easy to fix

[EricWittmann]

Not bad - the only thing that jumped out at me when skimming the result is the Sample Configuration sections.

[msavy]

I had a little attempt at porting to antora this evening, and it worked very well. With a small amount of additional effort I'll push up something onto a GH repo for us to preview.

Just a screenshot to tease (left vs right menu not a fixed thing). Doesn't show everything; for example, there's a menu on the left also for your top-level nav items.

Anyway, good experiment! Will get back to you shortly with more so we can compare our options.

[volkflo]

Not bad - the only thing that jumped out at me when skimming the result is the Sample Configuration sections.

Yeah, the problem here is that the conversion escapes the markdown syntax:

\`\`\`json { "rules" : \[ { "pathPattern": "/admin/.**", "verb": "**",
"role": "admin" }, { "pathPattern": "/.\*", "verb": "GET", "role":
"user" } \], "multiMatch": true, "requestUnmatched": false } \`\`\`

if this is done right we got

 {
 	"rules": [{
 		"pathPattern": "/admin/.**",
 		"verb": "**",
 		"role": "admin"
 	}, {
 		"pathPattern": "/.*",
 		"verb": "GET",
 		"role": "user"
 	}],
 	"multiMatch": true,
 	"requestUnmatched": false
 }

But the other tool seems to work also pretty well :)

[msavy]

I've pushed up a rough version just so we can have a look and figure which approach is better before investing any more time.

https://msavy.github.io/test-apiman-docs/public/apiman-user-guide/latest/gateway/policies.html#_basic_authentication_policy

This is the default theme without any customisations (it's trivial to do). There's also a super-cool free Algolia search function that we can use as a FLOSS project (see the search box here -- https://docs.antora.org/antora/2.1/playbook/configure-ui/#default-ui-output-directory)

Another interesting Antora feature is that you can generate the docs against tags and branches also, and that will be linked in. Would be cool, but I doubt we want to go to the effort of reworking all previous release docs. But you can see that on the antora docs at the bottom left where you can select a specific version to view.

NB: if we go ahead with this, git history will be preserved. This is just a rough proto. We would just use GH actions to automatically update in the same manner GitBook does.

Also, I'm not familiar with new GitBook. It may have some cool new features? What do you all think?

One caveat for both approaches is that we possibly need something for switching between 'properties' and 'JSON' samples for deployment guide. I think a simple addon in Antora would do that but I've not looked yet; alternatively we just show both at once!

[msavy]

Forgot to say, I didn't bother fixing up the variable substitutions yet -- but that's a trivial job.

I am 100% open-minded on this; interested to hear thoughts.

[EricWittmann]

This looks really nice at first glance!

[msavy]

@volkflo any thoughts?

[volkflo]

@msavy I think going with antora is a good choice. I briefly looked at the default UI they offer (https://antora.gitlab.io/antora-ui-default/) and this looks pretty good for me :)

I personally prefer markdown over asciidoc but I can understand that we do not want to migrate all of the documentation.
Also, I don't see any must-have features on gitbook website :)

[msavy]

Florian is taking my PoC and will develop it into something that should be ready for automation and production-grade.

We've mapped out a plan that we think should work very well for our use-cases.

[msavy]

See also https://github.com/apiman/apiman-docs-antora-playbooks

[volkflo]

I would like to remove the Production Guide](https://github.com/apiman/apiman-docs-production-guide) and merge it into the installation guide (maybe a section for prod deployments)

Any concerns about that? :)

[msavy]

Seems OK to me 👍

[msavy]

We've made some really nice progress on this both in terms of automation and getting something generated. Not far from being ready to deploy.

@volkflo has done a good job taking the PoC to a production-ready state, and I've added a few bits of automation around it too. I also want to auto-trigger website rebuilds when docs repos are updated, so that's on my list.

You can check it out on: http://www.apiman.io/apiman-docs

• I need to create a table & intro for the overview page
• All variables need to be ported over from GitBook format.
• Prepare to update version numbers via CI
• Add icons to admonition blocks (style thing) will do some other time
• Trigger docs GitHub Actions (rebuild & redeploy) when separate docs repos are updated. This will likely be via repository_dispatch.
• Cut branch for 1.x docs vs 2.x/latest docs?

[msavy]

Have added a ytt (https://carvel.dev/ytt) overlay that will allow update of version numbers via CI rather easily. However, this would require the apiman-docs-user-guide repo to know when a release has been done, so we can either do that by repository dispatch or add that to the release scripts.

https://github.com/apiman/apiman-docs-user-guide/blob/antora/antora-overlay.yml

This works by:

./ytt -f antora.yml -f antora-overlay.yml -f project.yaml

antora.yml is the original antora file
antora-overlay.yml is the ytt overlay
project.yaml is the project metadata file from apiman/apiman in ~/.github

it simply writes out the result to console/file and we commit that back to the repo

❯ ./ytt -f antora.yml -f antora-overlay.yml -f project.yaml
name: user-guide
title: Apiman User Guide
version: latest
start_page: ROOT:index.adoc
nav:
- modules/ROOT/nav.adoc
asciidoc:
  attributes:
    apiman-version-release: 2.0.0.Final
    apiman-snapshot-release: 2.0.1-SNAPSHOT

[msavy]

Ok, here's the plan for the final bits:

• We will create 1.x branches, and will name them in the docs as the latest version (1.5.7.Final and 2.1.0.Final for master/main)
• At some some later point we will automate the renaming to always track the latest release version using the project.yaml file on the corresponding branch.
• For now we'll do some of this stuff manually, but the YTT overlay is ready to do this automatically for all subsequent releases 👍.
• Algolia search (see Add Algolia search to docs #1349)