Opening up the docs.ansible publishing process

[oraNod]

Summary

The Ansible community team plans to open up the publishing mechanisms for Ansible core and collections documentation. The end result of this effort should be that control of the full lifecycle for content on docs.ansible and docs.testing.ansible is visible and available to the Ansible community.

In submitting this topic I would like to ask the Ansible community their help in identifying the best solution.

In a nutshell the current process is to use Jenkins and do a sphinx build then rsync to a web server.

[oraNod]

I don't think a GH action would be a good option because sphinx is a bit of a memory hog. The job to run the package build for Ansible 7 took 45 minutes to complete.

One option might be to set up a publicly accessible Jenkins instance, create an Ansible community group with admin access, and migrate the jobs.

[briantist]

GitHub Actions has larger runners available (not for free): https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners

In addition, free GHA can still be used with self-hosted runners, so the machines used for Jenkins workers now, could be registered as GHA runners instead.

I am familiar with Jenkins (though don't particularly like it), I'm sure I'm not the only one (not to mention all the RH folks who already manage it), so the public Jenkins instance could also work... might be the fastest transition option since the jobs already run there, and then other options could be explored later.

[samccann]

fwiw 45 minutes is not that bad a time. typical jenkins docs builds are 35-40 minutes on average today.

[samccann]

If going with GHA allows us to do things like trigger docs builds per docs PR so people can see the end results up on test or something, that would be a bit plus in favor of going to GHA.

[acozine]

Two questions:

• where do we document the publishing process?
• how do we track who published the docs and when?

We probably need some kind of login process for the "who published these last" part. We could consider adding this information in the footer of the docs themselves, I think we have some data there already.

[oraNod]

@acozine I've added some details here. Really it's not so much the full process but just the Jenkins jobs: https://hackmd.io/RsBL0CSuTwGHZ59lOJug6Q

In the Jenkins jobs there is a log with the kerberos ID of the person who kicks off the build and whether or not it was for production.

If we use a GH action, you'd need to be logged in. For instance, check out this workflow in the AAP docs repo: https://github.com/ansible/aap-docs/actions/workflows/sync_docs_lint.yaml

Can you run that workflow manually? It's safe to do and you won't break anything.

[samccann]

I don't see a run workflow button -

[oraNod]

Here is the groovy script for the docs publishing job in Jenkins: https://gist.github.com/oraNod/394d8b97d983e3d75129bb73ae6d7aab

I'm working on converting this to a set of gh actions. I've been debating it but think the actions should go in the ansible/docsite repository.

[oraNod]

I've created an action to build the package docs, which is essentially running "make webdocs" on a runner. So we'll probably need a "not free" option. I need to look into the self-hosted runner option more but I don't think it's feasible to use the Jenkins host for that.

Another option might be to try building package docs in a batch of smaller, async jobs.

[oraNod]

I'm also curious if anyone has tried caching pip dependencies with https://github.com/actions/setup-python#caching-packages-dependencies

This might be beneficial for smaller, more frequent doc builds though.

[oraNod]

Finally got build success on a self-hosted runner. AWS ec2 m5.xlarge. I tried different instances and using the CPUS flag with make.

Job: https://github.com/oraNod/docsite/actions/runs/3929877123/jobs/6719204771

Also used RHEL 9 with Python 3.9. For building older versions of the package docs we might need to set up older python versions. @samccann Do you have any idea?

[oraNod]

Here is a screengrab of the action with the job inputs:

[samccann]

@oraNod - I would say it's fine for now to have only python 3.9 etc. We only actively publish the most recent release (as latest) and devel. I'd say we add a python option if/when we need it.

[felixfontein]

3.9 should work fine down to Ansible 2.9 (or even more, if someone actually still wants to build that). But 3.10 will be necessary for ansible-core 2.16 / Ansible 9.

[oraNod]

Thanks @samccann and @felixfontein

[samccann]

Closing in favor of ansible-community/community-team#497