ansibulled docs design plan

Progress

• Website documentation command in ansible/ansible repo
  • Implement the backwards compat interface for full website deploy
  • Need this for people to start testing and iterating on the "product"
  • Will need other modes of operation for people to efficiently develop the website in the future
  • Will need other modes of operation for testing of the website in CI
• ansible-test build-website test
• Fix any ansibulled commands that were broken by these changes
• Build docs for devel (Possibly later PR)
  • Parse the deps from an ansibulled .in file
  • Download the latest versions of those collections
  • Git clone the latest version of ansible-base
• Build docs for a collection (Later PR)
  • ?
• Build docs for a single plugin (Later PR)
  • ?
• Write tests (Later PR)
• Documentaation of ansibulled itself (Later PR)

Website builder

• Lives in ansible/ansible
• Plugin of hacking/build-ansible.py combined with Makefile targets and sphinx build script
• normally called from make webdocs, and other make targets

Use cases

How things used to work

As a test case

1. CI calls make singlehtmldocs on a fresh checkout
2. the Makefile calls build-ansible document-plugins on the plugins
3. the Makefile calls build-ansible to generate the other rst
4. the Makefile calls sphinx to build a single html page

As a deployment of the website

1. Jenkins calls make webdocs on a fresh checkout
2. the Makefile calls build-ansible document-plugins on the plugins
3. the Makefile calls build-ansible to generate the other rst
4. the Makefile calls sphinx to build the html

For a user testing the website

1. User calls make webdocs
2. the Makefile calls build-ansible document-plugins on the plugins which have changed
3. the Makefile calls build-ansible to generate the other rst
4. the Makefile calls sphinx to build the html for static site and things which have changed

For a user testing changes to specific modules/plugins

1. User calls make webdocs MODULEARGS='foo bar'
2. Makefile calls build-ansible document-plugins limited to foo and bar of type module
3. build-ansible generates rst for foo and bar if they have changed
4. Makefile calls build-ansible to generate other rst
5. Makefile calls sphinx to generate the html for the static site and those modules which have changed.

Backwards compat

Are all of the previous cases important to maintain verbatim? Maybe they can be done better/differently now.

As a deployment of the website

1. Jenkins calls make webdocs on a fresh checkout
2. Makefile parses the version that it is from the ansible-base releases file
3. Makefile copies the deps file for the version of ansible-base
4. the Makefile calls antisbull-docs to create rst for the plugins
5. the Makefile calls build-ansible to generate the other rst
6. the Makefile calls sphinx to build the html

As a test case

1. CI calls make singlehtmldocs on a fresh checkout
2. the Makefile generates a deps file which only contains the dependency on ansible-base at the version in the current checkout.
3. the Makefile calls antsibull-docs with that deps file and path to the existing ansible install to generate plugin rst for that setup.
4. antsibull-docs needs to have the ability to be given a path to the ansible source.
5. the Makefile calls build-ansible to generate the other rst
6. the Makefile calls sphinx to build a single html page

For a user testing the website

1. User calls make webdocs
2. Makefile checks out ansible-build-data
3. Makefile calls antsibull-docs with the deps file from the build-data for the present version of ansible to create the plugin rst
   1. Currently antsibull-docs will create all of the plugin rst every time.
   2. Will need to implement saved state to tell whether this has changed:
   3. Record version numbers of collections so they don't have to be re-downloaded
   4. Hash of individual plugins so we don't have to re-run them if they haven't changed
   5. Save this in the rst/collections/ directory so it can be used by docs build as well? (docs build would download it from the website)
4. Makefile calls build-ansible to create the other rst
5. Makefile calls sphinx to create the html for the changed files

For a user testing changes to specific modules/plugins

1. User calls make webdocs MODULEARGS='foo bar'
2. makefile parses MODULEARGS into individual modules
3. Makefile assembles paths to each of those modules
4. Makefile calls antsibull-docs on those specific modules foo and bar of type module
   1. Does this work for modules in collections?
   2. No: ansible-base only:
      1. antsibull-docs needs to have the document-plugin-file mode implemented
   3. Yes: collections can be specified as community.general.zypper
      1. Makefile needs to parse the collection name out of each module
      2. Makefile needs to download each collection or antsibull-docs needs to gain an option to limit its work to specifically named plugins
      3. Makefile assembles the path to each plugin in the collection.
      4. antsibull-docs needs to have the document-plugin-file mode implemented
5. Makefile calls build-ansible to generate other rst
6. Makefile calls sphinx to generate the html for the static site and those modules which have changed.

New ways of enabling the old use cases

Moving more of the logic from the Makefile to build-ansible would let us more easily implement more complex use cases (like building only a few modules from a collection).

Build the whole website (jenkins)

• Called by jenkins to build the website
• Builds collections and ansible-base documentation

Build the whole website (user)

1. User runs make webdocs
2. make webdocs invokes build-ansible.py
3. build-ansible.py runs antsibull-docs to extract rst from all of the files
4. build-ansible.py invokes sphinx
5. website is built.

Called by developer to iterate over docs they are writing

• Developer writes some new content
• Invokes a make target to build that subset of docs

1. User changes the documentation in three plugins
2. User calls make webdocs MODULEARGS=foo,bar,baz
3. Make runs build-ansible.py website --module foo --module bar --module baz
4. build-ansible.py builds the rst for those
5. build-ansible.py builds the website along with just those.

New workflows

User working on static site content

1. User writes new static site content
2. User calls make webstatic
3. Make runs build-ansible.py website --static-site
4. build-ansible.py builds just the staticsite

Called by ansible-test to test that the website is building correctly

• in CI, ansible-test runs to build the pieces of the website that are in ansible-base
• Use antsibull to extract the pieces from ansible-base plugins
• use website build to put the rest together.

Antsibull-docs

Inputs

• deps file (when we're going to document a stable ansible release list of collections without versions (when we're going to document devel)
• Single collection (when we're going to document a third party collection)
• path to collection or collection name (When we're going to document a third party collection or test a collection [might not work with fragments?)
• path to a single file (for testing) Should we warn on fragments?

Outputs

Always rst files

• In a collection hierarchy: ** collections/community.general/FOOBAR_module.rst ** collections/community.general/index.rst ** @ Debatable: collections/index.rst; the proof of concept will include this
• A single collection: ** FOOBAR_module.rst ** index.rst
• A single file: ** FOOBAR_module.rst

Responsibilities of other tools

• @ Debatable: construction of collections/index.rst; this will be in ansibulled-docs for the proof of concept
• turning the rst into a website (website builder in ansible-base will do this for us. Third parties will use sphinx and link what we generate into an upper level index.rst) that give an overview)
• Generate a list of redirects. This will need to parse routing.yml in ansible-base and collections and generate apache redirect from the ansible-base name to whatever the newest name is. I'll write a different script for this
• Determining the collections to be used (This is done by ansibulled when it builds a collection)