Sphinx documentation. #2046

oliver-sanders · 2017-02-01T13:05:17Z

~~NOT READY FOR REVIEW.~~

This change adds API reference pages auto-generated by the Sphinx (GPL) documentation builder.

Examples provided in doctest format are tested (via Sphinx) in t/docs/03-sphinx-doctests.t.
Documentation is built within a virtualenv (sphinx is installed via pip) to avoid installation requirements.
- ~~Wrapper script for building docs can be found at doc/bin/make-docs.~~ Use make -C doc html to build.
- Options for deployment:
  - Supply the pre-built docs.
  - Supply the wrapper script to build using a virtualenv. Implemented via makefile
  - ~~Rely on user installation.~~
I've put together some quick developer notes at doc/sphinx/_build/sphinx.html (have to be built).
Thus far I have documented rose.config.*, rose.macro.MacroBase, rose.macro.MacroReporter.
Sphinx has a built-in search feature!

At present Sphinx is being used for its autodoc feature but is capable of building entire sites (e.g. www.docs.python.org).

~~Currently the default Sphinx theme is being used (same as the python2 docs), other themes are available, see in particular~~:

alabaster Current Theme
~~readthedocs~~
~~sphinx-bootstrap~~

There are also two small behaviour changes brought in by this pull:

Rose macro will now exit upon encountering a macro without a docstring line (before there was traceback)
The kwarg keys in the rose.config.ConfigNodeDiff.(set_added_setting, set_modified_setting, set_removed_setting) methods can now be provided as a list as well as a tuple for consistency with rose.config.ConfigNode.

~~A bit of polish is still required - putting this up for comment @metomi/core.~~

~~Building of docs needs to be automated in our git workflow~~ Ditched Travis build/deploy scripts
~~Build scripts require improvement~~ Transitioned to makefile

oliver-sanders · 2017-03-21T16:19:04Z

Updates:

Documentation themed with alabaster, a plain minimalist template (happy to change).
Build scripts migrated to a Makefile, run make -C doc html [doctest] to build.

Info:

Documentation is built to doc/sphinx/_build.
Some notes on Sphinx and docstrings can be found at doc/sphinx/_build/sphinx.html (have to be built).
Sphinx pages cannot be included in the single page documentation so I have written a place-holder for the Config API page.

Automated deployment:

It if possible to auto build/deploy via Travis-Ci but it requires giving Travis [force] push access via a user token (so that it can nuke the gh-pages branch).
For completeness the following is an outline of how to do this:

travis.yml

install:
    ...
    - pip install sphinx

script:
    - make -C doc doctest html && deploy-script "${PWD}"
    - rose test-battery ...

deploy-script

#!/bin/bash

ROSE_DIR=$1

# Only deploy on merge to master.
if [[ "${TRAVIS_BRANCH}" != 'master' || \
      "${TRAVIS_PULL_REQUEST}" != 'false' ]]; then
    # Documention should not be deployed.
    echo 'Skipped Deployment.'
    exit 0
fi

git clone 'https://github.com/davisp/ghp-import'

echo 'Deploying documentation ...'

# Place docs in "doc" directory for historic reasons.
echo -e '\tSymlinking documentation into site structure...'
GHP_DIR="$(mktemp -d)"
ln -s "${ROSE_DIR}/doc" "${GHP_DIR}/doc"
ln 'index.html' "${GHP_DIR}/index.html"

# Push new documentation.
echo -e '\tConstructing gh-pages branch...'
./ghp-import/ghp_import.py -ln "${GHP_DIR}"

echo -e '\tpushing gh-pages branch...'
git push -fq "https://${GH_TOKEN}@github.com/${TRAVIS_REPO_SLUG}.git" gh-pages

echo -e '\tDeployment successfull.'

setup:

Generate a github user token with public_repo authorisation.
Create an environment variable in Travis called GH_TOKEN, git it the value <user>:<token>.

oliver-sanders · 2017-03-22T09:32:15Z

Note the test battery now builds the docs.

matthewrmshin

2 minor issues:

venv/ directory not cleaned up.
t/syntax/01-doc.t failed in my environment.

Looks good otherwise.

matthewrmshin · 2017-03-23T11:52:39Z

Test not happy.

oliver-sanders · 2017-03-24T12:41:24Z

Test now ecstatic.

matthewrmshin · 2017-03-28T11:15:02Z

@arjclark please sanity check.

matthewrmshin · 2017-03-29T08:49:24Z

lib/python/rose/config.py


        """
-        node = self.load(source, node)
+        if not node:
+            node = self.load(source, node)


Just noticed that this change will break loading of site/user global configuration. After this change, user global will never get loaded because node will be defined. Please revert.

* Fixes traceback for a rose macro without a docstring

arjclark

Various minor comments. @matthewrmshin, as dicussed, can you double check the docsting entries for confignode are consistent with behaviour?

arjclark · 2017-04-05T10:07:35Z

doc/rose-api.html

@@ -785,7 +785,8 @@ <h2 id="macro">Rose Macros</h2>
              e.g. adding/removing options.</li>

              <li>Upgraders - these are special transformer macros
-              for upgrading and downgrading configurations.</li>
+              for upgrading and downgrading configurations. (covered in the
+              <a href="#upgrade">next section</a>)</li>


Can you describe this as the "upgrade" section rather than "next"

arjclark · 2017-04-05T10:10:26Z

doc/rose-config-api.html

+      </div>
+
+      <div class="panel-body">
+        <p>View the API docs <a href="rose-config-api.html">here</a>.


Please close the <p> tag for consistency

arjclark · 2017-04-05T10:15:49Z

doc/sphinx/conf.py

+
+# General information about the project.
+project = u'rose-api-documentation'
+copyright = u'2017, Met Office'


Why do some strings in here have the unicode markup while others don't?

This is the way Sphinx auto-generates them, I don't imagine it's necessary though.

arjclark · 2017-04-05T10:17:30Z

doc/sphinx/sphinx.rst

+
+        """  # Blank line.
+
+        def __init(self, param1, param2, kwarg=None):


Shouldn't this be __init__?

arjclark · 2017-04-05T10:22:16Z

lib/python/rose/macro.py

+
+        Args:
+            filename (str): The filename of the resource to request the path
+                to.


Alignment of this looks odd - maybe in line with the "T"

This is the google docstring convention (overflow lines should have an extra 4 space indentation) see the guide I wrote in doc/sphinx/sphinx.rst and also http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html.

arjclark · 2017-04-05T10:57:03Z

lib/python/rose/config.py

@@ -459,13 +996,36 @@ def get_reversed(self):
        return rev_diff

    def delete_removed(self):
-        """Deletes all 'removed' keys from this diff."""
+        """Deletes all 'removed' keys from this ConfigNodeDiff..


Only the one "."

arjclark · 2017-04-05T11:01:23Z

doc/rose-api.html

@@ -963,6 +970,8 @@ <h4 id="macro:code:api">API Reference</h4>

            <pre class="prettyprint">
    def report(self, config, meta_config=None):
+        """ Write some information about the configuration to a report file."""
+        # Note: report methods do not have a return value.


Can you combine the docstring and the immediately following # comment

arjclark · 2017-04-19T08:54:55Z

Assuming @matthewrmshin has confirmed the ConfigNode docstring entries are consistent with behaviour then good to merge.

oliver-sanders added this to the soon milestone Feb 1, 2017

oliver-sanders self-assigned this Feb 1, 2017

oliver-sanders force-pushed the doc-sphinx branch from c50dd56 to 33a7a79 Compare February 2, 2017 11:52

oliver-sanders force-pushed the doc-sphinx branch from 867bd01 to 9107f7b Compare February 23, 2017 09:22

oliver-sanders force-pushed the doc-sphinx branch from 9107f7b to 5052fac Compare March 21, 2017 15:39

oliver-sanders requested a review from matthewrmshin March 21, 2017 16:58

matthewrmshin reviewed Mar 22, 2017

View reviewed changes

oliver-sanders closed this Mar 22, 2017

oliver-sanders deleted the doc-sphinx branch March 22, 2017 13:00

oliver-sanders restored the doc-sphinx branch March 22, 2017 13:44

oliver-sanders reopened this Mar 22, 2017

oliver-sanders force-pushed the doc-sphinx branch from 54e5c23 to 8bfd53c Compare March 24, 2017 12:16

matthewrmshin approved these changes Mar 28, 2017

View reviewed changes

matthewrmshin requested a review from arjclark March 28, 2017 11:14

matthewrmshin reviewed Mar 29, 2017

View reviewed changes

oliver-sanders force-pushed the doc-sphinx branch from 66d50b2 to 9a2ab0f Compare March 29, 2017 15:37

oliver-sanders added 2 commits March 30, 2017 16:16

Added Sphinx documenting of rose.(config,macro).

73dc0ec

* Fixes traceback for a rose macro without a docstring

Fixed load_with_opts.

754a5fd

arjclark requested changes Apr 5, 2017

View reviewed changes

Address feedback.

1a52ec7

oliver-sanders force-pushed the doc-sphinx branch from 9a2ab0f to 1a52ec7 Compare April 7, 2017 14:33

benfitzpatrick mentioned this pull request Apr 18, 2017

Evaluate document generation tool like Sphinx / Doxygen metoppv/improver#12

Closed

matthewrmshin assigned arjclark Apr 19, 2017

arjclark approved these changes Apr 19, 2017

View reviewed changes

arjclark assigned matthewrmshin and unassigned arjclark Apr 19, 2017

matthewrmshin merged commit c688162 into metomi:master Apr 19, 2017

matthewrmshin modified the milestones: next-release, soon Apr 20, 2017

oliver-sanders mentioned this pull request May 2, 2017

Suite API - On-The-Fly cylc/cylc-flow#2123

Open

3 tasks

arh89 mentioned this pull request Jun 5, 2017

Implement document generation tool like Sphinx / Doxygen #12 metoppv/improver#13

Closed

oliver-sanders mentioned this pull request Aug 3, 2017

user guide: infrastructure oliver-sanders/rose#5

Merged

oliver-sanders deleted the doc-sphinx branch May 18, 2018 09:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sphinx documentation. #2046

Sphinx documentation. #2046

oliver-sanders commented Feb 1, 2017 •

edited

Loading

oliver-sanders commented Mar 21, 2017

oliver-sanders commented Mar 22, 2017 •

edited

Loading

matthewrmshin left a comment

matthewrmshin commented Mar 23, 2017

oliver-sanders commented Mar 24, 2017

matthewrmshin commented Mar 28, 2017

matthewrmshin Mar 29, 2017

arjclark left a comment

arjclark Apr 5, 2017

arjclark Apr 5, 2017

arjclark Apr 5, 2017

oliver-sanders Apr 5, 2017

arjclark Apr 5, 2017

arjclark Apr 5, 2017

oliver-sanders Apr 5, 2017 •

edited

Loading

arjclark Apr 5, 2017

arjclark Apr 5, 2017

arjclark commented Apr 19, 2017


		""" # Blank line.

		def __init(self, param1, param2, kwarg=None):

Sphinx documentation. #2046

Sphinx documentation. #2046

Conversation

oliver-sanders commented Feb 1, 2017 • edited Loading

oliver-sanders commented Mar 21, 2017

oliver-sanders commented Mar 22, 2017 • edited Loading

matthewrmshin left a comment

Choose a reason for hiding this comment

matthewrmshin commented Mar 23, 2017

oliver-sanders commented Mar 24, 2017

matthewrmshin commented Mar 28, 2017

Choose a reason for hiding this comment

arjclark left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliver-sanders Apr 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arjclark commented Apr 19, 2017

oliver-sanders commented Feb 1, 2017 •

edited

Loading

oliver-sanders commented Mar 22, 2017 •

edited

Loading

oliver-sanders Apr 5, 2017 •

edited

Loading