Create a unit testing framework

[fralau]

It would be useful to have a unit testing framework, in particular for non-regression-testing.

With PR #239, @timvink proposes a pattern already used on mkdocstrings.

For a detailed discussion see #237 about hooks, particularly:

You can test the functions / custom classes separately of course. I actually do prefer small 'integration' testing using mkdocs build.

What's worked for me is to build up a collection of small standalone examples (like here https://github.com/timvink/mkdocs-table-reader-plugin/tree/master/tests/fixtures) and test whether they fail (sometimes they should) or succeed, and whether the resulting html page(s) contain expected output. See f.e. https://github.com/timvink/mkdocs-table-reader-plugin/blob/master/tests/test_build.py

Originally posted by @timvink in #237 (comment)

[fralau]

General description

The testing rig proposed is based on a number of components:

• pytest: allows a programmer to write test functions in Python, containing assertions in plain syntax (assert a == b). Quote: the command pytest "will run all files of the form test_*.py or *_test.py in the current directory and its subdirectories. "

• CliRunner is part of the Python Click library, used for creating command-line interfaces (CLIs) in Python. In this particular case, CliRunner is used to simulate the mkdocs build command as if it were being executed from the command line. It provides an exit code and an output, which can then be tested.

• the file test_builds.py orchestrates those tools, by building an MkDocs directory.

• A Github workflow (testing.yaml), triggered by a push event (see example). When a push is made, this workflow is automatically executed -- this is not mandatory, but part of Continuous Integration.

[timvink]

I think you could replace CliRunner with a basic subprocess.call command also, if you want to keep your dev dependencies light

[fralau]

I have played around enough with subprocess.call() to appreciate the simplicity of CliRunner 🙂.

[fralau]

@timvink I am studying the solution you adopted, as well as pytest.

I would have question: what is the advantage of creating a separate test directory for each build, compared to using the regualr site subdirectory (which is in .git_ignore)?

[timvink]

Instead of pytest --pdb to drop into the debugger, I can use mkdocs serve -f path_to_yaml together with a breakpoint() anyehere in the code to drop into the debugger there.

[fralau]

The follow-up will be done on #244. Closing this, with the label "keep in mind".