Python Package Template

This repository is intended to be a base template, a cookiecutter for a new Python package project while keeping PEP518 in mind. Because it’s hosted on Github it already utilizes a few Github Actions that enforce repository-side checks for continuous integration and that implement a semantic release setup. And while this package is a starting point for a Python project with good engineering practices, it’s intended to be improved and added to in various ways — see the Wiki for more suggestions.

Features

The badges above give you an idea of what this project template provides. It’s work in progress, and I try to enable as much engineering goodness as is possible and is sensibly bearable using git hooks (see below) and Github Actions.

Typing

The package requires a minimum of Python 3.9 and all code requires comprehensive typing. The mypy static type checker is invoked by a git hook and through a Github Action to enforce continuous type checks.

Quality Assurance

A number of git hooks are invoked before and after a commit, and before push. These hooks are all managed by the pre-commit tool and enforce a number of software quality assurance measures (see below).

Unit testing

Comprehensive unit testing is enabled using tox, and pytest combined with Hypothesis (to generate test payloads and strategies), and test code coverage is measured using coverage (see below).

Documentation

Documentation is important, and Sphinx is set up already to produce standard documentation for the package, assuming that code contains docstrings with reStructuredText (see below).

Versioning and publishing

Automatic package versioning and tagging, publishing to PyPI, and Changelog generation are enabled using Github Actions (see below).

Dependency Analysis

Dependabot is enabled to scan the dependencies and automatically create pull requests when an updated version is available.

Standalone

In addition to being an importable standard Python package, the package is also set up to be used as a runnable and standalone package using Python’s -m command-line option.

How to use this repository

If you’d like to contribute to the project template, please open an issue for discussion or submit a pull request.

If you’d like to start your own Python project from scratch, you can either copy the content of this repository into your new project folder or fork this repository. Either way, consider making the following adjustments to your copy:

• Change the LICENSE.md file and the badge according to your needs.

• Install pre-commit and set it up for your new package repository to ensure that all git hooks are active:

  pre-commit install
  pre-commit install --hook-type commit-msg
  pre-commit install --hook-type pre-push

• Rename the src/package/ folder to whatever your own package’s name will be, and adjust the Github Actions in .github/workflows/, setup.py, pyproject.toml, pre-commit-config.yaml and the unit tests accordingly.

• Adjust the content of the setup.py file according to your needs, and reset the package’s version number in src/package/__init__.py. Don’t forget to delete the content of the CHANGELOG.md file (except for the first placeholder line).

• If you’d like to publish your package to PyPI then set the upload_to_pypi variable in the pyproject.toml file to true.

To develop your new package, create a virtual environment and install its dev, test and docs dependencies:

python3.9 -m venv .
source ./bin/activate
pip install --upgrade pip
pip install --editable .[dev,test,docs]

With that in place, you’re ready to build your own package.

Git hooks

Using the pre-commit tool and its .pre-commit-config.yaml configuration, the following git hooks are active in this repository:

• When committing code, a number of pre-commit hooks ensure that your code is formatted according to PEP 8 using the black tool, and they’ll invoke flake8 (and various plugins), pylint and mypy to check for lint and correct types. There are more checks, but those two are the important ones. You can adjust the settings for these tools in one of the pyproject.toml or tox.ini or pylintrc or mypy.ini configuration files.
• The commit message hook enforces conventional commit messages and that, in turn, enables a semantic release of this package on the Github side: upon merging changes into the master branch, the semantic release action produces a changelog and computes the next version of this package and publishes a release — all based on the commit messages.
• Using a pre-push hook this package is also set up to run pytest; in addition, the coverage plugin makes sure that all of your package’s code is covered by tests and Hypothesis is already installed to help with generating test payloads.

Testing

As mentioned above, this repository is set up to use tox and pytest either standalone or as a pre-push git hook. Tests are stored in the tests/ folder, and you can run them manually like so:

tox

For more options, see the tox command-line flags and the pytest command-line flags. Also note that pytest includes doctest, which means that module and function docstrings may contain test code that executes as part of the unit tests.

Test code coverage is already tracked using coverage and the pytest-cov plugin for pytest. Code coverage is tracked automatically when running tox; in addition, the plugin can be explicitly invoked with the following command line:

pytest --cov package tests

and measures how much code in the src/package/ folder is covered by tests:

platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- ...
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/.../.hypothesis/examples')
rootdir: /.../python-package-template, configfile: pyproject.toml, testpaths: tests
plugins: hypothesis-6.23.2, cov-3.0.0
collected 1 item  

tests/test_something.py::test_something PASSED                           [100%]

---------- coverage: platform darwin, python 3.9.7-final-0 -----------
Name                   Stmts   Miss  Cover   Missing
----------------------------------------------------
package/__init__.py        1      0   100%
package/something.py       4      0   100%
----------------------------------------------------
TOTAL                      5      0   100%

Required test coverage of 100.0% reached. Total coverage: 100.00%

============================== 1 passed in 0.07s ==============================

Note that code that’s not covered by tests is listed under the Missing column.

Hypothesis is a package that implements property based testing and that provides payload generation for your tests based on strategy descriptions (more). Using its pytest plugin Hypothesis is ready to be used for this package.

Documentation

As mentioned above, all package code should make use of Python docstrings in reStructured text format. Using these docstrings and the documentation template in the docs/source/ folder, you can then generate proper documentation in different formats using the Sphinx tool:

cd docs
make html

This example generates documentation in HTML, which can then be found here:

open _build/html/index.html

Versioning, publishing and changelog

To enable automation for versioning, package publishing, and changelog generation it is important to use meaningful conventional commit messages! This package template already has a semantic release Github Action enabled which is set up to take care of all three of these aspects — every time changes are merged into the master branch.

For more configuration options, please refer to the tool.semantic_release section in the pyproject.toml file, and read the semantic release documentation.

You can also run the tool manually, for example:

semantic-release changelog
semantic-release version

Use the --verbosity=DEBUG command-line argument for more details.