Ansible role development environment powered by Molecule.
Ansible has slowly been phasing out the use of legacy roles in favour of the new collections format.
At this time, I've made the decision to put this project on hiatus, but if there is sufficient interest I'll refactor it into a solution for collections.
This cookiecutter template generates a development environment for Ansible Roles with comprehensive CI/CD for GitHub Actions.
Poetry powers a Python virtual environment loaded with Molecule and Ansible-Lint allowing you to develop and test roles with a focus on quality.
CICD-Tools provides managed CI/CD components giving you a functional pipeline from day one, and maintains the pipeline components for you.
Windows Users:
- If you are working locally with Windows, you'll need a Posix or Linux virtual environment of some kind to use this template.
- Neither Ansible, nor Ansible-Lint support this platform.
OSX, Linux, BSD Users:
- Please follow the Quick Start Guide below.
- A container runtime environment such as Docker or Colima is recommended, to make use of the configured pre-commit hooks.
- You'll have to disable the pre-commit hooks if you choose not to follow this path.
- An account on Ansible Galaxy is needed if you intend to publish your role.
- An account on GitHub is needed to make use of the CI/CD.
- A configured Slack Webhook is also handy for keeping tabs on the CI/CD.
Open your shell, and run the following commands:
pip install cookiecutter poetrycookiecutter https://github.com/niall-byrne/ansible-workbench.git
When prompted:
- Give your project a name, and populate the other required template inputs.
- See the section on TOML Linting for details on this optional tooling.
- If you plan on using GitHub Actions with your project, consider using the Workflow Linting.
Once the templating is finished:
cd <your new project directory>poetry shell(to interact with Ansible and Molecule inside a virtualenv)
You can then interact with the example role and Molecule scenarios:
molecule lintmolecule test -s docker1
More importantly: you can now build off this existing scaffolding to develop your own high quality Ansible role.
Modify the templated requirements.yml file to add or remove the external roles or collections your role requires.
The pyproject.toml file is there to store your project's Python dependencies in accordance with PEP 518.
Poetry is leveraged to manage the Python dependencies:
| Package | Description |
|---|---|
| ansible | Simple, agentless IT automation. |
| ansible-lint | Checks playbooks for best practices and behaviours. |
| commitizen | Standardizes commit messages. |
| poetry | Python package manager. |
| pre-commit | A framework for managing pre-commit hooks. |
| molecule | Ansible test framework. |
The template also presents the option to render a pre-commit hook and a CI step to lint the pyproject.toml file:
- This is accomplished via the golang binary tomll.
- This is the most robust TOML formatter I'm aware of right now.
- Use of the pre-commit hooks "as-rendered" requires you have a container runtime such as Docker installed on your system.
If you prefer to install a binary on your system:
- You can download the latest binary here.
- Alternatively, your OS's package manager may support this tool.
- For example: Ubuntu
- You can also compile the tool yourself fairly easily from source.
- If you have go installed:
$ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
- If you have go installed:
A fundamental pillar of Ansible Workbench is the use of Conventional Commits.
- Following this standard has numerous advantages, but among the largest is its tight integration with Semantic Versioning.
- The CI/CD in particular, changelog generation and release automation is made possible through adherence to this format.
- Being able to read commits from different people that conform to a standard also makes interactive rebasing relatively painless.
- This tool leverages commitizen to enforce Conventional Commits as well as Semantic Versioning.
- Stage any changes as your normally would:
git add tasks/new_task_file.yml - Launch commitizen:
poetry run cz c(See the next section to shorten that command.) - Alternatively, you might instead opt to manually make your commits:
git commit -m "feat(NEW_TASK): added my new task file"
It's a bit more practical to work inside the Poetry virtual environment itself:
- Start the virtual environment:
poetry shell - Stage your changes:
git add new_task_file.yml - Make your commit:
cz c
It's recommended to install a container runtime such as Docker or Colima on your system:
- This allows you to leverage the templated pre-commit Hooks
- Commits are linted both by the pre-commit hooks and the templated GitHub CI/CD.
- Tags are ALSO linted. Please see the related section on tagging your releases.
- If you have legacy (non-conventional) commits in your git history, you can still use the CI/CD.
- See the section on configuring the 'push' workflow:
- Setting
ci_commitzen_rev_rangeto: "<my first conventional commit hash>..HEAD" will stop failing the CI on older commits
- Setting
- With this workaround in place you may still have CI failures when rebasing legacy commits that you don't fix-up.
- To ensure the CI passes, you could trigger a manual run of the workflow.
There are 3 example Molecule scenarios created during templating:
| Scenario Name | Description |
|---|---|
| default | Intended as a configuration reference and for use in linting and caching dependencies. |
| docker1 | An example using Molecule's docker driver. |
| hostmachine1 | An example using Molecule's delegated driver. |
It's recommended to avoid modifying the default scenario, to make the most of the rendered CI/CD.
- Molecule actually supports different requirements.yml files for each scenario you create, however to optimize dependency caching the CI will try to cache the dependencies using this default scenario.
- We recommend keeping one set of requirements in the requirements.yml file to speed up CI execution.
To add tests to your role, create new scenarios with Molecule:
poetry run molecule init scenario my_scenario
If you are using the rendered GitHub CI/CD, you can tweak which scenarios are executed by the CI via the workflow configuration. Name your workflow with reference to the regex specified in the ci_molecule_excluded_scenarios_regex key.
You could of course also modify the workflow itself for more fine-tuned control.
Connections to the Ansible Galaxy API can sometimes time out, unfortunately.
In your Molecule Scenario file set the dependency option force: false as shown here.
In concert with this setting, it's also strongly recommended to avoid modifying the default Molecule scenario:
- This will also help you get the most speed out of the default CI/CD configuration.
- All of this assumes that your role always uses the same set of dependencies. Multiple sets of requirements for different scenarios is supported by Molecule, but will impact CI execution speed due to inefficient caching.
In your Molecule Scenario file set the dependency option timeout: 120 as shown here.
The python library pre-commit is installed during templating with a few useful initial hooks.
These hooks are your quality controls, there to help you avoid senseless time-wasting mistakes.
These hooks are deeply integrated with CICD-Tools and are executed TWICE for all commits:
- Whenever you make a commit locally to your project.
- Whenever you push a commit to GitHub. (The CI/CD will ALSO execute the check.)
These hooks all rely on 3rd party software to perform different types of static analysis on your code:
- Some of this software can be managed by pre-commit itself (Especially if it's Python based.)
- Other steps require the use of binaries that you may not be familiar with. (See the complete list in the next section.)
To simplify the process of seamlessly installing these 3rd party tools we leverage containers:
- CICD-Tools provides several vetted binaries via its own container.
- There are additional binary tools configured to run in 3rd party containers following this same paradigm.
For this reason we strongly recommend installing a container runtime such as Docker or Colima on your development machine.
If you do NOT wish to use a container runtime, please see the section on removing pre-commit hooks.
| Hook Name | Description |
|---|---|
| check-jsonschema | Runs check-jsonschema to confirm the CI/CD configuration files are correct. |
| commitizen | Runs commitizen on your commit message to validate it. |
| format-shell | Runs shfmt to format your shell scripts. |
| format-toml | Runs tomll to format your TOML configuration files. |
| lint-ansible | Runs ansible-lint to check for Ansible best practices and behaviours. |
| lint-github-workflow | Optionally runs actionlint on all GitHub workflows. |
| lint-github-workflow-header | Optionally runs a shell script to lint GitHub workflow headers. |
| lint-markdown | Runs markdown-lint on your Markdown files. |
| lint-shell | Runs shellcheck to lint all shell scripts. |
| spelling-commit-message | Runs vale on your git commit messages to check for spelling errors. |
| spelling-markdown | Runs vale on your Markdown files to check for spelling errors. |
| yamllint | Runs yamllint on all YAML files. |
- The CICD-Tools pre-commit hooks provide base configuration.
- The cookiecutter pre-commit hooks overrides these hooks for development of the template itself.
- The project pre-commit hooks overrides these hooks in projects you create with this template.
If you wish to remove or deactivate a particular hook remove its reference from these TWO places in your project:
- Find and remove the hook's definition in the .pre-commit-config.yaml file.
- Find and remove the hook's integration in the CI/CD workflow definition file.
| Hook Name | Exe Source | Configuration File(s) |
|---|---|---|
| check-jsonschema | pre-commit | .pre-commit-config.yaml |
| commitizen | poetry/pre-commit | pyproject.toml |
| format-shell | CICD-Tools | .pre-commit-config.yaml |
| format-toml | CICD-Tools | None |
| lint-ansible | poetry | .ansible-lint |
| lint-github-workflow | CICD-Tools | actionlint.yaml |
| lint-github-workflow-header | shell | lint-github-workflow-header.sh |
| lint-markdown | 3rd party | .markdownlint.yml |
| lint-shell | CICD-Tools | .pre-commit-config.yaml |
| spelling-commit-message | CICD-Tools | .vale.ini, vocab lists |
| spelling-markdown | CICD-Tools | .vale.ini, vocab lists |
| yamllint | pre-commit | .yamllint.yml |
This project has deep integration with CICD-Tools which provides managed, reusable CI/CD components. This allows you to focus on your project, while befitting from the extensive quality control tools this integration provides.
At this time CICD-Tools has chosen to focus exclusively on GitHub Actions Perhaps eventually other CI platforms will be supported, but for now to get the most out of this project it's recommended to use the GitHub platform.
Whenever you push to GitHub, the Push Workflow is triggered.
- The configuration in workflow-push.json is read by the workflow.
- Customize this file to modify workflow parameters without having to modify the workflow itself.
- The active Molecule Scenarios are identified
- The repository is scanned for any checked in secrets.
- The documentation is checked for any invalid HTML links.
- Ansible Lint is run to validate the role.
- The Molecule Scenarios are tested.
- The Pre-Commit hooks themselves are tested to ensure they are still working. (i.e. Do bad commits still fail correctly?)
- The commit messages are linted.
- The CI then executes the configured Pre-Commit hooks for the repository against all files, ensuring the integrity of the code base.
- Read the pre-commit section for details on which checks are included.
- If you have pushed a tag conforming to Semantic Versioning, the workflow creates a release for you with an automated changelog.
If you have created something you want to share with the Ansible community, there is an included automated release process which enables you to publish your role to both on GitHub and Ansible Galaxy:
- The Push Workflow will respond to a pushed Semantic Versioned tag by creating a draft GitHub release with an automatically generated changelog.
- Edit this release on GitHub to your liking and then "publish" it.
- If you have created the required GitHub Secrets and configured your role's metadata correctly, your role will now automatically be published to Ansible Galaxy.
For more details please read section on creating a release.
The values you've entered into the cookiecutter template determine what your repository should be named, and who it should be owned by.
- Create a repository in GitHub using the
github_handleyou specified, and theproject_slugname that was created during templating from the project name you input. - If you want to change the
project_nameorgithub_handlevalue because of this, you should run cookiecutter again. - If you don't remember the
project_slugvalue, the file .cookiecutter/cookiecutter.json will be templated in your role's repository with all the values you entered during templating.
Connect your new remote repository and push:
- Checkout the
masterbranch:git checkout master - Add the new remote origin:
git remote add origin [email protected]:\<your_github_handle\>/\<your_project_slug\>.git - Push to the new remote origin:
git push -u origin master
To make working with the CI/CD easy to get started with, there is configuration that is templated with sensible defaults. This gives you a low complexity option for modifying several common workflow settings.
The configuration files will be created in the .github/config folder:
- gaurav-nelson-github-action-markdown-link-check.json is used to customize the documentation step of the workflow.
- workflow-publish-to-galaxy.json is used to customize the publish workflow. (This is the workflow that runs whenever a release is published from your GitHub repository)
- workflow-push.json is used to customize the push workflow. (This is the workflow that runs on each git "push" to your GitHub repository.)
There are also JSON schemas available here to help you understand all settings, and ensure your configuration stays consistent.
You'll need to create some secrets to enable the full functionality of your CI/CD.
To make the most out of your CI/CD, create the following secrets:
SLACK_WEBHOOK: This optional secret value should contain a Slack Webhook you've configured to get status updates on how your commit is proceeding through the CI/CD.- The verbosity of this integration can be controlled by setting the
ci_verbose_notificationssetting in this rendered configuration file. - See this documentation on how to create a Slack Webhook for your team.
- If you do not wish to use CI/CD notifications, simply leave this secret unset.
- The verbosity of this integration can be controlled by setting the
GALAXY_API_KEY: This optional secret value can be created from the API key found on your Ansible Galaxy account page, and enables automated publishing to Galaxy.- If you do not wish to publish your role, simply leave this secret unset.
If you have created something you want to share with the Ansible community, there is an included automated release process which enables you to publish your role to both on GitHub and Ansible Galaxy.
Release generation depends in part on your use of Conventional Commits to format each individual commit to your code base.
- Please see the section on working with Ansible Workbench for more details.
Tag your release with Semantic Versioning. (Avoid prefixing a 'v' or 'version' to your tag.)
- It's recommended to use commitizen to manage the process of updating versions, and ensuring tags are in the correct format.
- To "bump" your release:
poetry run cz bump
- To "bump" your release:
- If you are managing this process manually, take care to ensure the
versionfield is updated inside your pyproject.toml file.
- Push your new semantic conforming tag to GitHub, and ensure the CI passes:
git push --tags - The CI will create a
draftrelease for you with a changelog on GitHub.- The changelog is generated in accordance with this configuration file.
- Customize this file according to it's JSON Schema
- Review it and if all is well,
editand thenpublishthe release on GitHub.
- If you have configured a secret for Ansible Galaxy more automation will now begin after you've published your GitHub release.
- The release workflow will be triggered, and will publish your release automatically to Ansible Galaxy.