-
Notifications
You must be signed in to change notification settings - Fork 29
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
🧑💻 Extensive and Reusable CI/CD Setup (#396)
## Description This PR tries to establish some new best practices for CI/CD within the MQT. First of all, this adds a Cirrus CI config for building Apple Silicon and Linux ARM wheels as well as perform Python testing on Apple Silicon. Most importantly, a series of smaller reusable workflows have been created. These include workflows for different types of tests (Python, C++, Windows, MacOS), Python packaging, and linting. The purpose of this change is to improve the modularity and readability of the workflows, and allow for better reuse across different workflows and projects. In the best case, these workflows can directly be reused in our top-level projects without having to write them from scratch again. Furthermore, code coverage reports are now split between Python and C++ test workflows and will be automatically uploaded to Codecov using the flags feature. Carry-forward support is enable for scenarios where not all reports are uploaded. Lastly, CI and CD workflows were split for better readability and maintainability. Wheels are now only built on releases. The respective CI runs are only started if some files relevant to that check have been modified. A single pass-check eases branch protection. ## Checklist: <!--- This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly. --> - [x] The pull request only contains commits that are related to it. - [x] I have added appropriate tests and documentation. - [x] I have made sure that all CI jobs on GitHub pass. - [x] The pull request introduces no new warnings and follows the project's style guidelines.
- Loading branch information
Showing
31 changed files
with
999 additions
and
600 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
.clone_script: &clone | | ||
if [ -z "$CIRRUS_PR" ]; then | ||
git clone --recursive --branch=$CIRRUS_BRANCH https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR | ||
git reset --hard $CIRRUS_CHANGE_IN_REPO | ||
else | ||
git clone https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR | ||
git fetch origin pull/$CIRRUS_PR/head:pull/$CIRRUS_PR | ||
git checkout $CIRRUS_BASE_BRANCH | ||
git -c user.email="[email protected]" merge --no-commit pull/$CIRRUS_PR | ||
git submodule update --init --recursive | ||
fi | ||
|
||
.statistics_script: &statistics | | ||
ccache -s -v | ||
echo $(python -m pip cache dir) | ||
echo $(python -m pip cache list) | ||
|
||
macos_arm64_test_task: | ||
name: 🐍 Test / 🍎 arm64 | ||
alias: macos_arm64_test | ||
clone_script: *clone | ||
macos_instance: | ||
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest | ||
ccache_cache: | ||
folder: .ccache | ||
populate_script: | ||
- mkdir -p .ccache | ||
fingerprint_key: ccache-macosx_arm64 | ||
pip_cache: | ||
folder: /Users/admin/Library/Caches/pip | ||
prepare_env_script: | | ||
brew install [email protected] ccache pipx ninja nox | ||
ln -s $(which python3.10) python | ||
export PATH=/opt/homebrew/opt/[email protected]/libexec/bin:$PATH | ||
export PATH=/opt/homebrew/opt/ccache/libexec:$PATH | ||
export PATH=/opt/homebrew/opt/pipx/libexec:$PATH | ||
export PATH=/opt/homebrew/opt/ninja/libexec:$PATH | ||
export PATH=/opt/homebrew/opt/nox/libexec:$PATH | ||
echo "PATH=$PATH" >> $CIRRUS_ENV | ||
echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV | ||
run_nox_script: | ||
- nox -s tests-3.10 --verbose | ||
statistics_script: *statistics | ||
|
||
build_and_store_wheels: &BUILD_AND_STORE_WHEELS | ||
install_cibuildwheel_script: | ||
- python -m pip install cibuildwheel~=2.15.0 | ||
run_cibuildwheel_script: | ||
- cibuildwheel | ||
wheels_artifacts: | ||
path: "wheelhouse/*" | ||
|
||
linux_aarch64_wheels_task: | ||
only_if: "$CIRRUS_RELEASE != ''" | ||
name: 🐍 Packaging / 🎡 🐧 arm64 | ||
alias: linux_aarch64_wheels | ||
clone_script: *clone | ||
compute_engine_instance: | ||
image_project: cirrus-images | ||
image: family/docker-builder-arm64 | ||
architecture: arm64 | ||
platform: linux | ||
cpu: 8 | ||
env: | ||
CIBW_SKIP: "*-musllinux_*" | ||
setup_pyhton_script: | ||
- apt-get install -y python3-venv python-is-python3 | ||
<<: *BUILD_AND_STORE_WHEELS | ||
|
||
macos_arm64_wheels_task: | ||
only_if: "$CIRRUS_RELEASE != ''" | ||
name: 🐍 Packaging / 🎡 🍎 arm64 | ||
alias: macos_arm64_wheels | ||
clone_script: *clone | ||
macos_instance: | ||
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest | ||
ccache_cache: | ||
folder: .ccache | ||
populate_script: | ||
- mkdir -p .ccache | ||
fingerprint_key: ccache-macosx_arm64-wheels | ||
pip_cache: | ||
folder: /Users/admin/Library/Caches/pip | ||
prepare_env_script: | | ||
brew install [email protected] ccache pipx ninja nox | ||
ln -s $(which python3.10) python | ||
export PATH=/opt/homebrew/opt/[email protected]/libexec/bin:$PATH | ||
export PATH=/opt/homebrew/opt/ccache/libexec:$PATH | ||
export PATH=/opt/homebrew/opt/pipx/libexec:$PATH | ||
export PATH=/opt/homebrew/opt/ninja/libexec:$PATH | ||
export PATH=/opt/homebrew/opt/nox/libexec:$PATH | ||
echo "PATH=$PATH" >> $CIRRUS_ENV | ||
echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV | ||
<<: *BUILD_AND_STORE_WHEELS | ||
statistics_script: *statistics | ||
|
||
publish_task: | ||
name: 🚀 Deploy to PyPI | ||
container: { image: "python:3.10-bullseye" } | ||
depends_on: | ||
- linux_aarch64_wheels | ||
- macos_arm64_wheels | ||
only_if: "$CIRRUS_RELEASE != ''" | ||
env: | ||
TWINE_REPOSITORY: pypi | ||
TWINE_USERNAME: __token__ | ||
TWINE_PASSWORD: "ENCRYPTED\ | ||
[776960f3f91eccd9d5036b8dd2ef02dd3d23caf52942443e\ | ||
d9aabf8fe4e7597406fd388e8b280a6c766703cc0e7b92fb]" | ||
install_script: pip install twine | ||
publish_script: | ||
- curl -L https://api.cirrus-ci.com/v1/artifact/build/$CIRRUS_BUILD_ID/wheels.zip -o wheels.zip | ||
- unzip wheels.zip | ||
- python -m twine check wheelhouse/* | ||
- python -m twine upload wheelhouse/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# Contributing | ||
|
||
Thank you for your interest in contributing to this project. | ||
We value contributions from people with all levels of experience. | ||
In particular if this is your first pull request not everything has to be perfect. | ||
We will guide you through the process. | ||
|
||
We use GitHub to [host code](https://github.com/cda-tum/mqt-core), to [track issues and feature requests](https://github.com/cda-tum/mqt-core/issues), as well as accept [pull requests](https://github.com/cda-tum/mqt-core/pulls). | ||
See <https://docs.github.com/en/get-started/quickstart> for a general introduction to working with GitHub and contributing to projects. | ||
|
||
## Types of Contributions | ||
|
||
You can contribute in several ways: | ||
|
||
- 🐛 Report Bugs | ||
: Report bugs at <https://github.com/cda-tum/mqt-core/issues> using the _🐛 Bug report_ issue template. Please make sure to fill out all relevant information in the respective issue form. | ||
|
||
- 🐛 Fix Bugs | ||
: Look through the [GitHub Issues](https://github.com/cda-tum/mqt-core/issues) for bugs. Anything tagged with "bug" is open to whoever wants to try and fix it. | ||
|
||
- ✨ Propose New Features | ||
: Propose new features at <https://github.com/cda-tum/mqt-core/issues> using the _✨ Feature request_ issue template. Please make sure to fill out all relevant information in the respective issue form. | ||
|
||
- ✨ Implement New Features | ||
: Look through the [GitHub Issues](https://github.com/cda-tum/mqt-core/issues) for features. Anything tagged with "feature" or "enhancement" is open to whoever wants to implement it. We highly appreciate external contributions to the project. | ||
|
||
- 📝 Write Documentation | ||
: MQT Core could always use some more documentation, and we appreciate any help with that. | ||
|
||
## Get Started 🎉 | ||
|
||
Ready to contribute? Check out the {doc}`Development Guide <DevelopmentGuide>` to set up MQT Core for local development and learn about the style guidelines and conventions used throughout the project. | ||
|
||
We value contributions from people with all levels of experience. | ||
In particular if this is your first PR not everything has to be perfect. | ||
We will guide you through the PR process. | ||
Nevertheless, please try to follow the guidelines below as well as you can to help make the PR process quick and smooth. | ||
|
||
## Core Guidelines | ||
|
||
- ["Commit early and push often"](https://www.worklytics.co/blog/commit-early-push-often). | ||
- Write meaningful commit messages (preferably using [gitmoji](https://gitmoji.dev) to give additional context to your commits). | ||
- Focus on a single feature/bug at a time and only touch relevant files. Split multiple features into multiple contributions. | ||
- If you added a new feature, you should add tests that ensure it works as intended. Furthermore, the new feature should be documented appropriately. | ||
- If you fixed a bug, you should add tests that demonstrate that the bug has been fixed. | ||
- Document your code thoroughly and write readable code. | ||
- Keep your code clean. Remove any debug statements, left-over comments, or code unrelated to your contribution. | ||
- Run `pre-commit run -a` to check your code for style and linting errors before committing. | ||
|
||
## Pull Request Workflow | ||
|
||
- Create PRs early. It is ok to create work-in-progress PRs. You may mark these as draft PRs on GitHub. | ||
- Describe your PR. Start with a descriptive title, reference any related issues by including the issue number in the PR description, and add a comprehensive description of the changes. We provide a PR template that you can (and should) follow to create a PR. Do not delete any sections from the template. | ||
- Whenever a PR is created or updated, several workflows on all supported platforms and versions of Python are executed. These workflows ensure that the project still builds, that all tests pass, and that the code is properly formatted and introduces no new linting errors. Your PR is expected to pass all these continuous integration (CI) checks before it can be merged. Here are some tips for finding the cause of certain failures: | ||
- If any of the `CI / 🇨 Test` checks fail, this most likely indicates build errors or test failures in the C++ part of the code base. Look through the respective logs on GitHub for any error or failure messages. | ||
- If any of the `CI / 🐍 Test` checks fail, this most likely indicates build errors or test failures in the Python part of the code base. Look through the respective logs on GitHub for any error or failure messages. | ||
- If any of the `codecov/\*` checks fail, this means that your changes are not appropriately covered by tests or that the overall project coverage decreased too much. Ensure that you include tests for all your changes in the PR. | ||
- If `cpp-linter` comments on your PR with a list of warnings, these have been raised by `clang-tidy` when checking the C++ part of your changes for warnings or style guideline violations. The individual messages frequently provide helpful suggestions on how to fix the warnings. If you don't see any messages, but the `🇨 Lint / 🚨 Lint` check is red, click on the `Details` link to see the full log of the check and a step summary. | ||
- If the `pre-commit.ci` check fails, some of the `pre-commit` checks failed and could not be fixed automatically by the _pre-commit.ci_ bot. Such failures are most likely related to the Python part of the code base. The individual log messages frequently provide helpful suggestions on how to fix the warnings. | ||
- If the `docs/readthedocs.org:mqt-core` check fails, the documentation could not be built properly. Inspect the corresponding log file for any errors. | ||
- Once your PR is ready, change it from a draft PR to a regular PR and request a review from one of the project maintainers. Please make sure to only request a review once you are done with your changes and the PR is ready to be merged. If you are unsure whether your PR is ready for review, please ask in the PR comments. | ||
- If your PR gets a "Changes requested" review, you will need to address the feedback and update your PR by pushing to the same branch. You don't need to close the PR and open a new one. Respond to review comments on the PR (e.g., with "done 👍" or "done in @\<commit\>") to let the reviewer know that you have addressed the feedback. Note that reviewers do not get a notification if you just react to the review comment with an emoji. You need to write a comment to notify the reviewer. | ||
- Be sure to re-request review once you have made changes after a code review so that maintainers know that the requests have been addressed. | ||
|
||
--- | ||
|
||
This document was inspired by and partially adapted from | ||
|
||
- <https://matplotlib.org/stable/devel/coding_guide.html> | ||
- <https://opensource.creativecommons.org/contributing-code/pr-guidelines/> | ||
- <https://yeoman.io/contributing/pull-request.html> | ||
- <https://github.com/scikit-build/scikit-build> |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Support | ||
|
||
If you are stuck with a problem using MQT Core or have questions, please get in touch at our [Issues](https://github.com/cda-tum/mqt-core/issues) or [Discussions](https://github.com/cda-tum/mqt-core/discussions). We'd love to help. | ||
|
||
You can save time by following this procedure when reporting a problem: | ||
|
||
- Do try to solve the problem on your own first. | ||
- Search through past [Issues](https://github.com/cda-tum/mqt-core/issues) and [Discussions](https://github.com/cda-tum/mqt-core/discussions) to see if someone else already had the same problem. | ||
- Before filing a bug report, try to create a minimal working example (MWE) that reproduces the problem. It is much easier to identify the cause for the problem if a handful of lines suffice to show that something is not working. | ||
|
||
You can also always reach us at [[email protected]](mailto:[email protected]). |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
name: CD | ||
on: | ||
release: | ||
types: [published] | ||
workflow_dispatch: | ||
|
||
jobs: | ||
python-packaging: | ||
name: 🐍 Packaging | ||
uses: ./.github/workflows/reusable-python-packaging.yml | ||
with: | ||
package-name: mqt.core |
Oops, something went wrong.