Skip to content

Commit

Permalink
CI overhaul
Browse files Browse the repository at this point in the history
This removes the docker image logic from the CI setup and instead
replaces it with a setup based on pixi. As part of that it moves the
dependency specification from localbuild/meta.yaml to pixi.toml and
pixi.lock. This turns the MSS repository into a single source of truth
for both the application code as well as the development environment
(whereas the latter was previously only specified in the docker images,
and not reproducible in any way).

Setting up a development environment is as simple as installing pixi and
running `pixi shell` (or `pixi run <cmd>`, or `pixi install` to just
create the environment, etc.). This environment will, by construction,
be the same that is used in the CI as well (modulo platform
differences).

There is a new workflow that periodically (once a week on Monday)
recreates the pixi lockfile and opens a PR for that update. The checks
in that PR essentially serve as a replacement for the previous scheduled
runs to ensure that no dependency update breaks MSS. Merging that PR is
a manual step that can be done just as with any other PR and would then
update the environment on the given target branch. This is essentially
what was previously the triggering of a docker image creation.

Including new dependencies can be done with `pixi add`, which will also
automatically add the dependency to the (pre-existing) lockfile. This
means dependency additions can be part of the PR that necessitate them
and they won't affect the entire environment (as they previously did,
where they would trigger a full image rebuild) but instead just add that
new package to the existing specification.
  • Loading branch information
matrss committed Dec 3, 2024
1 parent 6181d6a commit 5ffe851
Show file tree
Hide file tree
Showing 19 changed files with 18,373 additions and 395 deletions.
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
* text=auto
# GitHub syntax highlighting
pixi.lock linguist-language=YAML
14 changes: 7 additions & 7 deletions .github/workflows/build_docs_gallery.yml
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
name: Build Gallery

on:
pull_request:
pull_request:

jobs:
Test-MSS-docs:
runs-on: ubuntu-latest

container:
image: openmss/testing-develop

steps:
- uses: actions/checkout@v4

- uses: prefix-dev/[email protected]
with:
pixi-version: latest
cache: true
environments: dev
- name: Create gallery
timeout-minutes: 5
run: |
cd docs
mamba run --no-capture-output -n mssenv python conf.py
pixi run -e dev python conf.py
26 changes: 5 additions & 21 deletions .github/workflows/testing-all-oses.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,27 +21,11 @@ jobs:
os: ["macos-13", "macos-14", "ubuntu-latest"]
steps:
- uses: actions/checkout@v4
- name: Build requirements.txt file
run: |
sed -n '/^requirements:/,/^test:/p' localbuild/meta.yaml |
sed -e "s/.*- //" |
sed -e "s/menuinst.*//" |
sed -e "s/.*://" > requirements.tmp.txt
cat requirements.d/development.txt >> requirements.tmp.txt
echo "pytest-randomly" >> requirements.tmp.txt
sed -e '/^$/d' -e '/^#.*$/d' -e 's/\s*# \[not win\]$//' requirements.tmp.txt > requirements.txt
rm requirements.tmp.txt
cat requirements.txt
- name: Get current year and calendar week
id: year-and-week
run: echo "year-and-week=$(date +%Y-%V)" >> "$GITHUB_OUTPUT"
- uses: mamba-org/setup-micromamba@v2
- uses: prefix-dev/[email protected]
with:
environment-file: requirements.txt
environment-name: ci
cache-environment: true
# Set the cache key in a way that the cache is invalidated every week on monday
cache-environment-key: environment-${{ steps.year-and-week.outputs.year-and-week }}
pixi-version: latest
cache: true
environments: dev
- name: Run tests
timeout-minutes: 40
run: micromamba run -n ci env QT_QPA_PLATFORM=offscreen pytest -v -n logical --durations=20 --cov=mslib tests
run: pixi run -e dev env QT_QPA_PLATFORM=offscreen pytest -v -n logical --durations=20 --cov=mslib tests
19 changes: 0 additions & 19 deletions .github/workflows/testing-develop.yml

This file was deleted.

26 changes: 0 additions & 26 deletions .github/workflows/testing-scheduled.yml

This file was deleted.

19 changes: 0 additions & 19 deletions .github/workflows/testing-stable.yml

This file was deleted.

79 changes: 0 additions & 79 deletions .github/workflows/testing.yml

This file was deleted.

38 changes: 38 additions & 0 deletions .github/workflows/update-pixi-lockfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Update pixi lockfile

on:
schedule:
# At 04:00 on Monday
- cron: "0 4 * * 1"
# ...or manually
workflow_dispatch:

jobs:
update-pixi-lockfile:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
base_branch: ["develop", "stable"]
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ matrix.base_branch }}
- name: Remove old lockfile
run: rm pixi.lock
- name: Generate new lockfile
uses: prefix-dev/[email protected]
with:
pixi-version: latest
cache: false
- name: Create or update pull request
uses: peter-evans/create-pull-request@v6
with:
add-paths: pixi.lock
branch: automation/update-pixi-lockfile
commit-message: Update pixi lockfile
title: Update pixi lockfile
body: ""
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,6 @@ build/
mss.egg-info/
tutorials/recordings
tutorials/cursor_image.png
# pixi environments
.pixi

5 changes: 0 additions & 5 deletions localbuild/README.rst

This file was deleted.

7 changes: 0 additions & 7 deletions localbuild/bld.bat

This file was deleted.

7 changes: 0 additions & 7 deletions localbuild/build.sh

This file was deleted.

54 changes: 0 additions & 54 deletions localbuild/menu.json

This file was deleted.

Loading

0 comments on commit 5ffe851

Please sign in to comment.