Merge pull request #220 from pybop-team/v24.2

Make v24.3

.all-contributorsrc

@@ -50,7 +50,9 @@
       "contributions": [
         "ideas",
         "mentoring",
-        "review"
+        "review",
+        "code",
+        "test"
       ]
     },
     {
@@ -79,6 +81,16 @@
       "contributions": [
         "financial"
       ]
+    },
+    {
+      "login": "agriyakhetarpal",
+      "name": "Agriya Khetarpal",
+      "avatar_url": "https://avatars.githubusercontent.com/u/74401230?v=4",
+      "profile": "https://github.com/agriyakhetarpal",
+      "contributions": [
+        "code",
+        "infra"
+      ]
     }
   ],
   "contributorsPerLine": 7,

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,36 @@
+# Description
+
+Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
+
+## Issue reference
+Fixes # (issue-number)
+
+## Review
+Before you mark your PR as ready for review, please ensure that you've considered the following:
+- Updated the [CHANGELOG.md](https://github.com/pybop-team/PyBOP/blob/develop/CHANGELOG.md) in reverse chronological order (newest at the top) with a concise description of the changes, including the PR number.
+- Noted any breaking changes, including details on how it might impact existing functionality.
+
+## Type of change
+- [ ] New Feature: A non-breaking change that adds new functionality.
+- [ ] Optimization: A code change that improves performance.
+- [ ] Bug Fix: A non-breaking change that addresses an issue.
+- [ ] Documentation: Updates to documentation or new documentation for new features.
+- [ ] Refactoring: Non-functional changes that improve the codebase.
+- [ ] Style: Non-functional changes related to code style (formatting, naming, etc).
+- [ ] Testing: Additional tests to improve coverage or confirm functionality.
+- [ ] Other: (Insert description of change)
+
+# Key checklist:
+
+- [ ] No style issues: `$ pre-commit run` (or `$ nox -s pre-commit`) (see [CONTRIBUTING.md](https://github.com/pybop-team/PyBOP/blob/develop/CONTRIBUTING.md#installing-and-using-pre-commit) for how to set this up to run automatically when committing locally, in just two lines of code)
+- [ ] All unit tests pass: `$ nox -s tests`
+- [ ] The documentation builds: `$ nox -s docs`
+
+You can run integration tests, unit tests, and doctests together at once, using `$ nox -s quick`.
+
+## Further checks:
+- [ ] Code is well-commented, especially in complex or unclear areas.
+- [ ] Added tests that prove my fix is effective or that my feature works.
+- [ ] Checked that coverage remains or improves, and added tests if necessary to maintain or increase coverage.
+
+Thank you for contributing to our project! Your efforts help us to deliver great software.

.github/release_workflow.md

@@ -0,0 +1,34 @@
+# Release Workflow
+
+This document outlines the release workflow for publishing to PyPI and TestPyPI using GitHub Actions.
+
+## Creating a New Release
+
+To create a new release, follow these steps:
+
+1. **Prepare the Release:**
+   -  Create a new branch for the release (i.e. `v24.XX`) from `develop`.
+   -  Increment the following;
+         -  The version number in the `pyproject.toml` file following CalVer versioning.
+         -  The`CHANGELOG.md` version with the changes for the new version.
+   -  Open a PR to the `main` branch. Once the PR is merged, proceed to the next step.
+
+2. **Tag the Release:**
+   -  Create a new Git tag for the release. For a full release, use a tag like `v24.2`. For a release candidate, use a tag like `v24.2rc.1`.
+   -  Push the tag to the remote repository: `git push origin <tag_name>`.
+
+3. **Create a GitHub Release:**
+   -  Go to the "Releases" section of on GitHub.
+   -  Click "Draft a new release."
+   -  Enter the tag you created in the "Tag version" field.
+   -  Fill in the release title and description. Add any major changes and link to the `CHANGELOG.md` for a list of total changes.
+   -  If it's a pre-release (release candidate), check the "This is a pre-release" checkbox.
+   -  Click "Publish release" to create the release.
+
+4. **Monitor the Workflow:**
+   -  Go to the "Actions" tab of your repository to monitor the workflow's progress.
+   -  The workflow will build the distribution packages and then publish them to PyPI or TestPyPI, depending on whether the release is a full release or a pre-release.
+
+5. **Verify the Release:**
+   -  Check PyPI or TestPyPI to ensure that your package is available and has been updated to the new version.
+   -  Test installing the package using `pip` to ensure everything works as expected.

.github/workflows/periodic_benchmarks.yaml

@@ -0,0 +1,100 @@
+# Initial Source: pybop-team/PyBop
+
+# This workflow periodically runs the benchmarks suite in benchmarks/
+# using asv and publish the results, effectively updating
+# the display website hosted in the pybop-bench repo
+
+# Steps:
+# - Benchmark all commits since the last one that was benchmarked
+# - Push results to pybop-bench repo
+# - Publish website
+name: Benchmarks
+on:
+  # Everyday at 12 pm UTC
+  schedule:
+    - cron: "0 12 * * *"
+  # Make it possible to trigger the
+  # workflow manually
+  workflow_dispatch:
+
+jobs:
+  benchmarks:
+    runs-on: [self-hosted, macOS, ARM64]
+    if: github.repository == 'pybop-team/PyBOP'
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install python & create virtualenv
+        shell: bash
+        run: |
+            eval "$(pyenv init -)"
+            pyenv install 3.12 -s
+            pyenv virtualenv 3.12 pybop-312-bench
+
+      - name: Install dependencies & run benchmarks
+        shell: bash
+        run: |
+            eval "$(pyenv init -)"
+            pyenv activate pybop-312-bench
+            python -m pip install -e .[all,dev]
+            python -m pip install asv[virtualenv]
+            python -m asv machine --machine "SelfHostedRunner"
+            python -m asv run --machine "SelfHostedRunner" NEW --show-stderr -v
+
+      - name: Upload results as artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: asv_periodic_results
+          path: results
+
+      - name: Uninstall pyenv-virtualenv & python
+        if: always()
+        shell: bash
+        run: |
+            eval "$(pyenv init -)"
+            pyenv activate pybop-312-bench
+            pyenv uninstall -f $( python --version )
+
+  publish-results:
+    name: Push and publish results
+    needs: benchmarks
+    runs-on: ubuntu-latest
+    if: github.repository == 'pybop-team/PyBOP'
+    steps:
+      - name: Set up Python 3.12
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.12
+
+      - name: Install asv
+        run: pip install asv
+
+      - name: Checkout pybop-bench repo
+        uses: actions/checkout@v4
+        with:
+          repository: pybop-team/pybop-bench
+          token: ${{ secrets.PUSH_BENCH_TOKEN }}
+
+      - name: Download results artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: asv_periodic_results
+          path: new_results
+
+      - name: Copy new results and push to pybop-bench repo
+        env:
+          PUSH_BENCH_EMAIL: ${{ secrets.PUSH_BENCH_EMAIL }}
+          PUSH_BENCH_NAME: ${{ secrets.PUSH_BENCH_NAME }}
+        run: |
+          cp -vr new_results/* results
+          git config --global user.email "$PUSH_BENCH_EMAIL"
+          git config --global user.name "$PUSH_BENCH_NAME"
+          git add results
+          git commit -am "Add new benchmark results"
+          git push
+
+      - name: Publish results
+        run: |
+          asv publish
+          git fetch origin gh-pages:gh-pages
+          asv gh-pages

.github/workflows/release-action.yaml → .github/workflows/release_action.yaml

@@ -15,15 +15,15 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v4
       with:
-        python-version: "3.8"
+        python-version: "3.12"
     - name: Install pypa/build
       run: >-
         python3 -m
         pip install
         build
         --user
     - name: Build a binary wheel and a source tarball
-      run: python3 -m build
+      run: python -m build
     - name: Store the distribution packages
       uses: actions/upload-artifact@v3
       with:
@@ -35,9 +35,8 @@ jobs:
       Publish Python 🐍 distribution 📦 to PyPI
     if: >
       startsWith(github.ref, 'refs/tags/') &&
-      !contains(github.ref, 'rc')
-    needs:
-    - build
+      !contains(github.ref, 'rc') && github.repository == github.event.repository
+    needs: build
     runs-on: ubuntu-latest
     environment:
       name: pypi
@@ -91,7 +90,7 @@ jobs:
 
   publish-to-testpypi:
     name: Publish Python 🐍 distribution 📦 to TestPyPI
-    if: contains(github.ref, 'rc')  # only publish to TestPyPI for rc tags
+    if: contains(github.ref, 'rc') && github.repository == github.event.repository  # only publish to TestPyPI for rc tags
     needs:
     - build
     runs-on: ubuntu-latest

.github/workflows/scheduled_tests.yaml

@@ -6,65 +6,125 @@ on:
     branches:
       - main
 
-  # runs every day at 09:00 UTC
+  # runs every day at 09:00 and 15:00 UTC
   schedule:
     - cron: '0 9 * * *'
+    - cron: '0 15 * * *'
+
+# Check noxfile.py for associated environment variables
+env:
+  PYBOP_SCHEDULED: 1
 
 jobs:
+  # Dynamically create a matrix of OS, Python, and PyBaMM versions
+  create_pybamm_matrix:
+    name: Dynamically create GitHub Actions matrix
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out PyBOP repository
+        uses: actions/checkout@v4
+        with:
+          sparse-checkout-cone-mode: false
+          sparse-checkout: |
+            scripts/ci/build_matrix.sh
+
+      - name: Run script to create matrix
+        id: set-matrix
+        run: |
+          echo "matrix=$(bash scripts/ci/build_matrix.sh)" >> "$GITHUB_OUTPUT"
+    outputs:
+      pybop_matrix: ${{ steps.set-matrix.outputs.matrix }}
+
+  # filter the matrix to only include the macOS-latest entries
+  filter_pybamm_matrix:
+    name: Filter the matrix for macOS-latest entries
+    needs: [create_pybamm_matrix]
+    runs-on: ubuntu-latest
+    outputs:
+      filtered_pybop_matrix: ${{ steps.set-matrix.outputs.matrix }}
+    steps:
+      - name: Filter pybop matrix
+        id: set-matrix
+        run: |
+          import json
+          import os
+
+          # Get the matrix
+          matrix_json = '${{ needs.create_pybamm_matrix.outputs.pybop_matrix }}'
+          matrix = json.loads(matrix_json)
+
+          # Filter the matrix for macOS-latest entries only
+          filtered_entries = [entry for entry in matrix['include'] if entry['os'] == 'macos-latest']
+          filtered_matrix = {'include': filtered_entries}
+
+          # Set the output variable for other jobs to use
+          output_file = os.environ['GITHUB_OUTPUT']
+          with open(output_file, "a", encoding="utf-8") as output_stream:
+              output_stream.write(f"matrix={json.dumps(filtered_matrix)}\n")
+        shell: python
+
   build:
+    needs: [create_pybamm_matrix, filter_pybamm_matrix]
+    name: Build (${{ matrix.os }}, Python ${{ matrix.python_version }}, PyBaMM ${{ matrix.pybamm_version }})
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
-      matrix:
-        os: [ubuntu-latest, windows-latest, macos-latest]
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+      matrix: ${{fromJson(needs.create_pybamm_matrix.outputs.pybop_matrix)}}
+    env:
+      PYBAMM_VERSION: ${{ matrix.pybamm_version }}
 
     steps:
       - uses: actions/checkout@v4
-      - name: Set up Python ${{ matrix.python-version }}
+      - name: Set up Python ${{ matrix.python_version }}
         uses: actions/setup-python@v4
         with:
-          python-version: ${{ matrix.python-version }}
+          python-version: ${{ matrix.python_version }}
+
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip nox
+
       - name: Unit tests with nox
-        run: |
-          python -m nox -s unit
-          python -m nox -s notebooks
+        run: python -m nox -s coverage
 
-  #M-series Mac Mini
+      - name: Run notebooks with nox
+        run: python -m nox -s notebooks
+
+  # M-series Mac Mini
   build-apple-mseries:
+    needs: [filter_pybamm_matrix]
+    name: Build (MacOS M-series, Python ${{ matrix.python_version }}, PyBaMM ${{ matrix.pybamm_version }})
     runs-on: [self-hosted, macOS, ARM64]
+    if: github.repository == 'pybop-team/PyBOP'
     env:
       GITHUB_PATH: ${PYENV_ROOT/bin:$PATH}
+      PYBAMM_VERSION: ${{ matrix.pybamm_version }}
     strategy:
       fail-fast: false
-      matrix:
-        python-version: ["3.10"]
+      matrix: ${{fromJson(needs.filter_pybamm_matrix.outputs.filtered_pybop_matrix)}}
 
     steps:
       - uses: actions/checkout@v4
       - name: Install python & create virtualenv
         shell: bash
         run: |
           eval "$(pyenv init -)"
-          pyenv install ${{ matrix.python-version }} -s
-          pyenv virtualenv ${{ matrix.python-version }} pybop-${{ matrix.python-version }}
+          pyenv install ${{ matrix.python_version }} -s
+          pyenv virtualenv ${{ matrix.python_version }} pybop-${{ matrix.python_version }}-${{ matrix.pybamm_version }}
 
-      - name: Install dependencies & run unit tests
+      - name: Install dependencies & run unit + notebook tests
         shell: bash
         run: |
           eval "$(pyenv init -)"
-          pyenv activate pybop-${{ matrix.python-version }}
-          python -m pip install --upgrade pip wheel setuptools nox
-          python -m nox -s unit
+          pyenv activate pybop-${{ matrix.python_version }}-${{ matrix.pybamm_version }}
+          python -m pip install --upgrade pip nox
+          python -m nox -s coverage
           python -m nox -s notebooks
 
       - name: Uninstall pyenv-virtualenv & python
         if: always()
         shell: bash
         run: |
           eval "$(pyenv init -)"
-          pyenv activate pybop-${{ matrix.python-version }}
+          pyenv activate pybop-${{ matrix.python_version }}-${{ matrix.pybamm_version }}
           pyenv uninstall -f $( python --version )