Change: Use external project MatBox for developer tools (#27)

* Update tasks to use MatBox * Add upload of code issues report in "update" workflow * Move installMatBox to +ommtools namespace * Add codespellToolbox function * Delete writeBadgeJSONFile.m * Fix code issue * Minor change * Add pre-commit hook * Update code issues and tests badges * Update testToolbox.m * Remove functions for releasing that are now part of MatBox * Update MLToolboxInfo.json - Add ToolboxImageFile and ToolboxGettingStartedGuide to MLToolboxInfo * Update packageToolbox to use MatBox * rename dev to tools * Format whitespace * Update code issues and tests badges * Delete mustBeValidVersionNumber.m * Update release workflow * Update code issues and tests badges * Update badges in README.md * Create codecov.yml * Update release workflow based on latest release workflow in MatBox * Delete deprecated json badges * Update update.yml * Fix renamed files * Add test class for internal functions --------- Co-authored-by: Update openMINDS_MATLAB by ehennestad <>
openMetadataInitiative · Oct 30, 2024 · fbc8b6d · fbc8b6d
1 parent 209d3cb
commit fbc8b6d
Show file tree

Hide file tree

Showing 52 changed files with 378 additions and 696 deletions.
diff --git a/.github/badges/code_issues.svg b/.github/badges/code_issues.svg
diff --git a/.github/badges/tests.svg b/.github/badges/tests.svg
diff --git a/.github/badges/v0.9.3/tested_with.json b/.github/badges/v0.9.3/tested_with.json
diff --git a/.github/codecov.yml b/.github/codecov.yml
@@ -0,0 +1,12 @@
+coverage:
+   range: "75...100"
+   status:
+     project:
+       default:
+         target: 30 # Set the desired coverage target as 40%
+         threshold: 2 # Allowable drop in coverage
+     patch:
+       default:
+         # 50% of the changed code must be covered by tests
+         threshold: 50
+         only_pulls: true
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -1,7 +1,7 @@
 # Test openMINDS MATLAB Toolbox across all supported releases of MATLAB, package toolbox, create release
 # Adapted from: https://github.com/mathworks/climatedatastore/blob/main/.github/workflows/release.yml
 
-name: MATLAB Release
+name: Create new release
 
 # Run workflow when a tag is created
 on:
@@ -23,7 +23,8 @@ jobs:
 
     steps:
       # Checks-out the repository under $GITHUB_WORKSPACE, so the job can access it
-      - uses: actions/checkout@v4
+      - name: Checkout repository
+        uses: actions/checkout@v4
 
       - name: Set up MATLAB
         uses: matlab-actions/setup-matlab@v2
@@ -34,7 +35,9 @@ jobs:
       - name: Run tests
         uses: matlab-actions/run-command@v2
         with:
-          command: addpath(genpath("dev")), testToolbox('ReportSubdirectory',"${{ matrix.MATLABVersion }}")     
+          command: |
+            addpath(genpath("tools"));
+            testToolbox('ReportSubdirectory',"${{ matrix.MATLABVersion }}", 'CreateBadge', false)     
 
       # Upload code coverage information to Codecov
       - name: Upload code coverage report to Codecov (https://app.codecov.io/gh/openMetadataInitiative/openMINDS_MATLAB)
@@ -46,7 +49,7 @@ jobs:
           env_vars: ${{ matrix.MATLABVersion }}
 
       # Save the contents of the report directory from each release into an artifact.
-      - name: Save Report Directory
+      - name: Save report directory
         uses: actions/upload-artifact@v4
         if: always()
         with:
@@ -62,10 +65,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v4
+      # Use deploy key to push back to protected branch
+      - name: Checkout repository using deploy key
+        uses: actions/checkout@v4
         with:
-            ref: refs/heads/main
-
+          ref: refs/heads/main
+          ssh-key: ${{ secrets.DEPLOY_KEY }}
+
       - name: Set up MATLAB
         uses: matlab-actions/setup-matlab@v2
 
@@ -80,20 +86,20 @@ jobs:
       - name: Generate tested with badge
         uses: matlab-actions/run-command@v2
         with:
-          command: addpath(genpath("dev")), createTestedWithBadgeforToolbox("${{  github.ref_name }}")     
-          
+          command: addpath(genpath("tools")), createTestedWithBadgeforToolbox("${{  github.ref_name }}")     
+
       # Publish test results from all the releases
-      - name: Publish Test Results
+      - name: Publish test results
         uses: EnricoMi/publish-unit-test-result-action@v2
         if: always()
         with:
           junit_files: "docs/reports/*/test-results.xml"
 
       # Package the MLTBX
-      - name: Package Toolbox
+      - name: Package toolbox
         uses: matlab-actions/run-command@v2
         with:
-          command: addpath(genpath("dev")), packageToolbox("specific","${{  github.ref_name }}")     
+          command: addpath(genpath("tools")), packageToolbox("specific","${{  github.ref_name }}")     
 
       # Define the versionNumber using underscores, as this is used in the MLTBX
       - name: Set version number
@@ -102,45 +108,74 @@ jobs:
           versionNumber=$(echo "${{ github.ref_name }}" | sed 's/\./_/g')
           echo "versionNumber=$versionNumber" >> $GITHUB_ENV
 
-
       # Save the MLTBX.
-      - name: Save Packaged Toolbox
+      - name: Save packaged toolbox
         uses: actions/upload-artifact@v4
         with:
           name: openMINDS_MATLAB_${{ env.versionNumber }}.mltbx
           path: releases/openMINDS_MATLAB_${{ env.versionNumber }}.mltbx
 
-      # Commit the JSON for the MATLAB releases badge and ToolboxPackaging.prj
-      - name: commit changed files
+      # Commit the updated Contents.m
+      - name: Commit updated Contents.m file
         continue-on-error: true
         run: |
           git config user.name "${{ github.workflow }} by ${{ github.actor }}"
           git config user.email "<>"
           git status
           git add code/Contents.m
-          git add .github/badges/${{  github.ref_name }}/tested_with.json
           git commit -m "Final checkins for release ${{  github.ref_name }}"
           git fetch
           git push
 
+      # Commit the JSON for the MATLAB releases test badge to gh-badges branch
+      - name: Checkout gh-badges branch
+        uses: actions/checkout@v4
+        with:
+          ref: gh-badges
+          path: gh-badges
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Push to gh-badges
+        run: |
+          mkdir -p gh-badges/.github/badges/${{  github.ref_name }}
+          cp .github/badges/${{  github.ref_name }}/tested_with.json gh-badges/.github/badges/${{  github.ref_name }}/tested_with.json
+          cd gh-badges
+
+          git config user.name "${{ github.workflow }} by ${{ github.actor }}"
+          git config user.email "<>"
+
+          # Only proceed with commit and push if changes are detected
+          if [[ $(git add .github/badges/* --dry-run | wc -l) -gt 0 ]]; then
+            git add .github/badges/*
+            git commit -m "Update tested with badge for release"
+            git push -f
+          else
+            echo "Nothing to commit"
+          fi
+
       # Retag the repo so that the updated files are included in the release tag
-      - name: update tag
+      - name: Update tag
         if: always()
         continue-on-error: true
         run: |
           git config user.name "${{ github.workflow }} by ${{ github.actor }}"
           git config user.email "<>"
-          git tag -d "${{  github.ref_name }}" 
-          git push --delete origin ${{  github.ref_name }}
-          git tag -m "Release ${{  github.ref_name }}" ${{  github.ref_name }}
-          git push --tag
+
+          # Delete the existing tag locally and remotely
+          git tag -d "${{ github.ref_name }}" 
+          git push origin --delete "${{ github.ref_name }}"
+
+          # Recreate the tag with a message, including [skip ci] to prevent CI workflows
+          git tag -a "${{ github.ref_name }}" -m "Release ${{ github.ref_name }} [skip ci]" 
+
+          # Push the new tag to the remote repository
+          git push origin "${{ github.ref_name }}"
 
       # Create the release
-      - name: Create GitHub Release
+      - name: Create GitHub release
         uses: ncipollo/release-action@v1
         with:
           draft: true        
           artifacts: "releases/openMINDS_MATLAB_${{ env.versionNumber }}.mltbx"
           generateReleaseNotes: true
-          body: "![MATLAB Versions Tested](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2FopenMetadataInitiative%2FopenMINDS_MATLAB%2Fmain%2F.github%2Fbadges%2F${{  github.ref_name }}%2Ftested_with.json)"
-
+          body: "![MATLAB Versions Tested](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2FopenMetadataInitiative%2FopenMINDS_MATLAB%2Fgh-badges%2F.github%2Fbadges%2F${{  github.ref_name }}%2Ftested_with.json)"
diff --git a/.github/workflows/update.yml b/.github/workflows/update.yml
@@ -1,16 +1,17 @@
 name: Update openMINDS_MATLAB
 
 on:
-  # Triggers the workflow on push or pull request events for the "main" branch
+  # Triggers the workflow on push to any branch other than main or PR to main
+  # This way tests will run on any push to a feature branch and when feature 
+  # branches are merged to main, but not repeated when PRs are actually merged. 
   push:
-    branches-ignore: ["pipeline"]
+    branches-ignore: ["main", "pipeline", "gh-badges"]
     paths-ignore:
       - '*README.md'
-      - '.github/workflows/**'
-      - 'docs/reports/**'
+      - '*md'
+      - '.github/**'
   pull_request:
     branches: [ "main" ]
-
   # Allows for manually running this workflow from the Actions tab
   workflow_dispatch:
 
@@ -25,7 +26,7 @@ jobs:
     steps:
       # Checks-out the repository under $GITHUB_WORKSPACE, so the job can access it
       - name: Check out repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Set up MATLAB
         uses: matlab-actions/setup-matlab@v2
@@ -35,27 +36,38 @@ jobs:
         uses: matlab-actions/run-command@v2
         if: always()
         with:
-          command: addpath(genpath("dev")),codecheckToolbox()     
+          command: addpath(genpath("tools")),codecheckToolbox()
+
+      # Upload code issues report
+      - name: Upload SARIF file
+        uses: github/codeql-action/upload-sarif@v3
+        with:
+          # Path to SARIF file relative to the root of the repository
+          sarif_file: docs/reports/code_issues.sarif
 
-      # Runs all tests in the project.
+      # Run all tests in the project.
       - name: Run tests
         uses: matlab-actions/run-command@v2
         if: always()
         with:
-          command: addpath(genpath("dev")), testToolbox()
+          command: addpath(genpath("tools")), testToolbox()
 
-      # Commit the JSON for the badge for the issues and tests
-      - name: commit code issues badge JSON
+      # Commit updated SVG badges for the issues and tests (if changed)
+      - name: Commit svg badges if updated
         if: always()
         continue-on-error: true
         run: |
           git config user.name "${{ github.workflow }} by ${{ github.actor }}"
           git config user.email "<>"
-          git add docs/reports/badge/code_issues.json docs/reports/badge/tests.json
-          git commit -m "Update code issues and tests badges"
           git fetch
-          # git push returns error code 1 if there's nothing to push (which there often isn't in this case)
-          git push || true
+
+          if [[ $(git add .github/badges/* --dry-run | wc -l) -gt 0 ]]; then
+            git add .github/badges/*
+            git commit -m "Update code issues and tests badges"
+            git push -f
+          else
+            echo "Nothing to commit"
+          fi
 
       - name: Upload code coverage report to Codecov (https://app.codecov.io/gh/openMetadataInitiative/openMINDS_MATLAB)
         uses: codecov/codecov-action@v4
@@ -65,16 +77,41 @@ jobs:
           files: docs/reports/codecoverage.xml
 
       # Publish test results
-      - name: Publish Test Results
+      - name: Publish test results
         uses: EnricoMi/publish-unit-test-result-action@v2
         if: always()
         with:
           files: "docs/reports/test-results.xml"
 
       # Save the contents of the reports directory as an artifact
-      - name: Save Reports Directory
+      - name: Save reports directory
         uses: actions/upload-artifact@v4
         if: always()
         with:
           name: reports
           path: docs/reports
+
+      - name: Checkout gh-badges branch
+        uses: actions/checkout@v4
+        with:
+          ref: gh-badges
+          path: gh-badges
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Push to gh-badges
+        run: |
+          cp .github/badges/code_issues.svg gh-badges/.github/badges/code_issues.svg
+          cp .github/badges/tests.svg gh-badges/.github/badges/tests.svg
+          cd gh-badges
+
+          git config user.name "${{ github.workflow }} by ${{ github.actor }}"
+          git config user.email "<>"
+
+          # Only proceed with commit and push if changes are detected
+          if [[ $(git add .github/badges/* --dry-run | wc -l) -gt 0 ]]; then
+            git add .github/badges/*
+            git commit -m "Update code issues and tests badges"
+            git push -f
+          else
+            echo "Nothing to commit"
+          fi
diff --git a/.gitignore b/.gitignore
@@ -12,9 +12,6 @@ releases/
 # Ignore everything in docs/reports
 docs/reports/*
 
-# Don't ignore the badge file/directory
-!docs/reports/badge
-
 # Autosave files
 *.asv
 *.m~

diff --git a/README.md b/README.md
@@ -10,24 +10,37 @@
 [![Version Number](https://img.shields.io/github/v/release/openMetadataInitiative/openMINDS_MATLAB?label=version)](https://github.com/openMetadataInitiative/openMINDS_MATLAB/releases/latest)
 [![Open in MATLAB Online](https://github.com/openMetadataInitiative/openMINDS_MATLAB/blob/gh-badges/.github/badges/open-in-matlab-online.svg)](https://matlab.mathworks.com/open/github/v1?repo=openMetadataInitiative/openMINDS_MATLAB&file=code/gettingStarted.mlx) 
 [![View openMINDS_MATLAB on File Exchange](https://github.com/openMetadataInitiative/openMINDS_MATLAB/blob/gh-badges/.github/badges/matlab-file-exchange.svg)](https://se.mathworks.com/matlabcentral/fileexchange/134212-openminds_matlab)
-![MATLAB Tests](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2FopenMetadataInitiative%2FopenMINDS_MATLAB%2Fgh-badges%2F.github%2Fbadges%2Ftests.json)
+[![MATLAB Tests](.github/badges/tests.svg)](https://github.com/openMetadataInitiative/openMINDS_MATLAB/actions/workflows/update.yml)
 [![codecov](https://codecov.io/gh/openMetadataInitiative/openMINDS_MATLAB/graph/badge.svg?token=FTD5FHZSFA)](https://codecov.io/gh/openMetadataInitiative/openMINDS_MATLAB)
-![MATLAB Code Issues](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2FopenMetadataInitiative%2FopenMINDS_MATLAB%2Fgh-badges%2F.github%2Fbadges%2Fcode_issues.json)
+[![MATLAB Code Issues](.github/badges/code_issues.svg)](https://github.com/openMetadataInitiative/openMINDS_MATLAB/security/code-scanning)
 [![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://gitHub.com/openMetadataInitiative/openMINDS_MATLAB/graphs/commit-activity)
 
+<p align="center">
+  <a href="#requirements-and-installation">Requirements and Installation</a> •
+  <a href="#before-you-start">Before you start</a> •
+  <a href="#getting-started">Getting Started</a> •
+  <a href="#acknowledgements">Acknowledgements</a>
+</p>
+
+---
+
+
 MATLAB package for the openMINDS metadata framework for linked data in neuroscience. The package contains all the latest openMINDS schemas as MATLAB classes in addition to schema base classes and utility methods.
 
 To generally learn more about the openMINDS metadata framework please go to :arrow_right: [**ReadTheDocs**](https://openminds-documentation.readthedocs.io).  
 You can test and learn how to use openMINDS in MATLAB by going through a small :arrow_right: [**DEMO**](https://matlab.mathworks.com/open/github/v1?repo=openMetadataInitiative/openMINDS_MATLAB&file=code/gettingStarted.mlx) on MATLAB Online.  
 
 MathWorks provides a free basic version of [MATLAB Online](https://uk.mathworks.com/products/matlab-online.html), but you need to register for a [MathWorks Account](https://www.mathworks.com/mwaccount/register?uri=https%3A%2F%2Fwww.mathworks.com%2Fproducts%2Fmatlab.html).
-
+<p align="right">
 
 ## Requirements and Installation
-openMINDS for MATLAB requires **MATLAB R2022b** or later. The toolbox can be installed from MATLAB's [Add-On Explorer](https://se.mathworks.com/help/matlab/matlab_env/get-add-ons.html) (recommended). It is also possible to download the MATLAB toolbox from [FileExchange](https://se.mathworks.com/matlabcentral/fileexchange/134212-openminds_matlab) or from the [Releases](https://github.com/openMetadataInitiative/openMINDS_MATLAB/releases/latest) page of this repository and install it manually. If you are new to MATLAB, see the detailed [installation instructions](#Detailed-Installation-Instructions)
+[(Back to top)](#openminds-metadata-models-for-matlab)
 
+openMINDS for MATLAB requires **MATLAB R2022b** or later. The toolbox can be installed from MATLAB's [Add-On Explorer](https://se.mathworks.com/help/matlab/matlab_env/get-add-ons.html) (recommended). It is also possible to download the MATLAB toolbox from [FileExchange](https://se.mathworks.com/matlabcentral/fileexchange/134212-openminds_matlab) or from the [Releases](https://github.com/openMetadataInitiative/openMINDS_MATLAB/releases/latest) page of this repository and install it manually. If you are new to MATLAB, see the detailed [installation instructions](#Detailed-Installation-Instructions)
 
 ## Before you start
+[(Back to top)](#openminds-metadata-models-for-matlab)
+
 ```matlab
 % Verify that openMINDS_MATLAB is installed
 disp( openminds.toolboxversion )
@@ -41,6 +54,7 @@ If you have installed **openMINDS_MATLAB** and the above command does not work,
 
 
 ## Getting Started
+[(Back to top)](#openminds-metadata-models-for-matlab)
 
 See also: [Crew Member Collection Tutorial](./docs/tutorials/crewMemberCollection.md)
 
@@ -153,13 +167,16 @@ disp(subject1)
 ```
 
 ## Detailed Installation Instructions
+[(Back to top)](#openminds-metadata-models-for-matlab)
+
 The easiest way to install the openMINDS for MATLAB is to use the [**Add-on Explorer**](https://www.mathworks.com/products/matlab/add-on-explorer.html):
 1. Launch the Add-on Explorer from MATLAB's Home tab. Click Add-Ons -> Get Add-Ons<img width="860" alt="openminds_installation_step1" src="https://github.com/openMetadataInitiative/openMINDS_MATLAB/assets/17237719/71e7d8a3-1548-44e4-8ad2-84798773ce90">
 2. Search for "openminds"
 3. Select openMINDS Metadata Models for MATLAB<img width="860" alt="openminds_installation_step2" src="https://github.com/openMetadataInitiative/openMINDS_MATLAB/assets/17237719/f46eb742-b2c8-47a1-a46b-c96a3d4c0b35">
 4. Press the "Add" button.<img width="860" alt="openminds_installation_step4" src="https://github.com/openMetadataInitiative/openMINDS_MATLAB/assets/17237719/cc2edc9e-4a5d-43cd-a3fc-a94a52d8d84a">
 
 ## Acknowledgements
+[(Back to top)](#openminds-metadata-models-for-matlab)
 
 <div><img src="https://www.braincouncil.eu/wp-content/uploads/2018/11/wsi-imageoptim-EU-Logo.jpg" alt="EU Logo" height="23%" width="15%" align="right" style="margin-left: 10px"></div>