Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Add workflow for releases from GHA #1117

Merged
merged 18 commits into from
Jul 4, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions .github/workflows/make_release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: Make release

on:
push:
tags:
- 'v*'
pull_request:
paths:
- .github/workflows/make_release.yml

jobs:
build_wheels: # This job builds the wheels
runs-on: ubuntu-latest
permissions:
attestations: write
id-token: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: hynek/build-and-inspect-python-package@v2
with:
attest-build-provenance-github: 'true'

Comment on lines +18 to +24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a step to set up Python.

The job is missing a step to set up the Python environment before using hynek/build-and-inspect-python-package@v2. This could lead to issues if the correct Python version is not available.

      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
+     - uses: actions/setup-python@v5
+       with:
+         python-version: "3.10"
      - uses: hynek/build-and-inspect-python-package@v2
        with:
          attest-build-provenance-github: 'true'
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: hynek/build-and-inspect-python-package@v2
with:
attest-build-provenance-github: 'true'
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.10"
- uses: hynek/build-and-inspect-python-package@v2
with:
attest-build-provenance-github: 'true'


build_pyinstaller_bundle:
name: Build PyInstaller bundle
strategy:
fail-fast: false
matrix:
os: ["ubuntu-20.04", "windows-2019", "macos-13"] #, "macos-14"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.10"
- uses: tlambert03/setup-qt-libs@v1
- name: Install Windows OpenGL
if: runner.os == 'Windows'
run: |
git clone --depth 1 https://github.com/pyvista/gl-ci-helpers.git
powershell gl-ci-helpers/appveyor/install_opengl.ps1
if (Test-Path -Path "C:\Windows\system32\opengl32.dll" -PathType Leaf) {Exit 0} else {Exit 1}
shell: powershell
- name: Install PartSeg
run: python -m pip install -e .[pyinstaller] -c requirements/constraints_py3.10.txt
Copy link
Contributor

@coderabbitai coderabbitai bot Jul 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin the versions of dependencies.

The pip install command does not pin the versions of dependencies. This could lead to issues with future updates.

      - name: Install PartSeg
        run: python -m pip install -e .[pyinstaller] -c requirements/constraints_py3.10.txt
+       run: python -m pip install -r requirements.txt

Committable suggestion was skipped due to low confidence.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it pin using constraints file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Czaki, you are correct. The requirements/constraints_py3.10.txt file does pin the versions of the dependencies. Thank you for the clarification.

Analysis chain
Scripts executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Display the contents of the constraints file to verify if it pins the versions of dependencies.
cat requirements/constraints_py3.10.txt

Length of output: 8043


Learnings added
Learnt from: Czaki
PR: 4DNucleome/PartSeg#1117
File: .github/workflows/make_release.yml:46-47
Timestamp: 2024-07-03T14:07:21.406Z
Learning: The `requirements/constraints_py3.10.txt` file is used to pin the versions of dependencies in the `pip install` command for the PartSeg project.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


- name: Build PartSeg bundle
run: python build_utils/create_and_pack_executable.py

- name: Publish PartSeg bundle
uses: actions/upload-artifact@v4
with:
name: PartSeg_bundle_${{ matrix.os }}
path: dist2

- name: Test PartSeg bundle
uses: aganders3/headless-gui@v2
with:
run: dist/PartSeg/PartSeg _test



create_release:
name: Create release
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
needs:
- build_wheels
- build_pyinstaller_bundle
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
patterns: 'PartSeg_bundle_*'
path: pyinstaller
merge-multiple: true
- uses: actions/download-artifact@v4
with:
name: Packages
path: dist
- name: generate release notes
run: |
RELEASE_NOTES=$(python build_utils/cut_changelog.py)
echo "${RELEASE_NOTES}"
echo "name=contents=${RELEASE_NOTES}" >> $GITHUB_ENV

- name: Create Release
uses: "softprops/action-gh-release@v2"
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
with:
tag_name: ${{ github.ref }}
name: ${{ env.tag }}
body: ${{ steps.release_notes.outputs.contents }}
draft: false
prerelease: ${{ contains(github.ref, 'rc') }}
files: |
dist/*
pyinstaller/*

- name: Publish package distributions to PyPI
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
uses: pypa/gh-action-pypi-publish@release/v1
2 changes: 1 addition & 1 deletion Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ This application is designed to help biologist with segmentation based on thresh

- [Windows](https://github.com/4DNucleome/PartSeg/releases/latest/download/PartSeg-windows.zip) (build on Windows 10)
- [Linux](https://github.com/4DNucleome/PartSeg/releases/latest/download/PartSeg-linux.zip) (build on Ubuntu 20.04)
- [macOS](https://github.com/4DNucleome/PartSeg/releases/latest/download/PartSeg-macos.zip) (build on macOS 11)
- [macOS](https://github.com/4DNucleome/PartSeg/releases/latest/download/PartSeg-macos.zip) (build on macOS 13)
There are reported problems with permissions systems on macOS. If you have a problem with starting the application, please try to run it from the terminal.

- With pip:
Expand Down
3 changes: 3 additions & 0 deletions build_utils/create_and_pack_executable.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ def create_archive(working_dir, with_version=True):
file_name = f"PartSeg-{PartSeg.__version__}-{SYSTEM_NAME_DICT[platform.system()]}"
else:
file_name = f"PartSeg-{SYSTEM_NAME_DICT[platform.system()]}"
# if it is macos and arm64 the add arm64 to the name
if platform.system() == "Darwin" and os.uname().machine == "arm64":
file_name += "-arm64"
if platform.system() != "Darwin":
return zipfile.ZipFile(os.path.join(working_dir, "dist2", f"{file_name}.zip"), "w", zipfile.ZIP_DEFLATED)
arch_file = tarfile.open(os.path.join(working_dir, "dist2", f"{file_name}.tgz"), "w:gz")
Expand Down
103 changes: 103 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,108 @@
# Changelog

## 0.15.3 - 2024-07-03

### 🚀 Features

- Pydantic 2 compatybility (#1084)

### 🐛 Bug Fixes

- Fix rendering icons in colormap preview (#1040)
- Fix test for validation length of message for sentry-sdk 2.0 release (#1098)
- When fix reader check lowercase extension for validate compatybility (#1097)
- Fix napari 0.5.0 compatybility (#1116)

### 🚜 Refactor

- Fix Qt flags (#1041)
- Fix qt flags in roi mask code (#1042)
- Fix qt flags in roi analysis (#1043)
- Migrate from setup.cfg to `pyproject.toml` (#1070)

### 📚 Documentation

- Allow to use newer release of build docs dependencies (#1057)

### 🧪 Testing

- \[Automatic\] Constraints upgrades: `imagecodecs`, `imageio`, `ipykernel`, `ipython`, `numpy`, `oiffile`, `pandas`, `psygnal`, `pyinstaller`, `qtconsole`, `qtpy`, `sentry-sdk`, `simpleitk`, `superqt`, `tifffile`, `xlsxwriter` (#1020)
- \[Automatic\] Constraints upgrades: `h5py`, `imageio`, `ipython`, `numpy`, `packaging`, `pydantic`, `pyinstaller`, `pyqt5`, `scipy`, `sentry-sdk`, `superqt`, `tifffile`, `xlsxwriter` (#1027)
- \[Automatic\] Constraints upgrades: `imageio`, `magicgui`, `xlsxwriter` (#1030)
- \[Automatic\] Constraints upgrades: `ipykernel`, `pandas`, `qtpy` (#1032)
- \[Automatic\] Constraints upgrades: `imageio`, `ipykernel`, `ipython`, `numpy`, `pandas`, `psygnal`, `pygments`, `pyinstaller`, `qtconsole`, `scipy`, `sentry-sdk`, `simpleitk` (#1035)
- \[Automatic\] Constraints upgrades: `imagecodecs`, `imageio`, `ipykernel`, `magicgui`, `pandas`, `pyinstaller`, `qtawesome`, `sentry-sdk`, `tifffile` (#1048)
- \[Automatic\] Constraints upgrades: `ipykernel`, `numpy`, `pandas`, `partsegcore-compiled-backend`, `pydantic`, `scipy`, `sentry-sdk` (#1058)
- Improve test of PartSegImage (#1072)
- Improve test suite for `PartSegCore` (#1077)
- \[Automatic\] Constraints upgrades: `imageio`, `ipykernel`, `local-migrator`, `napari`, `numpy`, `pandas`, `partsegcore-compiled-backend`, `pyinstaller`, `sentry-sdk`, `tifffile`, `vispy`, `xlsxwriter` (#1063)
- \[Automatic\] Constraints upgrades: `magicgui`, `packaging`, `psygnal`, `pyinstaller`, `sentry-sdk`, `superqt` (#1086)
- \[Automatic\] Constraints upgrades: `psygnal`, `pydantic`, `sentry-sdk`, `vispy` (#1090)
- \[Automatic\] Constraints upgrades: `h5py`, `ipykernel`, `mahotas`, `pandas`, `psygnal`, `pydantic`, `pyinstaller`, `qtawesome`, `scipy`, `sentry-sdk`, `superqt` (#1092)
- \[Automatic\] Constraints upgrades: `imageio`, `tifffile` (#1100)
- \[Automatic\] Constraints upgrades: `pydantic`, `sentry-sdk`, `superqt`, `tifffile` (#1102)
- \[Automatic\] Constraints upgrades: `psygnal`, `pygments`, `qtconsole`, `sentry-sdk`, `superqt`, `tifffile` (#1105)
- \[Automatic\] Constraints upgrades: `imagecodecs`, `magicgui`, `oiffile`, `openpyxl`, `packaging`, `pydantic`, `pyinstaller`, `requests`, `scipy`, `sentry-sdk`, `superqt`, `sympy`, `tifffile`, `vispy` (#1107)

### ⚙️ Miscellaneous Tasks

- \[pre-commit.ci\] pre-commit autoupdate (#1019)
- Remove plugin page preview as it is no longer maintained (#1021)
- \[pre-commit.ci\] pre-commit autoupdate (#1022)
- \[pre-commit.ci\] pre-commit autoupdate (#1026)
- \[pre-commit.ci\] pre-commit autoupdate (#1031)
- \[pre-commit.ci\] pre-commit autoupdate (#1034)
- Use new semgrep configuration (#1039)
- Upload raw coverage information (#1044)
- \[pre-commit.ci\] pre-commit autoupdate (#1036)
- Run coverage upload in separate steep (#1053)
- Generate local report in `Tests` workflow and use proper script for fetch report (#1054)
- Move coverage back to main workflow (#1055)
- \[pre-commit.ci\] pre-commit autoupdate (#1056)
- \[pre-commit.ci\] pre-commit autoupdate (#1059)
- Update `actions/upload-artifact` and `actions/download-artifact` from 3 to 4 (#1062)
- \[pre-commit.ci\] pre-commit autoupdate (#1064)
- Group actions update (#1065)
- \[pre-commit.ci\] pre-commit autoupdate (#1068)
- Remove requirement of 2 builds upload to codecov.io (#1073)
- Re add tests to coverage report (#1074)
- Switch from setup.cfg to pyproject.toml in workflows (#1076)
- Fix compiling pyinstaller pre-deps (#1075)
- Add codespell to pre-commit and fix pointed bugs (#1078)
- Add new ruff rules and apply them (#1079)
- \[pre-commit.ci\] pre-commit autoupdate (#1080)
- \[pre-commit.ci\] pre-commit autoupdate (#1081)
- Fix upgrade depenecies workflow (#1083)
- Block using `mpmath==1.4.0a0` and `sentry-sdk` 2.0.0a1/a2 in pre-test (#1085)
- \[pre-commit.ci\] pre-commit autoupdate (#1089)
- Fix jupyter failing test by using constraints (#1093)
- \[pre-commit.ci\] pre-commit autoupdate (#1091)
- \[pre-commit.ci\] pre-commit autoupdate (#1096)
- Add python 3.12 testing (#1087)
- Exclude pyside2 on python 3.11 and 3.12 from testing (#1099)
- \[pre-commit.ci\] pre-commit autoupdate (#1101)
- \[pre-commit.ci\] pre-commit autoupdate (#1103)
- Bump macos runners to macos-13 (both azure and GHA) (#1113)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix grammar: Capitalize "macOS".

The operating system from Apple is written “macOS”.

- Bump macos runners to macos-13 (both azure and GHA)
+ Bump macOS runners to macOS-13 (both azure and GHA)
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Bump macos runners to macos-13 (both azure and GHA) (#1113)
- Bump macos runners to macos-13 (both azure and GHA)
+ Bump macOS runners to macOS-13 (both azure and GHA)
Tools
LanguageTool

[grammar] ~85-~85: The operating system from Apple is written “macOS”.
Context: ...] pre-commit autoupdate (#1103) - Bump macos runners to macos-13 (both azure and GHA...

(MAC_OS)

- \[pre-commit.ci\] pre-commit autoupdate (#1108)
- Remove pyqt5 from constraints (#1118)

### 🛡️ Security

- *(deps)* Bump actions/checkout from 3 to 4 (#1029)
- *(deps)* Bump conda-incubator/setup-miniconda from 2 to 3 (#1038)
- *(deps)* Bump aganders3/headless-gui from 1 to 2 (#1047)
- *(deps)* Bump actions/checkout from 3 to 4 (#1045)
- *(deps)* Bump hynek/build-and-inspect-python-package from 1 to 2 (#1050)
- *(deps)* Bump actions/setup-python from 4 to 5 (#1046)
- *(deps)* Bump github/codeql-action from 2 to 3 (#1051)
- *(deps)* Bump peter-evans/create-pull-request from 5 to 6 (#1067)
- *(deps)* Bump codecov/codecov-action from 3 to 4 (#1066)

### Build

- Fix not bundling `Font Awesome 6 Free-Solid-900.otf` file to executable (#1114)
- Update readme and release to point to GitHub releases (#1115)

## 0.15.2 - 2023-08-28

### Bug Fixes
Expand Down
8 changes: 4 additions & 4 deletions launcher.spec
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ from packaging.version import parse as parse_version
import platform
import zmq
import itertools
import pkg_resources
import debugpy._vendored
import importlib.metadata

Expand Down Expand Up @@ -65,7 +64,7 @@ hiddenimports = (
[f"imagecodecs.{y}" for y in (x if x[0] == "_" else f"_{x}" for x in imagecodecs._extensions())]
+ ["imagecodecs._shared"]
+ plugins
+ ["pkg_resources.py2_warn", "scipy.special.cython_special", "ipykernel.datapub"]
+ ["pkg_resources.py2_warn", "ipykernel.datapub"]
+ [
"numpy.core._dtype_ctypes",
"sentry_sdk.integrations.logging",
Expand All @@ -84,7 +83,6 @@ hiddenimports = (
"nme",
"defusedxml.cElementTree",
"vispy.app.backends._pyqt5",
"scipy.spatial.transform._rotation_groups",
"magicgui.backends._qtpy",
"freetype",
"psygnal._signal",
Expand All @@ -94,7 +92,9 @@ hiddenimports = (
"PartSeg.plugins.napari_widgets",
"PartSegCore.napari_plugins",
]
+ [x.module_name for x in imageio_known_plugins.values()] + [x for x in collect_submodules("skimage") if "tests" not in x]
+ [x.module_name for x in imageio_known_plugins.values()]
+ [x for x in collect_submodules("skimage") if "tests" not in x]
+ collect_submodules("scipy")
)


Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ PartSeg = "PartSeg:napari.yaml"

[project.optional-dependencies]
accelerate = [
"PyOpenGL-accelerate>=3.1.5",
"PyOpenGL-accelerate>=3.1.5 ; platform_machine != 'arm64'",
]
all = [
"PyOpenGL-accelerate>=3.1.5",
Expand All @@ -115,7 +115,7 @@ pyinstaller = [
"PartSeg[pyinstaller_base,pyqt5]",
]
pyinstaller_base = [
"PyOpenGL-accelerate>=3.1.5",
"PyOpenGL-accelerate>=3.1.5 ; platform_machine != 'arm64'",
"PyInstaller",
"pydantic",
]
Expand Down
8 changes: 4 additions & 4 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,15 @@ def readme():
this_directory = os.path.abspath(os.path.dirname(__file__))
reg = re.compile(r"(!\[[^]]*\])\((images/[^)]*)\)")
reg2 = re.compile(r"releases/latest/download/PartSeg")
with open(os.path.join(this_directory, "Readme.md")) as f:
with open(os.path.join(this_directory, "Readme.md"), encoding="utf8") as f:
text = f.read()
text = reg.sub(r"\1(https://raw.githubusercontent.com/4DNucleome/PartSeg/master/\2)", text)
with contextlib.suppress(ImportError):
from setuptools_scm import get_version

text = reg2.sub(f"releases/download/v{get_version()}/PartSeg-{get_version()}", text)

with open(os.path.join(this_directory, "changelog.md")) as f:
with open(os.path.join(this_directory, "changelog.md"), encoding="utf8") as f:
chg = f.read()
text += "\n\n" + chg.replace("# ", "## ")
return text
Expand All @@ -35,9 +35,9 @@ def readme():
changelog_path = os.path.join(os.path.dirname(__file__), "changelog.md")
changelog_result_path = os.path.join(os.path.dirname(__file__), "package", "PartSeg", "changelog.py")
if os.path.exists(changelog_path):
with open(changelog_path) as ff:
with open(changelog_path, encoding="utf8") as ff:
changelog_str = ff.read()
with open(changelog_result_path, "w") as ff:
with open(changelog_result_path, "w", encoding="utf8") as ff:
ff.write(f'changelog = """\n{changelog_str}"""\n')


Expand Down
Loading