Add workflow to update major alias tags, templates #13

nwiltsie · 2024-09-27T17:12:25Z

Description

This one is a little more sprawling than I intended, so apologies for the scope creep. Everything is kind of interdependent.

The major changes in this PR are:

A new workflow to update major release tags. This workflow is triggered when a release is published or deleted. If the modified release has a semantic version tag (e.g. v1.2.3), then the workflow will update the matching alias tag v1 to point at the highest full release with that major version. This follow's GitHub's guidance for Action and workflow release management.
When preparing a release, prerelease is now a separate checkbox rather than a bump type choice like major/minor/patch. Previously choosing prerelease would only bump the patch number, so there was no way to create the version 2.0.0-rc.1.
- The resulting GitHub Release created from a prerelease tag will also be marked as a prerelease.
I've added workflow templates in the templates/ directory (closes Documentation suggestion: Adding YAML files #9)! Those can be copied verbatim into another repository and they'll work.
I've enabled Dependabot to update the bundled python package, the workflows used by this repository, and the templates! That means that the non-used-by-this-repository workflow files will still be kept up-to-date for us.
I've done a bunch of refactoring to make this more DRY and (hopefully) prevent more of the v-prefix nonsense from spreading.
Finally, I've added unit tests for the version bumping (including the new prerelease flag) and the aliasing.

I have one more feature to implement in a different PR (updating hardcoded version numbers in __version__.py and nextflow.config files), and then I believe these workflows will be ready for widespread use. Given that, please do dig into the nitty-gritty details.

I've been testing this with the https://github.com/uclahs-cds/docker-internal-tests repository - it is using duplicates of the new template workflows (modified to point to this branch). Please feel free to test the workflows by creating releases, deleting releases, etc. All of this is intended to be robust and handle edge cases gracefully.

Closes #9

Checklist

This PR does NOT contain Protected Health Information (PHI). A repo may need to be deleted if such data is uploaded.
Disclosing PHI is a major problem¹ - Even a small leak can be costly².
This PR does NOT contain germline genetic data³, RNA-Seq, DNA methylation, microbiome or other molecular data⁴.

This PR does NOT contain other non-plain text files, such as: compressed files, images (e.g. .png, .jpeg), .pdf, .RData, .xlsx, .doc, .ppt, or other output files.

To automatically exclude such files using a .gitignore file, see here for example.

I have read the code review guidelines and the code review best practice on GitHub check-list.
I have set up or verified the main branch protection rule following the github standards before opening this pull request.
The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].
I have added the major changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

UCLA Health reaches $7.5m settlement over 2015 breach of 4.5m patient records ↩
The average healthcare data breach costs $2.2 million, despite the majority of breaches releasing fewer than 500 records. ↩
Genetic information is considered PHI.
Forensic assays can identify patients with as few as 21 SNPs ↩
RNA-Seq, DNA methylation, microbiome, or other molecular data can be used to predict genotypes (PHI) and reveal a patient's identity. ↩

aholmes · 2024-09-27T20:03:35Z

.github/workflows/internal-prepare.yaml

@@ -1,5 +1,5 @@
 ---
-name: Prepare new release
+name: 📦 Prepare new release


note: I thought this was bread :(

ChatGPT told me to do it!

For a workflow that is responsible for creating software releases, the most appropriate emojis to highlight the purpose of the workflow would be:

🚀 :rocket:

Represents launching or releasing something, making it a popular and intuitive choice for release workflows.

🏷️ :label:

Represents a tag or label, which aligns well with tagging versions or releases in a repository.

📦 :package:

Symbolizes packaging or bundling software, often associated with creating and distributing software releases.

I was just sad it wasn't bread ):

aholmes · 2024-09-27T22:22:36Z

bumpchanges/alias.py

+from .utils import dereference_tags, tag_to_semver, get_github_releases, Release
+
+
+class IneligibleAlias(Exception):


suggestion: rename to IneligibleAliasError

Excellent suggestion - done with d42c04c.

aholmes · 2024-09-27T22:27:44Z

bumpchanges/alias.py

+        """Confirm that the collected data is in a reasonable state."""
+        # All releases must have corresponding git tags
+        if (
+            unknown_tags := self.tag_to_release_map.keys()


suggestion: though not required, usages of := can be clarified by grouping, e.g.,

if ( unknown_tags := ( self.tag_to_release_map.keys() - self.tag_to_commit_map.keys() ) ): ...

This makes the intention clear that unknown_tags is the result of unknown_tags = (a - b) and not (unknown_tags = a) - b

Yeah what I have here is misleading. Fixed up with fa2cceb.

aholmes · 2024-09-27T22:31:14Z

bumpchanges/alias.py

+        """Confirm that the collected data is in a reasonable state."""
+        # All releases must have corresponding git tags
+        if (
+            unknown_tags := self.tag_to_release_map.keys()


suggestion: The various usages of some_var.keys() might be more readable if the values are realized earlier, and once, e.g.:

release_keys = tag_to_release_map.keys() commit_keys = tag_to_commit_map.keys() version_keys = tag_to_version_map.keys() if (release_keys ... commit_keys): ... if (version_keys ... release_keys): ... # etc.

Good call, this is way more readable: 64f3d3e.

yashpatel6 · 2024-09-30T16:35:29Z

.github/workflows/wf-alias-release.yaml

+          repository: uclahs-cds/tool-create-release
+          path: reusable
+          ref: ${{ steps.workflow-parsing.outputs.SHA }}
+          token: ${{ secrets.UCLAHS_CDS_REPO_READ_TOKEN }}


question (non-blocking): I guess once this repo is public, this token should also no longer be necessary?

Correct, the secret won't be necessary for this purpose.

Separately, I might need to add an input for the token to be used when creating the release (in the release workflow). I ran into that constant issue where the alias workflow would not be triggered unless the release workflow was run with a non-standard token.

Specifically this block:

tool-create-release/.github/workflows/wf-finalize-release.yaml

Lines 36 to 45 in 668db63

- id: parse-version

uses: actions/github-script@v7

env:

INPUT_DRAFT: ${{ inputs.draft }}

with:

# Use the separate token so that `published` events will be fired

github-token: ${{ secrets.UCLAHS_CDS_REPO_READ_TOKEN }}

script: |

const script = require('./scripts/finalize-release.js')

await script({github, context, core})

nwiltsie added 30 commits September 25, 2024 13:08

Add template workflow files

22e0312

Enable dependabot, including template workflows

672edc5

Update CHANGELOG

6abb5ff

Update README

d91a2c2

Add untested aliasing workflow

e391a69

Refactor and add tests

479a579

Small input tweaks

a2e0c34

Add template workflow for aliasing

04da392

Accessing this repository requires a secret

a8ae99c

Add the missing script

200d3b1

Add GH_TOKEN envvar

4f15f17

Add debug output for git tags

4842cb2

Properly configure logger

43a9a30

Adding more logs

f0a732a

Fix regex

11e655c

Add github-actions[bot] as the git identity

f6007fb

Set the _correct_ git config

fa502f2

Small bugfix

2c469d9

Format code

3d1fe31

Refactor to share more code

4081f41

Rename Version.version to ChangeLogVersion.version_str

3c1035b

Refactor more things into utils

7a40ea9

Add tests for getversion

c986b0b

Format code

2750437

Major refactor around prereleases

dee2b0e

Update README

8f3209b

Update templates

9f59d61

Bugfix

87f198c

Format code

da4ff7d

Properly make GitHub prereleases

1e7e52e

nwiltsie added 8 commits September 26, 2024 14:53

Bugfix

91d13aa

Bugfix

02bb39f

Use the separate token so that published events will fire

8921f7a

Actually return Release objects

801d43e

Improve annotate tag message

1695e3f

Update CHANGELOG

3e031ab

Update templates

ed12389

Update trigger workflows for this repository

6c933bb

nwiltsie requested a review from a team September 27, 2024 17:12

Fix lints

dad4001

aholmes reviewed Sep 27, 2024

View reviewed changes

yashpatel6 self-assigned this Sep 27, 2024

aholmes approved these changes Sep 27, 2024

View reviewed changes

nwiltsie added 5 commits September 27, 2024 15:51

s/IneligibleAlias/IneligibleAliasError

d42c04c

Clarify intention with parentheses

fa2cceb

Improve readability with intermediate variables

64f3d3e

Add two more test cases to demonstrate pre-releases

79e915a

Grumble grumble pylint vs ruff

668db63

yashpatel6 approved these changes Sep 30, 2024

View reviewed changes

nwiltsie merged commit 6c046d8 into main Sep 30, 2024
7 checks passed

nwiltsie deleted the nwiltsie-alias-release branch September 30, 2024 17:10

nwiltsie mentioned this pull request Oct 2, 2024

Update hardcoded versions, properly fill release notes #17

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add workflow to update major alias tags, templates #13

Add workflow to update major alias tags, templates #13

nwiltsie commented Sep 27, 2024

aholmes Sep 27, 2024

nwiltsie Sep 27, 2024

aholmes Sep 27, 2024

aholmes Sep 27, 2024

nwiltsie Sep 27, 2024

aholmes Sep 27, 2024

nwiltsie Sep 27, 2024

aholmes Sep 27, 2024

nwiltsie Sep 27, 2024

yashpatel6 Sep 30, 2024

nwiltsie Sep 30, 2024

nwiltsie Sep 30, 2024

		from .utils import dereference_tags, tag_to_semver, get_github_releases, Release


		class IneligibleAlias(Exception):

	- id: parse-version
	uses: actions/github-script@v7
	env:
	INPUT_DRAFT: ${{ inputs.draft }}
	with:
	# Use the separate token so that `published` events will be fired
	github-token: ${{ secrets.UCLAHS_CDS_REPO_READ_TOKEN }}
	script: \|
	const script = require('./scripts/finalize-release.js')
	await script({github, context, core})

Add workflow to update major alias tags, templates #13

Add workflow to update major alias tags, templates #13

Conversation

nwiltsie commented Sep 27, 2024

Description

Closes #9

Checklist

Footnotes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment