Make the release workflow more resilient #4728

konstin · 2023-05-30T15:58:05Z

Summary

Currently, it is possible to create a tag and then have the release fail, which is a problem since we can't edit the tag (#4468). This change the release process so that the tag is created inside the release workflow. This leaves as a failure mode that we have published to pypi but then creating the tag or GitHub release doesn't work, but in this case we can restart and the pypi upload is just skipped because we use the skip existing option.

The release workflow is started by a workflow dispatch with the tag instead of creating the tag yourself. You can start the release workflow without a tag to do a dry run which does not publish an artifacts. You can optionally add a git sha to the workflow run and it will verify that the release runs on the mentioned commit.

This also adds docs on how to release and a small style improvement for the maturin integration.

Test Plan

Testing is hard since we can't do real releases, i've tested a minimized workflow in a separate dummy repository.

github-actions · 2023-05-30T16:56:52Z

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      6.8±0.02ms     6.0 MB/sec    1.00      6.8±0.01ms     6.0 MB/sec
formatter/numpy/ctypeslib.py               1.00   1369.2±2.29µs    12.2 MB/sec    1.00   1367.8±3.93µs    12.2 MB/sec
formatter/numpy/globals.py                 1.00    132.8±0.36µs    22.2 MB/sec    1.00    132.4±3.71µs    22.3 MB/sec
formatter/pydantic/types.py                1.01      2.8±0.01ms     9.2 MB/sec    1.00      2.8±0.02ms     9.3 MB/sec
linter/all-rules/large/dataset.py          1.00     13.6±0.03ms     3.0 MB/sec    1.01     13.8±0.05ms     3.0 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.01      3.4±0.00ms     4.8 MB/sec    1.00      3.4±0.02ms     4.9 MB/sec
linter/all-rules/numpy/globals.py          1.00    363.0±1.05µs     8.1 MB/sec    1.00    361.8±1.07µs     8.2 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.0±0.03ms     4.2 MB/sec    1.00      6.1±0.02ms     4.2 MB/sec
linter/default-rules/large/dataset.py      1.00      7.1±0.01ms     5.8 MB/sec    1.01      7.1±0.02ms     5.7 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1477.9±4.80µs    11.3 MB/sec    1.01   1487.2±2.56µs    11.2 MB/sec
linter/default-rules/numpy/globals.py      1.00    158.3±0.46µs    18.6 MB/sec    1.00    158.4±0.21µs    18.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.2±0.01ms     8.0 MB/sec    1.00      3.2±0.01ms     7.9 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      8.6±0.36ms     4.8 MB/sec    1.01      8.6±0.46ms     4.7 MB/sec
formatter/numpy/ctypeslib.py               1.00  1732.4±88.51µs     9.6 MB/sec    1.02  1763.8±100.93µs     9.4 MB/sec
formatter/numpy/globals.py                 1.02   188.3±22.47µs    15.7 MB/sec    1.00   183.9±14.36µs    16.0 MB/sec
formatter/pydantic/types.py                1.00      3.6±0.18ms     7.2 MB/sec    1.00      3.6±0.28ms     7.2 MB/sec
linter/all-rules/large/dataset.py          1.00     16.4±0.57ms     2.5 MB/sec    1.01     16.6±0.56ms     2.5 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      4.4±0.19ms     3.8 MB/sec    1.01      4.5±0.22ms     3.7 MB/sec
linter/all-rules/numpy/globals.py          1.01   556.5±33.41µs     5.3 MB/sec    1.00   550.2±38.77µs     5.4 MB/sec
linter/all-rules/pydantic/types.py         1.00      7.5±0.42ms     3.4 MB/sec    1.02      7.6±0.38ms     3.4 MB/sec
linter/default-rules/large/dataset.py      1.01      8.9±0.47ms     4.6 MB/sec    1.00      8.8±0.33ms     4.6 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.01  1881.4±88.33µs     8.9 MB/sec    1.00  1857.9±78.38µs     9.0 MB/sec
linter/default-rules/numpy/globals.py      1.00   231.8±13.48µs    12.7 MB/sec    1.00   230.9±13.43µs    12.8 MB/sec
linter/default-rules/pydantic/types.py     1.02      4.1±0.30ms     6.3 MB/sec    1.00      4.0±0.21ms     6.4 MB/sec

.github/workflows/release.yaml

How_to_Release.md

konstin · 2023-05-31T10:21:21Z

started a dry run: https://github.com/charliermarsh/ruff/actions/runs/5131675732

konstin · 2023-05-31T11:18:51Z

through the environment, we also get reports of failures from workflow triggers

charliermarsh · 2023-06-02T14:58:13Z

.github/workflows/release.yaml

+        type: string
+      sha:
+        description: "Optionally, the full sha of the commit to be released"
+        type: string


I don't think this is used (at least, as far as I can tell).

Aren't we able to select a branch anyway in the UI?

This can be useful in cases where you want to re-run the release workflow on a specific commit. But you're right. We could create a branch pointing to that specific commit instead.

i've changed this to be an additional check you can opt in or ignore

charliermarsh · 2023-06-02T14:58:51Z

.github/workflows/release.yaml

-    types: [ published ]
+    inputs:
+      tag:
+        description: "The version to tag, without the leading 'v'. If not present, a dry run will be made"


Nit: "If omitted, will initiate a dry-run, skipping any upload steps."

charliermarsh · 2023-06-02T14:59:09Z

.github/workflows/release.yaml

+  push:
+    paths:
+      # When we change pyproject.toml, we want to ensure that the maturin builds still work
+      - pyproject.toml


I'd be fine to omit this and just do it manually, personally.

yes but i want tests for maturin updates :D

charliermarsh · 2023-06-02T14:59:32Z

.github/workflows/release.yaml

@@ -393,18 +402,29 @@ jobs:
      - linux-cross
      - musllinux
      - musllinux-cross
-    if: "startsWith(github.ref, 'refs/tags/')"
+    # If you don't set an input it's a practice run


Nit: use the "dry run" terminology here for consistency, like "If no tag was set, consider this a dry-run."

charliermarsh · 2023-06-02T15:00:37Z

How_to_Release.md

RELEASE.md for title consistency? Or should we just put this in CONTRIBUTING.md, which is effectively our development guide? (It also includes benchmarking instructions, as an example.)

charliermarsh · 2023-06-02T15:00:53Z

.github/workflows/ci.yaml

          args: --out dist
      - name: "Test wheel"
        run: |
-          pip install dist/${{ env.PACKAGE_NAME }}-*.whl --force-reinstall
+          pip install --force-reinstall --find-links dist ${{ env.PACKAGE_NAME }}


What's the difference here?

one works by bash globbing, the other is the pip-provided way if installing from a directory which abstract wheel naming and such away and also works if there are multiple wheels (pip selects the right one)

charliermarsh · 2023-06-02T15:01:26Z

How_to_Release.md

+   1. Attach artifacts to draft GitHub release
+   1. Trigger downstream repositories. This can fail without causing fallout, it is possible (if
+      inconvenient) to trigger the downstream jobs manually
+1. Create release notes in GitHub UI (<https://github.com/charliermarsh/ruff/releases/new>)


Does this also include promoting out of "draft"?

How_to_Release.md

konstin · 2023-06-13T14:01:30Z

This is ready for another round of review

charliermarsh · 2023-06-20T17:17:39Z

RELEASING.md

+      inconvenient) to trigger the downstream jobs manually
+1. Create release notes in GitHub UI and promote from draft to proper release(<https://github.com/charliermarsh/ruff/releases/new>)
+1. If needed, [update the schemastore](https://github.com/charliermarsh/ruff/blob/main/scripts/update_schemastore.py)
+1. If needed, update ruff-lsp and ruff-vscode


Can we put this in CONTRIBUTING.md? There's already a small section there on releases.

Currently, it is possible to create a tag and then have the release fail, which is a problem since we can't edit the tag (#4468). This change the release process so that the tag is created inside the release workflow. This leaves as a failure mode that we have published to pypi but then creating the tag or GitHub release doesn't work, but in this case we can restart and the pypi upload is just skipped because we use the skip existing option. The release workflow is started by a workflow dispatch with the tag instead of creating the tag yourself. You can start the release workflow without a tag to do a dry run which does not publish an artifacts. You can optionally add a git sha to the workflow run and it will verify that the release runs on the mentioned commit. This also adds docs on how to release and a small style improvement for the maturin integration. Testing is hard since we can't do real releases, i've tested a minimized workflow in a separate dummy repository.

This reverts commit b4bd5a5.

konstin requested review from MichaReiser and charliermarsh May 30, 2023 15:58

MichaReiser approved these changes May 30, 2023

View reviewed changes

.github/workflows/release.yaml Show resolved Hide resolved

How_to_Release.md Outdated Show resolved Hide resolved

konstin had a problem deploying to release May 31, 2023 10:53 — with GitHub Actions Failure

konstin had a problem deploying to release May 31, 2023 11:39 — with GitHub Actions Failure

charliermarsh reviewed Jun 2, 2023

View reviewed changes

MichaReiser approved these changes Jun 5, 2023

View reviewed changes

How_to_Release.md Outdated Show resolved Hide resolved

How_to_Release.md Outdated Show resolved Hide resolved

konstin marked this pull request as draft June 13, 2023 12:14

konstin force-pushed the release_rework branch 2 times, most recently from 0aabc75 to 64a87fd Compare June 13, 2023 14:00

konstin marked this pull request as ready for review June 13, 2023 14:01

charliermarsh reviewed Jun 20, 2023

View reviewed changes

charliermarsh approved these changes Jun 20, 2023

View reviewed changes

konstin added 2 commits June 20, 2023 19:21

Move RELEASING.md to CONTRIBUTING.md

6c26e1c

konstin force-pushed the release_rework branch from 64a87fd to 6c26e1c Compare June 20, 2023 17:21

konstin enabled auto-merge (squash) June 20, 2023 17:21

Fix mkdocs strict mode

9277f73

konstin merged commit b4bd5a5 into main Jun 20, 2023

konstin deleted the release_rework branch June 20, 2023 18:33

charliermarsh added a commit that referenced this pull request Jun 20, 2023

Revert "Make the release workflow more resilient (#4728)"

cf8c459

This reverts commit b4bd5a5.

charliermarsh added a commit that referenced this pull request Jun 20, 2023

Revert "Make the release workflow more resilient (#4728)"

1fa0a6f

This reverts commit b4bd5a5.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the release workflow more resilient #4728

Make the release workflow more resilient #4728

konstin commented May 30, 2023 •

edited

Loading

github-actions bot commented May 30, 2023 •

edited

Loading

konstin commented May 31, 2023

konstin commented May 31, 2023

charliermarsh Jun 2, 2023

charliermarsh Jun 2, 2023

MichaReiser Jun 5, 2023

konstin Jun 13, 2023

charliermarsh Jun 2, 2023

charliermarsh Jun 2, 2023

konstin Jun 13, 2023

charliermarsh Jun 2, 2023

charliermarsh Jun 2, 2023

charliermarsh Jun 2, 2023

konstin Jun 13, 2023

charliermarsh Jun 2, 2023

konstin commented Jun 13, 2023

charliermarsh Jun 20, 2023

Make the release workflow more resilient #4728

Make the release workflow more resilient #4728

Conversation

konstin commented May 30, 2023 • edited Loading

Summary

Test Plan

github-actions bot commented May 30, 2023 • edited Loading

PR Check Results

Ecosystem

Benchmark

Linux

Windows

konstin commented May 31, 2023

konstin commented May 31, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

konstin commented Jun 13, 2023

Choose a reason for hiding this comment

konstin commented May 30, 2023 •

edited

Loading

github-actions bot commented May 30, 2023 •

edited

Loading