Make the release workflow more resilient

## Summary

Currently, it is possible to create a tag and then have the release fail. Since we can't edit the tag (#4468), this reworks the release so that the tag is created inside the release workflow. This leaves as a failure mode that we have published to pypi but then creating the tag or GitHub release doesn't work, but in this case we can restart and the pypi upload is just skipped because we use the skip existing option.

The release workflow is started by a workflow dispatch with the tag instead of creating the tag yourself. You can start the release workflow without a tag to do a test run.

This also adds docs on how to release and a small style improvement for the maturin integration.

## Test Plan

Normal CI, thorough review and hoping that the next release will work

.github/workflows/ci.yaml

@@ -200,11 +200,10 @@ jobs:
       - name: "Build wheels"
         uses: PyO3/maturin-action@v1
         with:
-          manylinux: auto
           args: --out dist
       - name: "Test wheel"
         run: |
-          pip install dist/${{ env.PACKAGE_NAME }}-*.whl --force-reinstall
+          pip install --force-reinstall --find-links dist ${{ env.PACKAGE_NAME }}
           ruff --help
           python -m ruff --help
 

.github/workflows/release.yaml

@@ -2,8 +2,13 @@ name: "[ruff] Release"
 
 on:
   workflow_dispatch:
-  release:
-    types: [ published ]
+    tag:
+      description: "The version to tag, without the leading 'v'"
+      type: string
+  push:
+    paths:
+      # When we change pyproject.toml, we want to ensure that the maturin builds still work
+      - pyproject.toml
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
@@ -393,18 +398,27 @@ jobs:
       - linux-cross
       - musllinux
       - musllinux-cross
-    if: "startsWith(github.ref, 'refs/tags/')"
+    # If you don't set an input it's a practice run
+    if: "${{ inputs.tag }} != ''"
     environment:
       name: release
     permissions:
       # For pypi trusted publishing
       id-token: write
     steps:
+      - name: Consistency check
+        run: |
+          version=$(grep "version = " pyproject.toml | sed -e 's/version = "\(.*\)"/\1/g')
+          echo "${{ inputs.tag }}"
+          echo "${version}"
+          if [ "${{ inputs.tag }}" != "${version}" ]; then
+            exit 1
+          fi
       - uses: actions/download-artifact@v3
         with:
           name: wheels
           path: wheels
-      - name: "Publish to PyPi"
+      - name: Publish to PyPi
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           skip-existing: true
@@ -414,10 +428,17 @@ jobs:
         with:
           name: binaries
           path: binaries
+      - name: git tag
+        run: |
+          git tag -m "v${{ inputs.tag }}" "v${{ inputs.tag }}"
+          # If there is duplicate tag, this will fail. The publish to pypi action will have been a noop (due to skip
+          # existing), so we make a non-destructive exit here
+          git push --tags
       - name: Release
         uses: softprops/action-gh-release@v1
         with:
           files: binaries/*
+          tag_name: v${{ inputs.tag }}
 
   # After the release has been published, we update downstream repositories
   # This is separate because if this fails the release is still fine, we just need to do some manual workflow triggers

How_to_Release.md

@@ -0,0 +1,18 @@
+# Creating a new release
+
+This is a guide how to create a new ruff release
+
+1. Update the version with `rg 0.0.269 --files-with-matches | xargs sed -i 's/0.0.269/0.0.270/g'`
+2. Update [BREAKING_CHANGES.md](BREAKING_CHANGES.md)
+3. Create a PR with the version and BREAKING_CHANGES.md updated
+4. Merge the PR
+5. Run the release workflow with the tag (without starting `v`) as input. Make sure main has your merged PR as last commit
+6. The release workflow will do the following:
+   1. Build all the assets. If this fails (even though we tested in step 4), we haven’t tagged or uploaded anything, you can restart after pushing a fix
+   2. Upload to pypi
+   3. Create a git tag (from pyproject.toml) and push git tag. We create the git tag only here because we can't change it ([#4468](https://github.com/charliermarsh/ruff/issues/4468)), so we want to make sure everything up to and including publishing to pypi worked
+   4. Attach artifacts to draft GitHub release
+   5. Trigger downstream repositories. This can fail without causing fallout, it is possible (if inconvenient) to trigger the downstream jobs manually
+7. Create release notes in GitHub UI (https://github.com/charliermarsh/ruff/releases/new)
+8. If needed, [update the schemastore](https://github.com/charliermarsh/ruff/blob/main/scripts/update_schemastore.py)
+9. If needed, update ruff-lsp and ruff-vscode