Use GitHub App & REST PUT API for updating version.sbt

This changes the way we authenticate and make the 1 or 2 updates to `version.sbt` required for a release. Before: * Authenticate as: default `github-actions` bot * version.sbt update method: Cherry-pick the commits created by sbt-release, then push them to GitHub using `git push`, with the default `github-actions` bot using its credentials to make the push After: * Authenticate as: `gu-scala-library-release` GitHub App - https://github.com/apps/gu-scala-library-release * version.sbt update method: GitHub REST API for Repository Contents (PUT /repos/{owner}/{repo}/contents/{path}) https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28#create-or-update-file-contents Now we're not really specifying the whole commit, just the content change to one file. This has a few different benefits: * Addresses the need to coexist with our branch-protection rulesets, because GitHub Apps can be exempted from rules, as discussed in issue #5 * Produces `Verified` commits - the commits show up as `Verified` in the GitHub UI, and have a `gpgsig` header entry that is signed by GitHub itself, essentially GitHub attesting that the author of the commit authenticated with GitHub to perform the file update. https://git-scm.com/docs/signature-format#_commit_signatures https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification#signature-verification-for-bots https://blog.gitbutler.com/signing-commits-in-git-explained/#github-verification The commits now appear to be attributed to `gu-scala-library-release`, rather than, eg, '@rtyley using gha-scala-library-release-workflow' - it's a bit of shame that the person triggering the release is no longer so clearly visible, but it's probably less confusing. To compensate for that, the commit message itself has been updated to specifically state the responsible user.
guardian · Feb 23, 2024 · f75d81e · f75d81e
1 parent a7ab2ae
commit f75d81e
Show file tree

Hide file tree

Showing 6 changed files with 186 additions and 42 deletions.
diff --git a/.github/workflows/reusable-release.yml b/.github/workflows/reusable-release.yml
@@ -3,6 +3,13 @@ name: Scala Library Release Workflow
 on:
   workflow_call:
     inputs:
+      GITHUB_APP_ID:
+        description:
+          "App ID for a GitHub App that is allowed to push directly to the default branch. Eg, App ID on:
+          https://github.com/organizations/guardian/settings/apps/gu-scala-library-release"
+        default: '807361' # Only for use by the Guardian!
+        required: false # ...but if you're not the Guardian, you'll want to set this explicitly
+        type: string
       SONATYPE_PROFILE_NAME:
         description: 'Sonatype account profile name, eg "com.gu", "org.xerial", etc (not your Sonatype username)'
         default: 'com.gu' # Only for use by the Guardian!
@@ -25,9 +32,14 @@ on:
       PGP_PRIVATE_KEY:
         description:
           "A passphrase-less PGP private key used to sign artifacts, commits, & tags.
-          Should be in normal plaintext 'BEGIN PGP PUBLIC KEY BLOCK' (ASCII-armored) format, with no additional BASE64-encoding.
+          Should be in normal plaintext (ASCII-armored) format, starting 'BEGIN PGP PUBLIC KEY BLOCK', with no additional BASE64-encoding.
           The passphrase can be removed from an existing key using 'gpg --edit-key <key-id> passwd' : https://unix.stackexchange.com/a/550538/46453"
         required: true
+      GITHUB_APP_PRIVATE_KEY:
+        description:
+          "See https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/managing-private-keys-for-github-apps#generating-private-keys
+          Should be in normal plaintext format, starting '-----BEGIN RSA PRIVATE KEY-----'"
+        required: true
     outputs:
       RELEASE_VERSION:
         description: "The un-prefixed version number of the release, eg '3.0.1'"
@@ -40,6 +52,8 @@ env:
   LOCAL_ARTIFACTS_STAGING_PATH: /tmp/artifact_staging
   COMMITTER_NAME: "@${{github.actor}} using gha-scala-library-release-workflow"
   RUN_ATTEMPT_UID: ${{ github.run_id }}-${{ github.run_attempt }}
+  TEMPORARY_BRANCH: release-workflow/temporary/${{ github.run_id }}
+  GITHUB_REPO_URL: ${{ github.server_url }}/${{ github.repository }}
 
 jobs:
   init:
@@ -142,7 +156,15 @@ jobs:
       release_tag: ${{ steps.create-commit.outputs.release_tag }}
       release_version: ${{ steps.create-commit.outputs.release_version }}
       release_commit_id: ${{ steps.create-commit.outputs.release_commit_id }}
+      version_file_path: ${{ steps.create-commit.outputs.version_file_path }}
+      version_file_release_sha: ${{ steps.create-commit.outputs.version_file_release_sha }}
+      version_file_post_release_content: ${{ steps.create-commit.outputs.version_file_post_release_content }}
     steps:
+      - id: generate-github-app-token
+        uses: actions/create-github-app-token@v1
+        with:
+          app-id: ${{ inputs.GITHUB_APP_ID }}
+          private-key: ${{ secrets.GITHUB_APP_PRIVATE_KEY }} }
       - uses: actions/checkout@v4
         with:
           path: repo
@@ -161,6 +183,8 @@ jobs:
         env:
           KEY_FINGERPRINT: ${{ needs.init.outputs.key_fingerprint }}
           KEY_EMAIL: ${{ needs.init.outputs.key_email }}
+          GH_TOKEN: ${{ steps.generate-github-app-token.outputs.token }}
+          GH_REPO: ${{ github.repository }}
         run: |
           echo "GITHUB_REF_NAME=$GITHUB_REF_NAME"
           echo "GITHUB_REF=$GITHUB_REF"
@@ -169,39 +193,49 @@ jobs:
           RELEASE_TAG=$(git describe --tags --abbrev=0)
           
           cd ../repo
-          git status
-          git config user.email "$KEY_EMAIL"
-          git config user.name "$COMMITTER_NAME"
-          git config commit.gpgsign true
-          git config user.signingkey "$KEY_FINGERPRINT"
           
           git remote add unsigned ../repo-with-unsigned-version-update-commits.git
           git fetch unsigned
-          git cherry-pick -S$KEY_FINGERPRINT $GITHUB_REF_NAME..unsigned/$GITHUB_REF_NAME
-          git status
           
-          release_commit_id=$(git rev-parse HEAD^)
+          RELEASE_VERSION=${RELEASE_TAG#"v"}
+          VERSION_FILE_PATH=$(git diff-tree --no-commit-id --name-only -r $RELEASE_TAG | grep version.sbt)
+          VERSION_FILE_INITIAL_SHA=$( git rev-parse $GITHUB_REF:$VERSION_FILE_PATH )
+          VERSION_FILE_RELEASE_SHA=$( git rev-parse $RELEASE_TAG:$VERSION_FILE_PATH )
+          VERSION_FILE_RELEASE_CONTENT=$( git cat-file blob $RELEASE_TAG:$VERSION_FILE_PATH | base64 )
+          VERSION_FILE_POST_RELEASE_CONTENT=$( git cat-file blob unsigned/$GITHUB_REF_NAME:$VERSION_FILE_PATH | base64 )
+
+          cd ..
+          
+          cat << EndOfFile > commit-message.txt
+          $RELEASE_TAG published by ${{github.actor}}
           
+          ${{github.actor}} published release version $RELEASE_VERSION
+          using gha-scala-library-release-workflow: https://github.com/guardian/gha-scala-library-release-workflow
+
+          Release-Version: $RELEASE_VERSION
+          Release-Initiated-By: ${{ github.server_url }}/${{github.actor}}
+          Release-Workflow-Run: $GITHUB_REPO_URL/actions/runs/${{ github.run_id }}
+          GitHub-Release-Notes: $GITHUB_REPO_URL/releases/tag/$RELEASE_TAG
+          EndOfFile
+
+          # Create temporary branch to push the release commit- required for PREVIEW releases
+          gh api --method POST /repos/:owner/:repo/git/refs -f ref="refs/heads/$TEMPORARY_BRANCH" -f sha="$GITHUB_SHA"
+
+          release_commit_id=$(gh api --method PUT /repos/:owner/:repo/contents/$VERSION_FILE_PATH \
+            --field branch="$TEMPORARY_BRANCH" \
+            --field message="@commit-message.txt" \
+            --field sha="$VERSION_FILE_INITIAL_SHA" \
+            --field content="$VERSION_FILE_RELEASE_CONTENT" --jq '.commit.sha')
+                    
           cat << EndOfFile >> $GITHUB_OUTPUT
           release_tag=$RELEASE_TAG
-          release_version=${RELEASE_TAG#"v"}
+          release_version=$RELEASE_VERSION
           release_commit_id=$release_commit_id
+          version_file_path=$VERSION_FILE_PATH
+          version_file_release_sha=$VERSION_FILE_RELEASE_SHA
+          version_file_post_release_content=$VERSION_FILE_POST_RELEASE_CONTENT
           EndOfFile
           
-          git log --format="%h %p %ce %s" --decorate=short -n3
-          git status
-          
-          if [ "${{ needs.init.outputs.release_type }}" == "FULL_MAIN_BRANCH" ]
-          then
-            echo "Full Main-Branch release, pushing 2 commits to the default branch"
-            git push # push 2 commits (non-snapshot release version, then new snapshot version) onto the default branch
-          else
-            tag_for_pushing="preliminary-${{ github.run_id }}"
-            echo "Preview Feature-Branch release, pushing 1 commit with the temporary tag $tag_for_pushing"
-            git tag -a -m "Tag created merely to allow _pushing_ the release commit, which gains the signed $RELEASE_TAG tag later on in the workflow" $tag_for_pushing $release_commit_id
-            git push origin $tag_for_pushing # push only the single release version commit with a disposable tag
-          fi
-          
 
   create-artifacts:
     name: 🎊 Create artifacts
@@ -251,10 +285,18 @@ jobs:
     env:
       KEY_FINGERPRINT: ${{ needs.init.outputs.key_fingerprint }}
     steps:
+      - id: generate-github-app-token
+        uses: actions/create-github-app-token@v1
+        with:
+          app-id: ${{ inputs.GITHUB_APP_ID }}
+          private-key: ${{ secrets.GITHUB_APP_PRIVATE_KEY }} }
       - uses: actions/checkout@v4
         with:
           path: repo
           ref: ${{ needs.push-release-commit.outputs.release_commit_id }}
+          fetch-depth: 2 # To fast-forward the main branch, we need the commit on main, as well as the release commit
+          token: ${{ steps.generate-github-app-token.outputs.token }}
+          persist-credentials: true # Allow us to push as the GitHub App, and bypass branch ruleset
       - uses: actions/cache/restore@v4
         with:
           path: ${{ env.LOCAL_ARTIFACTS_STAGING_PATH }}
@@ -276,14 +318,19 @@ jobs:
           ARTIFACT_SHA256SUMS: ${{ needs.create-artifacts.outputs.ARTIFACT_SHA256SUMS }}
           KEY_EMAIL: ${{ needs.init.outputs.key_email }}
         run: |
-          echo "RELEASE_TAG=$RELEASE_TAG"
-          echo "RELEASE_COMMIT_ID=$RELEASE_COMMIT_ID"
           cd repo
           git config user.email "$KEY_EMAIL"
           git config user.name "$COMMITTER_NAME"
           git config tag.gpgSign true
           git config user.signingkey "$KEY_FINGERPRINT"
           
+          if [ "${{ needs.init.outputs.release_type }}" == "FULL_MAIN_BRANCH" ]
+          then
+            echo "Full Main-Branch release, fast-forwarding the default branch to the release commit"
+            git log --oneline -n 3
+            git push origin $RELEASE_COMMIT_ID:refs/heads/$GITHUB_REF_NAME
+          fi
+          
           cat << EndOfFile > tag-message.txt
           Release $RELEASE_TAG initiated by $COMMITTER_NAME
           
@@ -296,8 +343,6 @@ jobs:
           echo "Creating release tag (including artifact hashes)"
           git tag -a -F tag-message.txt $RELEASE_TAG $RELEASE_COMMIT_ID
           
-          echo "RELEASE_TAG=$RELEASE_TAG"
-          
           echo "Pushing tag $RELEASE_TAG"
           git push origin $RELEASE_TAG
       - uses: actions/cache/save@v4
@@ -350,10 +395,18 @@ jobs:
     env:
       RELEASE_TAG: ${{ needs.push-release-commit.outputs.release_tag }}
       RELEASE_VERSION: ${{ needs.push-release-commit.outputs.release_version }}
-      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       GH_REPO: ${{ github.repository }}
-      GITHUB_REPO_URL: ${{ github.server_url }}/${{ github.repository }}
     steps:
+      - id: generate-github-app-token
+        uses: actions/create-github-app-token@v1
+        with:
+          app-id: ${{ inputs.GITHUB_APP_ID }}
+          private-key: ${{ secrets.GITHUB_APP_PRIVATE_KEY }} }
+      - name: Clean-up temporary branch that was retaining the now-tagged release commit
+        env:
+          GH_TOKEN: ${{ steps.generate-github-app-token.outputs.token }}
+        run: |
+          gh api --method DELETE /repos/:owner/:repo/git/refs/heads/$TEMPORARY_BRANCH
       - name: Common values
         run: |
           GITHUB_ACTIONS_PATH="$GITHUB_REPO_URL/actions"
@@ -365,13 +418,28 @@ jobs:
           GITHUB_WORKFLOW_LINK=[GitHub UI]($GITHUB_WORKFLOW_URL)
           GITHUB_WORKFLOW_RUN_LINK=[#${{ github.run_number }}]($GITHUB_ACTIONS_PATH/runs/${{ github.run_id }})
           EndOfFile
-      - name: Create Github Release
+      - name: Create Github Release and update version.sbt post-release
         if: needs.init.outputs.release_type == 'FULL_MAIN_BRANCH'
+        env:
+          GH_TOKEN: ${{ steps.generate-github-app-token.outputs.token }}
         run: |
           gh release create $RELEASE_TAG --verify-tag --generate-notes --notes "Release run: $GITHUB_WORKFLOW_RUN_LINK"
           echo "GitHub Release notes: [$RELEASE_TAG]($GITHUB_REPO_URL/releases/tag/$RELEASE_TAG)" >> $GITHUB_STEP_SUMMARY
+          
+          cat << EndOfFile > commit-message.txt
+          Post-release of $RELEASE_TAG by @${{github.actor}}: set snapshot version
+          
+          Setting snapshot version after @${{github.actor}} published $GITHUB_REPO_URL/releases/tag/$RELEASE_TAG
+          EndOfFile
+          
+          gh api --method PUT /repos/:owner/:repo/contents/${{ needs.push-release-commit.outputs.version_file_path }} \
+            --field message="@commit-message.txt" \
+            --field sha="${{ needs.push-release-commit.outputs.version_file_release_sha }}" \
+            --field content="${{ needs.push-release-commit.outputs.version_file_post_release_content }}"
       - name: Update PR with comment
         if: needs.init.outputs.release_type == 'PREVIEW_FEATURE_BRANCH'
+        env:
+          GH_TOKEN: ${{ steps.generate-github-app-token.outputs.token }}
         run: |
           cat << EndOfFile > comment_body.txt
           @${{github.actor}} has published a preview version of this PR with release workflow run $GITHUB_WORKFLOW_RUN_LINK, based on commit ${{ github.sha }}:

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -1,16 +1,20 @@
 # Configuration
 
-Start here if you're setting up a repo use `gha-scala-library-release-workflow`!
+Start here if you're setting up a repo use `gha-scala-library-release-workflow`! If your organisation has never used
+`gha-scala-library-release-workflow` before, you'll need to follow the instructions in
+[Organisation Setup](org-setup.md) first.
 
-The workflow needs a `release.yml` GitHub workflow in your repo, and updated `sbt` settings.
+The release workflow needs a `release.yml` GitHub workflow in your repo, and specific updated `sbt` settings.
 
 [Example GitHub pull requests](#examples) making these changes can be found further below.
 
 ## Repo settings
 
-* Disable [Branch Protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches)
-  on any branch the workflow will be pushing to (ie the default branch). See issue
-  https://github.com/guardian/gha-scala-library-release-workflow/issues/5.
+* Disable [branch protection **rules**](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches)
+  on any branch the workflow will be pushing to (ie the default branch). Note that
+  PR https://github.com/guardian/gha-scala-library-release-workflow/pull/26 means that you _can_ use
+  branch protection **rulesets** to protect your default branch, so long as you allow your GitHub App
+  to bypass those restrictions.
 * **Guardian developers:** comply with the repository requirements of
   [`guardian/github-secret-access`](https://github.com/guardian/github-secret-access?tab=readme-ov-file#how-does-it-work),
   i.e. ensure the repository has a `production` topic label.

diff --git a/docs/credentials/generating-credentials.md b/docs/credentials/generating-credentials.md
@@ -3,6 +3,10 @@
 Normally you'll be using [shared organisation-wide credentials](supplying-credentials.md),
 but if you need to rotate those credentials, or just create some new ones for your organisation:
 
+## Updating a Sonatype OSSRH user's password
+
+See [Sonatype's instructions](https://central.sonatype.org/faq/ossrh-password/).
+
 ## Generating a new PGP key
 
 See [Sonatype's instructions](https://central.sonatype.org/publish/requirements/gpg/#generating-a-key-pair) for
@@ -16,6 +20,10 @@ should be plaintext, not BASE64-encoded.
 gpg --armor --export-secret-key [insert key fingerprint here] | pbcopy
 ```
 
-## Updating a Sonatype OSSRH user's password
+## Generating a new GitHub App private key
+
+See [GitHub's instructions](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/managing-private-keys-for-github-apps#generating-private-keys) for generating a private key. If you haven't already created a GitHub App for the
+release workflow, see [Setting up the GitHub App](github-app.md) first.
 
-See [Sonatype's instructions](https://central.sonatype.org/faq/ossrh-password/).
+**Guardian developers:** Here's a direct link to our GitHub App settings page, where you can generate a new private key:
+https://github.com/organizations/guardian/settings/apps/gu-scala-library-release
diff --git a/docs/credentials/supplying-credentials.md b/docs/credentials/supplying-credentials.md
@@ -4,7 +4,10 @@ Any repo that wants to use `gha-scala-library-release-workflow` needs to supply
 to the workflow:
 
 * [Sonatype OSSRH](https://central.sonatype.org/publish/publish-guide/) username & password
-* [PGP signing key](https://central.sonatype.org/publish/requirements/gpg/)
+* [PGP signing key](https://central.sonatype.org/publish/requirements/gpg/) - used for signing artifacts, and
+  the Git release tag.
+* [GitHub App private key](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/managing-private-keys-for-github-apps) - used
+  for jobs in the release workflow to authenticate & perform actions as the GitHub App with the GitHub API.
 
 For any given organisation, a single set of credentials can be shared with GitHub
 [Organization-level secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-an-organization)
@@ -14,9 +17,12 @@ has _access_ to those secrets.
 ### Guardian-specific access
 
 **Guardian developers:** We use [`guardian/github-secret-access`](https://github.com/guardian/github-secret-access)
-to grant repos access to the `AUTOMATED_MAVEN_RELEASE_PGP_SECRET` & `AUTOMATED_MAVEN_RELEASE_SONATYPE_PASSWORD`
-secrets - you need to raise a PR there (like [this example PR](https://github.com/guardian/github-secret-access/pull/24))
-to grant your repo access to the organisation-wide secrets.
+to grant repos access to the necessary Organisation secrets - you need to raise a PR there (like [this example PR](https://github.com/guardian/github-secret-access/pull/24))
+which will grant access to these:
+
+* `AUTOMATED_MAVEN_RELEASE_SONATYPE_PASSWORD`
+* `AUTOMATED_MAVEN_RELEASE_PGP_SECRET`
+* `AUTOMATED_MAVEN_RELEASE_GITHUB_APP_PRIVATE_KEY`
 
 ### Generating new credentials
 

diff --git a/docs/github-app.md b/docs/github-app.md
@@ -0,0 +1,38 @@
+# Setting up the GitHub App
+
+The GitHub App is used by the release workflow to perform actions on your repos, like creating releases and
+making PR comments.
+
+Each organisation that uses the release workflow will need to create their _own_ GitHub App.
+If `gha-scala-library-release-workflow` had its own server infrastructure, we could probably follow the more
+common model of a single GitHub App being used by many organisations, but instead we take advantage of all those
+free GitHub Actions minutes, so we need to pass the workflow the private key of the GitHub App so that it can
+authenticate as the GitHub App... therefore we must each have our own GitHub App, so that we don't share private keys.
+
+## 1. Create the GitHub App
+
+### GitHub App for a single user account
+
+You can just click this link to get taken to a pre-filled page to create a new GitHub App - you'll just need to
+customise the app name:
+
+https://github.com/settings/apps/new?name=scala-library-release&url=https://github.com/guardian/gha-scala-library-release-workflow&public=false&contents=write&pull-requests=write&webhook_active=false
+
+### GitHub App for an organisation account
+
+You can use the link above, but change the url so that it starts like this (the url query parameters stay the same),
+and replace `ORGANIZATION` with your organisation's name (eg `guardian`):
+
+https://github.com/organizations/ORGANIZATION/settings/apps/new
+
+## 2. Install the GitHub App
+
+Once your GitHub App is created, it'll be _owned_ by your organisation, but it'll still need to be _installed_
+on your organisation. You can do this from the `Install App` tag on the GitHub App's settings page. For example,
+for the `guardian` organisation, and the `gu-scala-library-release` app, the URL would be:
+
+https://github.com/organizations/guardian/settings/apps/gu-scala-library-release/installations
+
+At this point, you need to decide whether to install the app for all repositories, or just for selected
+repositories. Selected repositories is better, as it limits the possible damage a rogue workflow could inflict -
+but you'll need make sure you add all relevant repositories to the list as they come along.
diff --git a/docs/org-setup.md b/docs/org-setup.md
@@ -0,0 +1,20 @@
+# Organisation Setup
+
+These instructions are the initial setup for an organisation that's going to use `gha-scala-library-release-workflow`.
+When we say "organisation", we mean a GitHub organisation (like [github.com/guardian](https://github.com/guardian)), but
+these instructions more or less also apply to a single GitHub user account, if you want to set up your own personal
+repos to use this workflow.
+
+## 1. Set up the GitHub App
+
+See [Setting up the GitHub App](github-app.md) for details on how to create the necessary GitHub App.
+
+## 2. Create credentials
+
+See [Generating new credentials](credentials/generating-credentials.md) for details on how to create the necessary
+credentials, and [Supplying credentials](credentials/supplying-credentials.md) for how to store and supply them to
+your repos.
+
+## 3. Configure repos to use the release workflow
+
+See [Configuration](configuration.md) for details on how to configure your repos to use the release workflow.