Simplified Workflow

[blink1073]

Problem

• It is cumbersome to make releases using a separate repository.
• It requires every maintainer to set up their fork and maintain a token map.
• It requires several manual steps.

Proposed Solution

We could have an optional usage that works as follows:

• Maintainer creates a draft release from the GitHub UI - this does not yet create a tag.
  • Use the GitHub-provided release notes and edit as desired.
• Maintainer Runs Publish Draft Release workflow in the Repo.
  • Optionally takes a draft workflow url, otherwise uses the latest.
• The Publish Draft Release workflow will:
  • Bump the version
  • Update the changelog entry to use github-activity formatting (based on a setting)
  • Build the assets
  • Create a tag and pushes it
  • Upload the assets to the GitHub release
  • Publish the assets to npm and/or PyPI
  • Finalize the release
  • Open a forwardport PR if necessary
  • Bump to dev version automatically after the release based on config. I had thought Improve handling of get_version #294 fixed this but it only handles the cases where "next", "patch", and "minor" are used. Here we'll have explicit tags
• Maintainer merges the forwardport changelog PR(s) if applicable.

By using a tag protection rule only users with Maintain and Admin roles would be able to publish.

We already have most of the building blocks to build the new action.

The action should check to see whether the assets are already included, which allows you to re-try if there is a publish failure. If there is a problem with the draft release itself you can delete it and start over.

[blink1073]

Example automated changelog from GitHub for use in testing:

## What's Changed
* Fix docs on config values by @blink1073 in https://github.com/jupyter-server/jupyter_releaser/pull/291
* Update check-links and settings by @blink1073 in https://github.com/jupyter-server/jupyter_releaser/pull/292
* Automated Changelog Entry for 0.17.0 on main by @blink1073 in https://github.com/jupyter-server/jupyter_releaser/pull/293


**Full Changelog**: https://github.com/jupyter-server/jupyter_releaser/compare/v0.16.0...v28

[blink1073]

Basically we can look for ## What's Changed, then extract the part up to "by @" and match the pull at the end.

If there's anything before What's Changed, we use that as front matter, to give space for some optional narrative docs.

[blink1073]

Another intermediate step is to fetch the latest draft GitHub release, which can be used as the default value for Publish Release as well.

https://docs.github.com/en/rest/reference/releases#list-releases
"created_at": "2013-02-27T19:35:32Z",
"draft": false,

gh = GhApi(owner=owner, repo=repo_name, token=auth)
for release in gh.repos.list_releases():
   ...

[blink1073]

New thought that will simplify both this workflow and the existing one, requiring a V2:

• Draft Changelog is replaced by a new Draft Release that:

  • Creates the new changelog entry
  • Creates a draft release with the entry
  • Adds additional metadata as an attachment to the draft release (instead of requiring it to be entered separately across two workflows)

• Publish Release will change to:

  • Accept either a repo name or draft URL
  • Get the changelog entry and metadata from the draft release
  • Update the changelog and tag, push, and publish assets/release

• Full Release will be available to folks who want to combine the previous two steps

For the version that sits on a repo the only difference is that you wouldn't give the repo name or use a PYPI_TOKEN_MAP, and the Publish Release would work with no inputs for the latest draft release.
We would show example workflows in our docs.

[blink1073]

More notes on for the refactor:

• Use a mock GitHub server in tests and in check_release so we can actually verify the handling of the GitHub release
• Dry run should only prevent pushing commits/tags - it should be using mock PyPI, mock GitHub, and verdaccio.
• No more checkout directory - use the cwd in all of the cli except for prep-git which creates a checkout
  • In the action, we clone and set up git unless it is already the current dir, then we make sure the branch is correct
• No more pip/npm check in extract release - rely on shas and the fact that we tested them before uploading
• No more verify changelog - check to see if the current HEAD of the branch matches the one in the metadata
• Add metadata file to release
• Remove Draft Changelog Action in favor of: Draft Release, Prepare Assets, Publish Release
• Workflows are: Step 1: Draft Release, Step 2: Publish Release
  • Publish Release should be able to handle being run either before or after Prepare Assets

[blink1073]

I think as a first step that is backwards compatible I can add the mock GitHub server from https://github.com/blink1073/mock_github and use it in check-release and unit tests. Then I can do the same with verdaccio.

The next step would be refactoring the actions/workflows and releasing a v2 tag for the actions.

The CLI improvements can follow on later and result in a 1.0 release of the python library.

[blink1073]

Now that we have mock GitHub capability, we can do the following:
We can modify the workflows/actions in place and then later rename them and
handle the changelog in the draft GitHub release instead of a PR.

Draft Changelog

• Take all of the same inputs as current draft release action
• Create draft GitHub Release
• Create a metadata file to upload
  • All of the input data
  • The current sha of the target branch
  • The previous version from the changelog
• Make the changelog PR as we currently do
  • always set "next" and "patch" based on the previous version from the changelog

Draft Release

• Input is an optional draft release url
• Basically do the current draft release action based on data in the metadata file
  • do a sha check instead of a changelog check - for now it should verify that only the changelog has changed
  • always set "next" and "patch" based on the previous value in metadata file from the changelog
  • add the sha of each dist file to the metadata file
  • update the metadata file to say that assets were prepared

Publish Release

• Input is an optional draft release url
• If the metadata file says that prepare assets hasn't been run, do that first
• Otherwise basically do the current publish release action
• Verify the sha of each file against metadata file instead of inspecting the commit message
• Do not check dists in extract_release

The new names can be "Step 1: Prep Changelog", "Prep Release", and "Step 2: Publish Release"