Mechanics of patch release v0.5.4

[matthewfeickert]

There has been ongoing discussions today (started by @alexander-held — thanks!) on releasing a patch release v0.5.4 to avoid the problems of namespace overlap between uproot v3.X and uproot4 v4.X

$ python -m pip install "pyhf[xmlio]<0.5.4"
$ python -m pip list | grep "pyhf\|uproot"
pyhf           0.5.3
uproot         3.13.1
uproot-methods 0.8.0
$ python -m pip install uproot4
$ python -m pip list | grep "pyhf\|uproot"
pyhf           0.5.3
uproot         4.0.0
uproot-methods 0.8.0
uproot4        4.0.0

This could be solved by preparing a v0.5.4 patch release that has the requirements

pyhf/setup.py

Line 11 in 0370dbd

'xmlio': ['uproot3~=3.14'], # Future proof against uproot4 API changes

currently on master. As @jpivarski has noted on PR #1209

In particular, I think it's a better idea to release a patch version of pyhf that only changes the Uproot/Awkward dependencies without stealthily adding 0.6.0 features. Even though there isn't a branch for 0.5.x, it's not hard to make one: (1) git checkout the latest 0.5.x tag (0.5.3), (2) create a branch for this detached HEAD state, (3) spot-change the dependencies, and (4) push to GitHub, where you can make a tag to deploy from this 0.5.x branch.

I have done this on the branch release-v0.5.4, which uses the uproot3 namespace and has had bump2version ran on it and passes CI

$ git log --oneline
8f3e7903 (HEAD -> release-v0.5.4, tag: v0.5.4, origin/release-v0.5.4) Bump version: 0.5.3 → 0.5.4
956ed6e7 fix: Update JAX DeviceArray type in docstrings for doctest
8a062628 fix: Use uproot3 namespace for patch release v0.5.4
1e3f729f (tag: v0.5.3) Bump version: 0.5.2 → 0.5.3

If @kratsg and @lukasheinrich are in favor of this plan my question now turns to the mechanics of how to make the release.

If I were to push the v0.5.4 tag to GitHub I'd need to verify exactly what the behavior of the CI would. Do we need to manually make this release to TestPyPI and PyPI? Additionally, would we ever merge the Bump version: 0.5.3 → 0.5.4 commit into master?

[matthewfeickert]

If I were to push the v0.5.4 tag to GitHub I'd need to verify exactly what the behavior of the CI would. Do we need to manually make this release to TestPyPI and PyPI? Additionally, would we ever merge the Bump version: 0.5.3 → 0.5.4 commit into master?

I have checked the logic of the workflow (and run it on my throw away test repo) for

pyhf/.github/workflows/publish-package.yml

Lines 1 to 11 in fd0bb76

	name: publish distributions
	on:
	push:
	branches:
	- master
	tags:
	- v*
	pull_request:
	branches:
	- master
	workflow_dispatch:

and confirm that if I do

$ git push --tags

from the release-v0.5.4 branch now that a publish distributions workflow will get kicked off that publishes to PyPI. This would mean that we would need to manually publish to TestPyPI first to check. As tags can exist on a repo even if the branch that contained them is deleted I propose the following workflow:

1. I manually release v0.5.4 to TestPyPI.
   • python -m pip install --upgrade build twine
   • python -m build --sdist --wheel --outdir dist/ .
   • twine check dist/*
   • twine upload --repository testpypi dist/*
   • rm -r build dist
2. We verify with @alexander-held that this release works.
3. I run push --tags on branch release-v0.5.4 to kickoff publication to PyPI.
4. We make the v0.5.4 tag a GitHub release to trigger Conda-forge and Zenodo.
5. I write a notification email to the pyhf-announcements mailing list.
6. To make sure that the release progression numbers make sense moving forward we make a chore PR that cherry-picks the effects of the bump2version that got v0.5.4 and merge that into master.
7. We resume work on release v0.6.0 as before.
8. After the release of v0.6.0 we can safely delete the release-v0.5.4 branch and leave it as just a tag that can always still be checked out.

Thoughts?

[alexander-held]

Thanks for the summary. I am happy to check the step 2 you propose on my end, though I do not expect any issues as long as pyhf sticks with the uproot3 namespace.

[matthewfeickert]

I am happy to check the step 2 you propose on my end, though I do not expect any issues as long as pyhf sticks with the uproot3 namespace.

Agreed — I would be very confused and concerned if that somehow did break things! :P But I figured being overly cautious here never hurts.

[kratsg]

I think this proposal works.

[matthewfeickert]

I have drafted release notes and an email to the pyhf-announcements mailing list so as soon as we have unanimous approval by the dev team we can pull the trigger. 👍

[matthewfeickert]

The branch release-v0.5.4 has been deleted, but the tag remains and so can still be checked out. 👍

[lukasheinrich]

LGTM

…

On Sat, Dec 12, 2020 at 12:11 AM Matthew Feickert ***@***.***> wrote: I have drafted release notes and an email to the pyhf-announcements mailing list so as soon as we have unanimous approval by the dev team we can pull the trigger. 👍 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1211 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARV6A7RTYH3GYQWDEDYJRDSUKRLZANCNFSM4UUDNWDA> .

[matthewfeickert]

v0.5.4 has been manually released to TestPyPI.

$ python -m pip install --extra-index-url https://test.pypi.org/simple/ --pre pyhf[xmlio]
$ python -m pip list | grep "pyhf\|uproot"
pyhf            0.5.4
uproot3         3.14.1
uproot3-methods 0.10.0
$ python -m pip install uproot4
$ python -m pip list | grep "pyhf\|uproot"
pyhf            0.5.4
uproot          4.0.0
uproot3         3.14.1
uproot3-methods 0.10.0
uproot4         4.0.0

@alexander-held Can you please verify that this works as expected for cabinetry (though from the above everything looks good)?

[alexander-held]

For posteriority: yes, works as expected. Thank you very much!

[matthewfeickert]

Awesome! :)

[matthewfeickert]

• v0.5.4 is released on TestPyPI
• v0.5.4 is released on PyPI and Conda-forge
• v0.5.4 is tagged on GitHub and on Zenodo
• Announcement email has been sent out on mailing list

I think this release is out and okay, so this can now stand as an example of how we can manually roll out patch releases if needed.