Updating Release Process documentation based on 18.0 release experience

[pradyunsg]

As I said in #5516, I've made notes about things that caught my eye. I figure that pip's release notes should be updated to mention some of these details.

Click to see notes

• Update the version before using invoke to generate files.
  • towncrier should see the newer version when run.
• NEWS entries should not be git-commit style message
  (summary/description) since towncrier simply adds the issue number
  at the end of the full string and not the summary line.
• There is no CHANGES.txt to update
• There's no discussion about branching/PRs at all in this process.
• We should remove the "required to merge" from travis-ci/push CI
  check since we only merge PRs and pushes don't need to be required for
  merging them.
• twine should print an error message instead of traceback when ^C is
  pressed on password prompt.

[pradyunsg]

Here's a non-wordy, not-polished way of mentioning the approach I took for the steps releasing pip.

If this looks good to @pypa/pip-committers, I'll be happy to update the Release Process documentation to mention all these details, with commands (after #5526).

• master is "locked"
• make a release/YY.MM.patch branch release branch and checkout to it
  (in the future, this should be a single invoke command that we can run)
  • bump version for development
  • generate things
  • push to pypa/pip
    • This results in 2 extra CI jobs; the reason that matters is that AppVeyor doesn't run them in parallel and they took 45x2 minutes.
• create a PR
  • we need to put a trivial label here until pypa-bot learns that
    removing a NEWS fragment is good enough let things pass
  • this will continue to trigger CI that's designed to skip non-code
    changes since we'd modify __init__.py for bumping the version
• merge the PR when CI approves
  • the CI takes almost 2 hours
  • this is a good time to draft an announcement for the mailing lists
• create a signed release tag off master
  (in the future, this is where we'd hand off to CI release automation)
  • push the tag
  • make the release (build/upload sdist, wheel)
    • I didn't sign any of these files for 18.0; should we?
  • generate get-pip.py wrappers and push
• announce on mailing lists
• make a release/bump-for-development (on your fork)
  • bump version for development of the next version
• create a PR
• merge the PR when CI approves
  • the CI takes almost 1 hour
• master is "unlocked" for development

[dstufft]

FWIW, I never bothered creating a PR, and I just manually merged into master locally. I don't think there's a lot of value in creating the PR, because the code changes should be changing the version number, which is pretty low risk.

That also means we don't have to "lock" the master branch. I think that's a good thing, because we don't have any mechanism to enforce a lock, so we're relying on communication and that introduces a larger chance for a human to introduce error.

Skipping x2 PRs, and keeping everything local until the very end makes the whole thing easier.

[pradyunsg]

keeping everything local until the very end makes the whole thing easier.

The alternative of doing everything offline works well since we get to ensure there's no possibility of human error. However, since we require status checks to be passing for master, I don't know if simply pushing to master would work. I somehow remember @pfmoore had mentioned that he couldn't do so during 10.0 -- so I didn't explore if that's possible.

[pradyunsg]

I don't know if simply pushing to master would work.

I'm going to try it now; with a trivial NEWS fragment. I doubt it'd work though.

[pfmoore]

I preferred the process of making a PR, simply because it felt "safer" to me. But having said that, I was pretty firmly sending out "please don't merge anything, I'm going into release mode" messages leading up to the release, and I didn't really mind the CI waits. I think it's OK to have this left as a choice for whoever does the release.

On the question of communications, I do think that we should do more in the way of pre-release comms. I'm biased because with pip 10 I did a lot of that beforehand, but while I'm fine with the "what's on master is what gets released" principle, I think we should give 3rd party contributors a bit of warning that their PRs need to be sorted or they are facing a 3 month wait for the next chance (I'm thinking in particular of #5571 here, but it's true in general). Again, I did a lot of herding of PRs for pip 10, and that level of work isn't sustainable, but we could probably manage a bit more "this has a week before the release, we need X, Y and Z if it's not to get left till the next release in 3 months". Specifically on things that are on the current milestone (so we've set an expectation).

But this is all pretty minor. I think this release went very well and more notes on how to keep it this smooth would be great.

[pradyunsg]

I'm going to try it now; with a trivial NEWS fragment. I doubt it'd work though.

To github.com:pypa/pip.git
 ! [remote rejected]   master -> master (protected branch hook declined)
error: failed to push some refs to '[email protected]:pypa/pip.git'

The error line is in red. :P

[pradyunsg]

Basically, pushing to master isn't something that can be done without disabling the status checks.

[pradyunsg]

we don't have any mechanism to enforce a lock

We can change access to master in https://github.com/pypa/pip/settings/branches/master. I'm personally fine if we need an RM modify that to "lock" for release though.

[pfmoore]

I'm personally inclined to just leave it up to the RM how they prefer to work it, I don't think this is something we need to overthink.

[pradyunsg]

We can skip a discussion on branching if that's what we want. I do prefer we mention an approach there that's not as time consuming. :)

---

Regardless, we'd still need to update the order (bump version before generating NEWS) and update file names (it refers to CHANGES.txt right now).

[pradyunsg]

(this isn't strictly about automation but I guess there's some points here that might be helpful during automation)

[pradyunsg]

I'm happy with what the section looks like now. I'm going to go ahead and close this issue.

[pradyunsg]

If there's anything else to be done here; we can take that up as it comes up. :)