Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip 21.0 release (Q1 2021) #9282

Closed
pradyunsg opened this issue Dec 15, 2020 · 47 comments
Closed

pip 21.0 release (Q1 2021) #9282

pradyunsg opened this issue Dec 15, 2020 · 47 comments
Assignees
Milestone

Comments

@pradyunsg
Copy link
Member

Oh yea, it's happened. We have two separate release issues open at the same time.

This one is a big one, coz we're dropping Python 2 support. :)

@pradyunsg pradyunsg added this to the 21.0 milestone Dec 15, 2020
@pradyunsg
Copy link
Member Author

pradyunsg commented Dec 15, 2020

I'm becoming a little wary that I'm becoming the SPOF for our release processes and, I'd like to avoid that. :)

@pypa/pip-committers I'd like for someone else to be the RM for this release. I'm happy to have a voice/video chat if that'd be helpful for whomsoever becomes the RM, but I think everything is pretty much documented already.

(for the uninitiated: RM is short for release manager, who would be the person in charge of making the call of what goes into the release closer to the release date and does the actual magic of generating the release artifacts for PyPI)

@pfmoore
Copy link
Member

pfmoore commented Dec 15, 2020

I could do it, but it's extremely likely that I'd have to do the release over a weekend and then be (mostly) unavailable until the following weekend (because work doesn't provide me with any open source time, and I expect things to be busy Q1 next year). Specifically, I would not be able to respond to the sort of post-release fallout we've had this time, and I worry that the removal of Python 2 support, and the probable deprecation of the search command, might generate similar levels of issues.

I'm very happy for someone else to take it if they want, though.

@sbidoul
Copy link
Member

sbidoul commented Dec 15, 2020

I'm a bit in the same situation as Paul. I could do it but, due to limited availability, I'd prefer to handle a "simpler" release than 21.0, if such as thing ever exists :)

@pradyunsg
Copy link
Member Author

@uranusjr would you be available to manage this release?

@uranusjr
Copy link
Member

Chinese New Year is 12th February, and weeks before that are usually crunch time. So I’ll likely not be able to react in timely fashion during the time. It may work for me if the release can happen on 11th February or later.

@pradyunsg
Copy link
Member Author

Ohkay... It seems like I'm in the best position (availability wise) to be wearing the hat again for 21.0. If no one is opposed to that, I'll assign myself to this issue before the end of the year.

That said, I do think that someone else will need to be the RM for 21.1, because I'm pretty certain that I won't be as available as I am right now, when that release is due.

@pfmoore
Copy link
Member

pfmoore commented Dec 25, 2020

We may just have to accept that we do a release with the resources we have and deal with the consequences - @pradyunsg like you say, having you as the SPOF isn't either sustainable nor fair on you.

If you want me to, I'll take 21.0. We just need to be aware that you've been unusually available post-release these last few releases, and we don't guarantee that going forward.

BTW, what's the expectation on when we do 21.0? Normally it would be January, do we want to aim for that given that 20.3 was pretty late? I'm inclined to say yes, we should get back on track. I'd also say that 21.0 should be back to "release from master on the planned date" and not try to wait for anything specific (like completion of the removal of Python 2 support).

@pradyunsg
Copy link
Member Author

Works for me. :)

I'll be fairly available in Jan FWIW, and I do agree that a release in early Jan works best.


I think we'd probably want to defer removing the legacy resolver in 21.0 though (to 21.1 or 21.2) given that there's still a bunch of bugs open for it, and we've just not had enough time for folks to adjust perhaps. That's a #6536 discussion but whoever wears the RM hat needs to decide on that so that we can do a round onlf communicating about it. :)

I think that communication is gonna be the main thing this release, especially coz we won't have any funded folks to actively look into communicating about it.

Personally, I'm not too concerned about Python 2. And, well, pip search can stay for a while. :P

@pradyunsg
Copy link
Member Author

FWIW, in my opinion, we'd dropped support for Python 2 when we updated the python_requires and removed it from the CI. That's all that really matters from a user's PoV.

All the modernisation and trimming (shoutout to @hugovk and @jdufresne on that front) is entirely a maintainer-only thing, and I don't expect there to be any user-facing implications of those not being "done", unless we make significant improvements (or make a mistake).

@pradyunsg
Copy link
Member Author

Oh, and heads up on a fun thing that needs to be handled/deferred: we should update pip's vendoring configuration, coz we're going to be able to vendor newer versions of setuptools, which would put additional files in the vendoring directory that the current configuration won't delete.

@uranusjr
Copy link
Member

This might sound crazy, but what if we skip a quarter. Remembering 20.3 was already delayed a month for various reasons, 20.3 and 21.0 would still only be ~120 days apart if we target the first half of April.

@pradyunsg
Copy link
Member Author

pradyunsg commented Dec 25, 2020

To be frank, I just want to have a release that drops Python 2 in Jan 2021, because we've explicitly documented and communicated that, and because I want to be able to close bug reports with "no Python 2, thanks". :)

I would be fine if we upload a 21.0 that is literally 20.3.x, but with updated python_requires and classifiers. Actually, I would be on board if the RM (which seems to be @pfmoore) decided to do it that way. :P

@pfmoore
Copy link
Member

pfmoore commented Jan 1, 2021

OK. I'm thinking that I will do the release on the weekend of 23/34 January. That's not set in stone yet, I may move it a week either way depending on my availability, but it'll do for now as a plan 🙂

I will be releasing whatever's on master at the time of release - we have a "Release 21.0" milestone, but I intend to treat this as a reminder list for the authors of the relevant PRs, not as an action list for the RM, so I'll ping them closer to the release to check if they are expecting to be ready, and if not, I'll simply bump the items to 21.1.

I'd like 21.1 to be a relatively "quiet" release - we'll be dropping Python 2 support which may catch some people out, but otherwise I'd like to focus on any remaining tidy-up from the new resolver release, and apart from that stick to "business as usual" changes. Our users have had a fair bit of change to deal with recently, and I think that a quiet start to 2021 would be welcomed by all!

One thought - presumably we'll need a final Python 2 version of get-pip.py. Has anyone done that as part of the PRs dropping Python 2 compatible code, or is that still to be done? I don't keep track of the get-pip repo, so I'm not aware of what's going on there. If not, does anyone want to pick this up or is it something I should add to the list of things for the release? I assume it would have to be based off 20.3.3, not master.

@pradyunsg
Copy link
Member Author

pradyunsg commented Jan 1, 2021

We'd need to update get-pip.py's generator script.

https://github.com/pypa/get-pip/blob/master/tasks/generate.py#L121

I'd like 21.1 to be a relatively "quiet" release

I think you meant 21.0 here? ;)

@pfmoore
Copy link
Member

pfmoore commented Jan 1, 2021

< I think you meant 21.0 here?

Correct. I wish we'd decided to number releases as 1,2,3,4 rather than 0,1,2,3, but hey, too late now...

@pfmoore
Copy link
Member

pfmoore commented Jan 9, 2021

I've just added reminders to all issues/PRs marked against the 21.0 milestone. If anyone has anything else they'd like to get into 21.0, you have a bit under 2 weeks to get it merged to master...

@hugovk
Copy link
Contributor

hugovk commented Jan 9, 2021

pip search deprecation? It looked like there was initial agreement to include a deprecation warning in 21.0 (#5216), but @nlhkabu asked to share some user research first.

According to https://status.python.org/incidents/grk0k7sz6zkp, the PyPI issue is still not resolved.

@pfmoore
Copy link
Member

pfmoore commented Jan 16, 2021

Vendoring - @pradyunsg am I right in thinking that re-vendoring is done by manually updating versions in the vendor.txt file, and then running nox -s vendor? There's no automatic process for updating the versions in vendor.txt?

The big question here is whether we update setuptools, as I think we pinned 44.0.0 because of Python 2 support. So we could now update it, but is there a risk in doing so? I'm inclined to do it in a separate PR and see what CI thinks. But if anyone who's more directly involved with removing Python 2 support wants to chip in, that would be fine too 🙂

The only other changes here are point releases in msgpack and requests (and its dependencies), which are recent and so I assume OK.

@pfmoore
Copy link
Member

pfmoore commented Jan 16, 2021

#9460 created for the non-controversial vendoring changes.

@uranusjr
Copy link
Member

I browsed through the changelog quickly, and it seems the pkg_resources package has not changed much, pypa/setuptools#1563 is the only change relevant to pip.

Alternatively we could keep setuptools < 45 (there is a 44.1 IIUC) and work on replacing it entirely instead.

@pradyunsg
Copy link
Member Author

I'm OK to keep the older setuptools/pkg_resources for now. If we're upgrading, the thing to take care of is that we'd need to account for changes in setuptools' structure/installed-artifacts in >= 50.0, since it's got a new _distutils_hack package and .pth file and stuff.

@pfmoore
Copy link
Member

pfmoore commented Jan 18, 2021

Should be a case of bumping up the version numbers now, I think.

Well, no, because the tests fail on 3.9 due to the cryptography pin. I'm not keen on messing with the test suite in the week before a release, TBH, so let's park this until after the release is done.

@pradyunsg
Copy link
Member Author

Do I also need to create a new template script?

Your call TBH. We only need to do that if we move the main function. We haven't, so there's no need to.

OTOH, creating a new template now would mean that if we do move main() in a future refactor, we don't need to do the split then.

@pfmoore
Copy link
Member

pfmoore commented Jan 18, 2021

Apparently there's also something needed in "salt states" to add a Python 2.7 directory to bootstrap.pypa.io. I've no clue about this 🙁 @ewdurbin from what I can see you did the last one of these (for Python 2.6) - would you be able to do it again?

@ewdurbin
Copy link
Member

I believe the necessary directory needs to be added to get-pip then the necessary symlinks can be created in the psf-salt repo to point to them.

@pfmoore
Copy link
Member

pfmoore commented Jan 18, 2021

OK, I'll do the get-pip change in the next day or two, but I don't know what's needed in the salt repo, so if I ping you will you be able to do that bit? (I can create a PR that blindly creates a 2.7 copy of the bit you highlighted, but I'm a bit scared of breaking something as I know nothing about salt...)

@ewdurbin
Copy link
Member

Either way is fine @pfmoore! If you'd prefer I make the change, just mention me here or on the PR to get-pip requesting it.

@pfmoore
Copy link
Member

pfmoore commented Jan 23, 2021

@ewdurbin Sorry! I never got the chance to make the necessary PR for the salt repo. Is this something you could do? I'm in the process of doing the new pip release now.

@ewdurbin
Copy link
Member

@pfmoore it's done and is live.

@pfmoore
Copy link
Member

pfmoore commented Jan 23, 2021

@ewdurbin Awesome! Thanks for doing that so quickly 🙂

@pfmoore
Copy link
Member

pfmoore commented Jan 23, 2021

Pip 21.0 has now been released.

One issue, around the classifiers (a previous commit had left them incorrect). We should probably have a lint check that the classifiers are valid, to avoid this sort of issue in the future.

@pfmoore
Copy link
Member

pfmoore commented Jan 24, 2021

Note for myself. I still need to create a PR for CPython to include 21.0 with ensurepip. I'll hold off on that until next weekend to see what happens with #9506 (we may do a 21.0.1).

@groodt
Copy link
Contributor

groodt commented Jan 25, 2021

Thank you @pfmoore and all the volunteers! I appreciate your work.

@pfmoore
Copy link
Member

pfmoore commented Jan 29, 2021

Looks like packaging may have a release ready for this weekend, which will address #9506. Assuming they do, I'll release 21.0.1 this weekend from master. Anything else that needs to go into 21.0.1 should be in master by then.

I don't intend to release a 21.0.2 (barring unexpected disasters) so anything that doesn't make 21.0.1 will need to wait for 21.1 (in April).

@pfmoore
Copy link
Member

pfmoore commented Jan 30, 2021

OK, release 21.0.1 has been published!

Thanks @henryiii for assistance with the vendoring, that made my life (and weekend!) a lot easier 🙂

@henryiii
Copy link
Contributor

Fantastic! Thank you so, so much for your support! We should have cibuildwheel ready to produce Universal2 wheels very soon now!

@brainwane
Copy link
Contributor

If we are not removing the old resolver in any 21.0.x release, we should update the deprecation timeline at https://pip.pypa.io/en/latest/user_guide/#deprecation-timeline .

@pfmoore
Copy link
Member

pfmoore commented Feb 5, 2021

The 21.0 release cycle is complete now, there's no expectation of a 21.0.2, so yes, removal of the old resolver will happen in a future release. I suspect we should avoid being specific about when, simply because it depends on some volunteer doing the work, so we can't say when it will actually happen.

Maybe we should update the deprecation to note that as of 21.0 the old resolver is no longer supported, and will be removed without further warning in some future release (when resources permit)? That probably reflects the reality better than trying to predict when it will happen.

As an immediate measure, someone could submit a PR to remove the --use-deprecated=legacy-resolver flag. That would be a smaller change than full removal (not trivial, because it would also need to remove any tests that still check the old resolver) and has the same effect as far as end users are concerned. If submitted (and merged) now, it would be released with 21.1.

@MrMino
Copy link
Contributor

MrMino commented Feb 21, 2021

@brainwane @pfmoore see #9631

@pradyunsg pradyunsg added the S: needs triage Issues/PRs that need to be triaged label Mar 6, 2021
@uranusjr uranusjr added S: needs triage Issues/PRs that need to be triaged and removed S: needs triage Issues/PRs that need to be triaged labels Mar 7, 2021
@pradyunsg
Copy link
Member Author

Funny how I never closed this. :)

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 26, 2021
@pradyunsg pradyunsg removed the S: needs triage Issues/PRs that need to be triaged label Mar 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants