Removing old Python support

[henryiii]

I'd like to remove old Python support. I'd really like to go to official EoL, which would be Python 3.8+. This will simplify maintenance and help with adding new features like VectorCall support. We are currently 3.6+.

[MartinThoma]

Maybe a pro/con list would help, similar to
py-pdf/pypdf#2418?

But so far the community vote seems to be rather clear 😊

[henryiii]

For 3.8:

Pros:

• Users on 3.7 don't have to worry about us breaking them anymore
• Easier to add VectorCall support
• Can use uv (3.8+) to speed up our CI
• Can use Apple Silicon runners more easily (CPython 3.8 was the first version to support Apple Silicon)
• Easier maintenance for us - can use modern versions with fewer workarounds (I see 5 cases of 0x03080000 in the code, 4 for 0x03070000, and a 0x03060000 we missed - and we have PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX, which is also 3.8, with another 7 cases.)
• Get to remove PyPy 3.7, which is no longer released (PyPy 3.8 isn't either, actually)
• Probably can update the setup.py based build system
• We'll have to figure out how to set up the import paths for tests on Windows on Python 3.8+

Cons:

• No new features for 3.7 users
• Users embedding a specific git checkout can't support 3.7 and new features - though that's not really recommended anymore with better build systems like scikit-build-core.
• We'll have to figure out how to set up the import paths for tests on Windows on Python 3.8+

[davidhewitt]

If it's useful, in PyO3 we are still stuck on 3.7+ because there's an outsize number of downloads coming from old versions of Amazon Linux.

Other than that, we usually track official EoL fairly closely.

[henryiii]

Are people using old version of Amazon Linux wanting new features, though? I usually find people stuck on old versions like to avoid upgrades since it breaks packages that have already dropped support.

For example, let's NumPy releases 2.0 with some breaking changes for Python 3.9. If scikit-learn, for example, has already dropped Python 3.9, now old versions of scikit-learn no longer install, and they have to go back and make a new release from an old branch either limiting NumPy<2, or fixing the incompatibility. I don't remember the exact package, but at least 1-2 packages like this are asking NumPy to remove Python 3.9 support before making the final NumPy 2 release!

I realize I used the common term "support". We are not really dropping support 3.7 or 3.6, we are just stopping new features for old Pythons. 3.6 will continue to work - just with an old version.

(Just a dump of some thoughts, though, based on the observation - thanks for that, though!)

[davidhewitt]

I think you're right about a disinterest in new features. As cryptography is built atop PyO3 they are extra cautious to only drop support when the proportion of downloads is small, as they fear security consequences of aging systems without cover.

The tension there is that cryptography also want to be able to use the latest PyO3 version because things like support for latest PyPy have typically needed additional feature work due to the nature of the FFI from Rust.

[henryiii]

PyO3 does support PyPy? So I need to figure out how to support PyPy while supporting CPython with ABI3 wheels, then, in a different project...

[davidhewitt]

Yes, PyPy support is quite painful because of the separate versioning scheme for ABI stability. Generally we just support PyPy 7.3's latest patch release and build wheels with pp73-cpXY platform tag (iirc)

[henryiii]

We've started the 3.6 removal here before our next release: #5177. 3.7 removal will probably occur immediately following the next release.

[henryiii]

3.6 is gone in our current release (2.13), and #5191 is removing 3.7 support.

[henryiii]

We are also increasing our compiler requirements (based on what we can test, mostly), and I'll be updating the CMake minimum to 3.15 for the next (non-patch) release as well.