Python 3.12?

[nedbat]

Description

Now that CPython 3.12.0b1 is out, I would like to produce wheels for it. I know the cibuildwheel docs say to wait for rc1, but the advice from Brett Cannon and Thomas Wouters is that it is fine to ship wheels now as long as I produce new wheels in the unlikely event that the ABI changes.

I tried specifying 3.12, but it failed

cibuildwheel: No build identifiers selected: BuildSelector(build_config='cp312-*', skip_config='', requires_python=<SpecifierSet('>=3.7')>, prerelease_pythons=True)

Build log

https://github.com/nedbat/coveragepy/actions/runs/5073139235/jobs/9111774802#step:6:72

CI config

https://github.com/nedbat/coveragepy/blob/nedbat/312-kits/.github/workflows/kit.yml

[henryiii]

(Pasted from the pypa discord, where I was replying to something related, but the contents roughly is of interest here.) Also I could have sworn that I responded that we have to wait till pypa/manylinux#1483, which got merged yesterday. I forget the build frequency, but it's either weekly or daily.

---

manylinux has roughly the same warning. One key difference is a wheel built with an ABI difference will likely segfault, compared to a Python wheel that will produce a readable error. And the first 3.x binary is very important for NumPy, which will forever use that wheel to build all 3.12 wheels in the future (okay, patch releases could possibly be moved to, but never a minor version. And even going up a patch version has to be extensively tested in oldest-supported-numpy. So NumPy is likely the last possible (though I'm with some of the NumPy team this week, so I could ask to see what they think).

I think we need a guarantee that binary compatibility will be preserved before we can recommend building and uploading support for Python-beta wheels. Or a tag to indicate that wheels were made with a beta (<4) Python. If a project tests cross-beta release compatibility (that is, downloads the wheels again and tests them against the new betas) and immediately deletes them if there's an issue, it's probably fine. But then if you are doing that you probably know what you are doing and don't need advice.

It also takes some time for projects to test, so for even the projects that want to upload beta wheels, I'd probably at the minimum wait till beta 2. I think there's a ton of movement right before beta 1, so I'd not really consider relying on beta 1's stability to be safe.

[nedbat]

Maybe I don't understand some of the factors here (likely!)... I'm not counting on the ABI being stable. I'm gambling that it is likely to be stable, and accepting that if it changes, I will need to make new 3.12 wheels. This is the advice from the Steering Council.

I don't understand why you are mentioning Numpy in your answer. If they have unusual requirements, can't they can decide their own path?

I get it that building wheels on beta releases can involve complications that wheels on stable releases don't have. I'm asking for the ability to make those wheels and navigate those complications myself.

[joerick]

The short answer is that it'll be available under the CIBW_PRERELEASE_PYTHONS flag, probably within the next few days.

[henryiii]

Apologies, that made less sense out of context than I expected it to. The discussion was about the steering council recommendation clashing with cibuildwheel and manylinux's warning, which state pretty much the opposite.

I'm gambling that it is likely to be stable, and accepting that if it changes, I will need to make new 3.12 wheels.

You can't upload a new Python 3.12 wheel, you only get to upload it once per release. You can delete the broken wheel and (ideally) make a new release with the updated 3.12 wheel, but many projects don't make releases based on fixing 3.12 wheels. If you know what to do, it's fine to do it, I was just defending the cibuildwheel/manylinux stance of making this available but with a warning and a custom flag, and this issue was a handy place to dump the text in a (more) public place. It absolutely will be available soon, and you absolutely can upload if you know what you are doing.

[henryiii]

Just checked, the latest manylinux has Python 3.12.0b1!

[nedbat]

Great, thanks for clearing that up.

Just checked, the latest manylinux has Python 3.12.0b1!

I guess we need a new release of cibuildwheel for it to work?

[hugovk]

You can't upload a new Python 3.12 wheel, you only get to upload it once per release.

It's possible to upload a new wheel for a release via build numbers.

For example, we did it not once but twice in python-pillow/Pillow#6750

https://pypi.org/project/Pillow/9.4.0/#files has:

• Pillow-9.4.0-cp311-cp311-macosx_10_10_x86_64.whl
• Pillow-9.4.0-1-cp311-cp311-macosx_10_10_x86_64.whl
• Pillow-9.4.0-2-cp311-cp311-macosx_10_10_x86_64.whl

[henryiii]

That allows you to only upload one new wheel rather than the full set, right? I think I assumed it was the full set. That would help a lot (as long as you tracked failures and actually rebuilt wheels if there was one - and wheel tags would help).

I would like to know how common ABI breakages during betas are.

[hugovk]

That allows you to only upload one new wheel rather than the full set, right?

Yes, you can do it for as many or as few as you need. In this Pillow case, we specifically re-uploaded the x86-64 macOS wheels, as only those were broken.

[henryiii]

2.13 is out! https://github.com/pypa/cibuildwheel/releases/tag/v2.13.0