Add universal2 builds of Python to actions/setup-python

[bwoodsend]

Tool name

universal2 builds of Python for macOS

Tool license

Python software foundation

Add or update?

• Add
• Update

Desired version

latest (3.9.6)

Approximate size

40MB in .pkg form. 150MB installed

Brief description of tool

As you're probably aware, Apple are transitioning from x86_64 to arm64. Whilst x86_64 machines are still around, Apple are encouraging software authors to ship universal2 applications which are executables for both x86_64 and arm64 glued together into one file and the OS is able to select the right half of the executable to run at runtime. The latest versions of Python on https://python.org/downloads now have a macOS 64-bit universal2 installer download option.

Given that the CI/CD servers are all x86_64, you're probably wondering why any of the above matters. Well... Python package authors are now also required to build their C extensions for macOS universal2 too. On the plus side, Python does this all automatically if the Python installation itself is compiled universal2. On the down side, there is no other way to build universal2 Python packages.

So long story short, without trying to do some wget https://www.python.org/ftp/python/3.9.6/python-3.9.6-macos11.pkg; sudo installer -pkg python-3.9.6-macos11.pkg -target / scariness to install Python directly from python.org, it is impossible to build Python packages from macOS arm64 on Github Actions.

One thing to note, building universal2 Pythons does not require access to arm64 machines - rather they can be cross compiled provided you have XCode >= 12.2 installed.

URL for tool's homepage

No response

Provide a basic test case to validate the tool's functionality.

test `lipo -archs $(which python)` = "x86_64 arm64"

Virtual environments affected

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11
• Windows Server 2016
• Windows Server 2019
• Windows Server 2022

Can this tool be installed during the build?

No response

Tool installation time in runtime

No response

Are you willing to submit a PR?

I can try...

[miketimofeev]

Hi @bwoodsend!
Just to clarify what we have on the image:

• Python3 installed from homebrew https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/[email protected]
• Pre-cached python versions built from source using this repository https://github.com/actions/python-versions to use with https://github.com/actions/setup-python

We can't do anything with the one from brew but we can probably try to re-build the pre-cached one with the arm64 support. Unfortunately, it may take some time so if you have a chance you can submit a PR to the python-version repo modifying the macos-python-builder script https://github.com/actions/python-versions/blob/main/builders/macos-python-builder.psm1

[bwoodsend]

OK, I'll do a PR for option 2. The python-versions source code looks a lot less intimidating than I expected so this shouldn't be too hard.

Do you have any thoughts on whether all Python versions that are capable of universal2 builds should become universal2 or should it be opt-in via a universal2: true flag to actions/setup-python? I can't think of any reason why a universal2 build would impair someone's workflow. They have a bigger filesize but that shouldn't slow builds down because these Python installers are cached I believe so the VMs don't even have to download them?

[miketimofeev]

What is the difference in size? If it's just a few megabytes then we should be good to go with the universal by default

[maxim-lobanov]

We would avoid adding additional parameters on actions/setup-python side so rebuilding existing binaries with universal2 support is more preferable. Of course, in case if it is not a breaking changes

[bwoodsend]

They're effectively two copies in one so the binaries about double the size of a single arch Python. But they compress down well when you archive them.

	single	universal2
unpacked	161MB	215MB
archive	29MB	36MB

[bwoodsend]

I've opened actions/python-versions#114 which assumes universal2 by default.

[jborean93]

Just as an FYI I use cibuildwheel successfully to build both an x86_64 and arm64 wheel using GitHub actions. I haven't tried the universal2 format as I prefer them being split but I'm surprised that having a universal2 Python is required to build those wheels. Especially since I don't need it to build the arm64 wheels.

The documentation for cibuildwheel also doesn't mention this as a requirement https://cibuildwheel.readthedocs.io/en/latest/faq/#universal2. Maybe cibuildwheel bundles it's own copy for this so it could still be required just not for this tool.

[bwoodsend]

cibuildwheel installs Python from python.org instead of using actions/setup-python (see here and here) so no, universal2 for actions/setup-python is not a requirement for them because they don't use actions/setup-python.

[ssbarnea]

Apparently there is zero support for arm64 in setup-python action, not even a single entry listing this architecture on https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json -- all of them are x64 and that is problematic.

[bwoodsend]

Hmm, I don't blame GitHub for lack of support for anything Apple's arm64 related. Apple refused to give VirtualBox the support they needed to support their new platform and because we can't have CI/CD without virtualisation or containerism to isolate each job, the whole platform essentially doesn't support CI/CD making it untestable and requiring this elaborate cross compile + fat binaries polava just to build binarues, with no way to test the arm64 slices, releasing them into wild completely untested. If you want to see a real example of this, look up why SciPy doesn't doesn't ship wheels for Big Sur M1.

Despite being the one to raise this issue in the first place and having submitted a PR to address it, I'm rather leaning towards recommending that GitHub rejects it mostly due to it encouraging developers to release untested binaries but also because it only partially solves the problem of building - plenty of C/C++ projects aren't cross compile friendly, and you've still got to encourage every one of your non pure Python dependencies to do likewise.

[ssbarnea]

Let me put it in a different angle: there is not arm64 platform even for linux inside the versions-manifest.json either.

I recently reproduced https://marcin.juszkiewicz.com.pl/2022/05/06/installed-top5000-python-packages-on-aarch64/ tests on M1 and discovered 4711/5000 did build and install and many of those failures were only caused by missing header files, so the situation on aarch64/aarm64 is not as bad as some claim.

I am inclined to believe that enabled universal would be less maintenance than keeping separated aarm64 versions. It does not need a lot of time to realise how brilliant are the universal binaries when you look at how the things are working on Windows side. Just one example from today experience where Windows support asked me use sysinternals procdump. The archive contained 3 executables, none of them for current architecture as I happened to be on Windows 11 ARM. I had to google to find another archive with some ports of systernals for ARM which contained two executables for procdump. Basically for one utility 5 executables. And in the end, none of them worked, but that is another story. I mentioned this only to raise awareness about costs of diverging. We clearly know what is the future on MacOS, so trying to avoid universal2 build, seems only a waste of time to me.

I personally doubt there are many python developers that would not want to produce universal2 wheels when running on macos. Lack of support from actions/setup-python is going to clearly make it much harder for them to achieve their goal.

[bwoodsend]

Let me put it in a different angle: there is not arm64 platform even for linux inside the versions-manifest.json either.

What would you do with it if there was? The only reason you should ever need to use aarch64 Linux is to build/test for that platform and to do that safely you need to run in minimal environment with carefully controlled old versions of glibc and core system libraries or downstream users will . Since the easiest way to do that is via Docker and a controlled docker container is unlikely to be able to run a Python environment compiled from the latest Ubuntu images on GitHub Action, you'd never actually use the Python environment provided by actions/setup-python - aarch64 or otherwise. Linux solved multi-architecture support with cross-architecture virtualisation years ago.

Basically for one utility 5 executables.

So the solution for Windows is a single >5 slice fat executable? And Linux, which supports literally 100s of architectures, should use fat binaries with 100s of slices despite having thrived quite successfully using virtualisation instead of cross compiling or fat binaries?

I personally doubt there are many python developers that would not want to produce universal2 wheels when running on macos.

You'll have to fight me on that one. I'd say that any developer who values not doubling their file sizes and the time it takes for gatekeeper to verify that engorged binary, the simplicity of knowing that build platform == runtime platform and the knowledge that the test suite has been ran on every supported platform will turn their nose up at fat binaries given a viable alternative. That post you link only shows that most packages build OK but certainly doesn't show that they actually work.

[misl6]

Considering actions/python-versions#114 changes got reverted, I guess this issue should be re-opened?

[albertosottile]

actions/python-versions#175 reverted the change for this issue. Is there a way to have a universal2 Python version in the GHA image? Otherwise I think this should be reopened. The reasons stated in the original issue are still valid today.

[ssbarnea]

@albertosottile I am also curious about this because that revert was bit weird, with an army of reviewers approving it but the explanation was extremely vague "breaking changes for some customers". No mention of those customers and the nature of the breakage.

On the bright side, it was mentioned as "temporary", even if there was no timeline regarding when it would be restored.

As universal2 is the standard recommended binary format on MacOS, I would really want to see more data. I am sure that some CI/CD pipelines could get broken if they encounter universal2 but it would be up to them to fix the issue ASAP. Holding arm64 builds hostage because on or two projects failed to update their build script to be able to cope with universal2, is not ok.

Clearly this ticket should have being reopened when the revert was made.

[albertosottile]

Should we create a new issue? I am afraid we will not get much visibility here...

[ssbarnea]

@vsafonkin Can you please reopen this issue because it was undone?

[al-cheb]

@albertosottile, Please create a new issue for visibility in actions/setup-python repo

[albertosottile]

@al-cheb I wanted to create a new issue, but then I found this actions/setup-python#271. Should I still create essentially a duplicate?

[bwoodsend]

Not a maintainer but I'd say go for it. Strictly speaking actions/setup-python#271 should have been closed at the same time this was closed.

[IvanZosimov]

Hi, @bwoodsend, @ssbarnea, @albertosottile 👋
Starting from Python 3.11.0-rc.2 the setup-python action installs official universal2 for macOS, so the feature requested from this issue is finally implemented.

[ssbarnea]

@IvanZosimov A big thank you! When I will visit Belgrade again, I will ping you to have a beer!

[shakfu]

What's a way to prevent universal2 wheels from being built on an x86_64 runner?

I've tried ARCHFLAGS='-arch x86_64' python3 setup.py bdist_wheel but that only gives me wheels tagged as universal2 but with only x86_64 contents... Has anyone else experienced this?

I've written about this problem much more extensively here, including different solutions which were attempted which I will post here:

1. Prefix ARCHFLAGS='-arch {ARCH} to python3 setup.py

   ARCHFLAGS='-arch x86_64' python3 setup.py bdist_wheel

   This does force the contents of the wheel to be {ARCH} but does not change its tag which remains universal2.

2. Set the tag using --plat-name {tag-name} to python3 setup.py bdist_wheel:

   python3 setup.py bdist_wheel --plat-name macosx_13_x86_64

   This doesn't work and gives an error while testing the wheel:

   ERROR: cyfaust-0.0.3-cp311-cp311-macosx_13_x86_64.whl is not a supported wheel on this platform.

What to do?