Possibility for a PyPi ARM Build

[BradyAJohnston]

I believe this has been touched on previously (#2727) to add CI for ARM platforms, but I'm wondering if it's possible to add an ARM build to PyPi for MDanalysis? I don't know what would be involved in setting up such a thing, so please do excuse my ignorance if is not currently done for a particular reason already (or if MDAnalysis isn't responsible for the PyPi builds, I don't really know how that whole thing works).

I've been running into an issue with installation for users who use M1 / M2 macs on a limited version of Python that Blender ships with (it's missing Python.h and other files required for compiling packages). Users on other platforms are able to install fine, and I believe I may have figured it out to be the missing arm build from PyPi that could be the difference (the workaround has been installing and linking a conda environment,

Any and all information about what might be the best strategy for me to tackle this outside of MDAnalysis as well if that would be best would also be very muchly appreciated!

[IAlibay]

I'll start by saying that this has been on my to-do list for CD improvements for a while, but it had remained on my low priority list because a) we provide os-arm64 builds on conda-forge, b) cirrus (see below) is a relatively recent CI provider and we didn't want to expend significant efforts towards a platform that was unproven. That being said, we had never anticipated this impacting a lot of users as we assumed folks would be able to build from sdists, but this assumption seems to have aged poorly..

Essentially the current gist of things is that:

1. We use cibuildwheels to make sure we consistently create the wheels we send out to PyPi on release (deploying from one's own machine is probably asking for trouble).
2. This is done as part of an automated CD/CI workflow using github actions which gets activated every time we push a release / tags: https://github.com/MDAnalysis/mdanalysis/blob/develop/.github/workflows/deploy.yaml
3. Unfortunately actions are limited to x86_64, so we can't deploy osx-arm64 wheels through them
4. The way around, and I believe the way scipy does it, is to use cirrus for deployments.

The scipy folks have, as always, made their CI quite human readable, so I have a reasonably good idea of how to get this done done for MDA. If @tylerjereddy is able to vouch for how reliable cirrus has been for scipy releases, I'm happy to try to get this done for the 2.5.0 release.

[tylerjereddy]

Yeah, Cirrus is working well for SciPy (and NumPy I think) with cibuildwheel (and in general for Linux ARM CI and MacOS M1/2 CI). You'll need to require a fairly recent version of the Mac operating system I think, but you could probably just copy much of what SciPy does, cutting out the stuff you/we don't need (Fortran compilers and stuff...).

[BradyAJohnston]

Thanks for the detailed explanation! Looking forward to the 2.5.0 release!

[IAlibay]

@BradyAJohnston - would you be able to check if the newly uploaded wheels for the 2.5.0 release fix this issue?

[BradyAJohnston]

@IAlibay I don't personally have access to an M1 / M2 mac to test on (sadly) but I've commented to ask if others who were having trouble find that this fixes the issue.

So far I've had 1 reply that theirs still doesn't work, but it also wasn't working for them with a previous solution involving pre-compiling .whl. I might try and track down a colleague in person and steal their laptop for a bit to do some testing.

[BradyAJohnston]

Just had another look at the user's error messages and it is a lack of biopython .whl for ARM MacOS that is likely the problem... I didn't think about the dependencies that would potentially also be problematic.

🫠

If you have any suggestions I'm all ears, but I would guess that the MDAnalysis component should be fixed so I very much appreciate the effort!

[IAlibay]

@richardjgowers @ianmkenney @hmacdope you all have M? laptops as far as I remember, can one of you help out here? We'd need to at least check the wheels are working.