Add Python builds for Apple Silicon

[lpugin]

It seems that this should be available for python 3.8 onwards.
https://cibuildwheel.readthedocs.io/en/stable/

Not sure what to change in the workflow. @musicEnfanthen ?

[lpugin]

I think we also need to adjust things here

verovio/.github/workflows/python-ci-wheel.yml

Lines 26 to 46 in a85a7b1

	os: [macos-latest, windows-latest, ubuntu-20.04]
	python-version: ["3.7", "3.8", "3.9", "3.10"]
	architecture: [x86, x64]
	include:
	- os: macos-latest
	architecture: x64
	platform_id: macosx_x86_64
	- os: windows-latest
	architecture: x64
	platform_id: win_amd64
	- os: windows-latest
	architecture: x86
	platform_id: win32
	- os: ubuntu-20.04
	architecture: x64
	platform_id: manylinux_x86_64
	exclude:
	- os: macos-latest
	architecture: x86
	- os: ubuntu-20.04
	architecture: x86

Any ideas what architecture: x64 means here

verovio/.github/workflows/python-ci-wheel.yml

Line 31 in a85a7b1

architecture: x64

and why x86 is excluded here

verovio/.github/workflows/python-ci-wheel.yml

Line 44 in a85a7b1

architecture: x86

[musicEnfanthen]

the line with architecture: x64 is needed for the setup/python action in the workflow. If I remember correctly, x86 architecture is only supported for Windows runners in the setup/python action, cf. https://github.com/actions/python-versions/blob/main/versions-manifest.json. Because of that, using macos-latest with x86 arch threw an error back then, and had to be excluded.

[musicEnfanthen]

For the arm64 configuration: We could try to skip cibuildwheel on python version below 3.8 which does not support Apple Silicon.

So here:

verovio/.github/workflows/python-ci-wheel.yml

Line 89 in a85a7b1

CIBW_SKIP: cp27-* # manylinux2014 not compatible with python 2.7

it should probably read

CIBW_SKIP: "cp27-* cp37-macosx_arm64" # manylinux2014 not compatible with python 2.7, macosX arm64 not compatible with python < 3.8

Did not test it, so not sure about the syntax with quotation marks, but took it from here: https://cibuildwheel.readthedocs.io/en/stable/options/#examples_1

If that does not work, because cibuildwheel internally would pass the arm64 config to the build step nonetheless, it could be an option to split up the cibuildwheel step in the gh actions workflow into two steps, one for python 3.7 without arm64, and one for python >= 3.8 with arm64 config.

[rettinghaus]

I tested @musicEnfanthen's suggestion here and it seems to work.

[lpugin]

Thanks for trying. I can see only four builds for macos, all seem to be for x86_64. E.g., https://github.com/rettinghaus/verovio/runs/5293392197?check_suite_focus=true#step:6:13

Am I overlooking something? Could you point me to the arm64 build?

[lpugin]

I added the arm64 in the matrix here.
lpugin@820fa56

This fails because python-setup does not seem to support arm64.
https://github.com/lpugin/verovio/runs/5299680915?check_suite_focus=true#step:4:13

See
actions/setup-python#108 (comment)

Can we cross-compile the arm64 wheel from a x64 machine? Or is there a way to install Python differently?

[rettinghaus]

With setup-python you only can install the x64 build, see https://github.com/actions/python-versions/releases
Also, as it stands, there are currently no plans to support M1 environments in GitHub actions.
And I don't think we can cross-compile to arm64 on x64, only the other way round with Rosetta 2.

[lpugin]

Cross-compiling looks possible https://cibuildwheel.readthedocs.io/en/latest/faq/#apple-silicon
I am looking at it.

[rettinghaus]

O yes, at least we have to CIBW_TEST_SKIP: *_arm64.

[lpugin]

Building works https://github.com/lpugin/verovio/runs/5300429546?check_suite_focus=true#step:8:11

I still need to fix the testing which pulls all the wheels. That is different from the CIBW_TEST_SKIP

[musicEnfanthen]

@lpugin Can you try the following change:

      - name: Install from wheel on Unix
        if: always() && runner.os != 'Windows'
        working-directory: wheelhouse
        run: python -m pip install ./*.whl  ## change to  -->  ./*_x86_64.whl

This should exclude the arm64 wheels from the pip install check, since they are not supported on the x64 runners (as you said, at the moment there aren't any arm64 compatible runners in GH actions). At the same time, x64 wheels should pass this check on ubuntu and macosx.

Please be aware that the arm64 wheel needs to be considered "untested" then.

[lpugin]

Thanks. This is what I did and it seems to work https://github.com/lpugin/verovio/runs/5301787587?check_suite_focus=true

[musicEnfanthen]

Perfect :)

[lpugin]

https://test.pypi.org/project/verovio/3.10.0.dev0/#files