Failure to find Python binary on MacOS GitHub runner

[Midnighter]

Hello,

I ran into the below error with running a command in a hatch environment. It happens to me on MacOS runners with Python 3.12 and 3.13 (I will test other versions, too). It happens both with pip or uv as the installer.

With pip installer

Run hatch run install:check
  hatch run install:check
  shell: /bin/bash -e {0}
  env:
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
    PKG_CONFIG_PATH: /Users/runner/hostedtoolcache/Python/3.12.7/arm64/lib/pkgconfig
    Python_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
    Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
    Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
Syncing environment plugin requirements
Creating environment: install
Checking dependencies
Syncing dependencies
cmd [1] | pip check
No broken requirements found.
cmd [2] | hatch build --clean
/Users/runner/Library/Application Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/hatch: line 2: /Users/runner/Library
Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/python3.12: No such file or directory
/Users/runner/Library/Application Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/hatch: line 2: exec: /Users/runner/Library
Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/python3.12: cannot execute: No such file or directory
Error: Process completed with exit code 126.

With uv installer

Run hatch run install:check
  hatch run install:check
  shell: /bin/bash -e {0}
  env:
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
    PKG_CONFIG_PATH: /Users/runner/hostedtoolcache/Python/3.12.7/arm64/lib/pkgconfig
    Python_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
    Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
    Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.7/arm64
Syncing environment plugin requirements
Creating environment: install
Checking dependencies
Syncing dependencies
cmd [1] | pip check
No broken requirements found.
cmd [2] | hatch build --clean
/Users/runner/Library/Application Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/hatch: line 2: /Users/runner/Library
Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/python3.12: No such file or directory
/Users/runner/Library/Application Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/hatch: line 2: exec: /Users/runner/Library
Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/python3.12: cannot execute: No such file or directory
Error: Process completed with exit code 126.

The environment itself is detached and defined as follows:

[tool.hatch.envs.install]
description = """Test the distribution of the package."""
dependencies = [
    "pip",
    "twine",
]
detached = true
installer = "uv"  # <-- same error when using pip!

[tool.hatch.envs.install.scripts]
check = [
    "pip check",
    "hatch build {args:--clean}",
    "twine check dist/*",
]

I'm using the same on several other projects and have not seen this issue before.

The workflow is defined as:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: ["3.12", "3.13"]

    steps:
    - uses: actions/checkout@v4

    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}

    - name: Install hatch
      uses: pypa/hatch@install

    - name: Check installation
      run: hatch run install:check

[Midnighter]

I've modified the workflow to only run on Mac and Python versions 3.9 - 3.13. It fails on all of them.

  cat "/Users/runner/Library/Application Support/pyapp/hatch/15969350296226343517/1.13.0/python/bin/hatch"

Output

#!/bin/sh
"exec" "$(dirname $0)/python3.12" "$0" "$@"
# -*- coding: utf-8 -*-
import re
import sys
from hatch.cli import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

[Midnighter]

I guess, the assumption that the Python binary is in the same directory as the hatch binary does not hold here.

[ofek]

Could you please show me the contents of that directory?

[Midnighter]

Now it gets interesting... 😖

 total 50824
drwxr-xr-x  26 runner  staff   832B Nov  8 19:15 .
drwxr-xr-x   6 runner  staff   192B Nov  8 19:15 ..
lrwxr-xr-x   1 runner  staff     9B Jan  1  2024 2to3 -> 2to3-3.12
-rwxr-xr-x   1 runner  staff   128B Oct 13 23:02 2to3-3.12
-rwxr-xr-x   1 runner  staff   243B Oct 13 23:02 hatch
-rwxr-xr-x   1 runner  staff   257B Oct 13 23:02 hatchling
-rwxr-xr-x   1 runner  staff   239B Oct 13 23:02 httpx
lrwxr-xr-x   1 runner  staff     8B Jan  1  2024 idle3 -> idle3.12
-rwxr-xr-x   1 runner  staff   126B Oct 13 23:02 idle3.12
-rwxr-xr-x   1 runner  staff   245B Oct 13 23:02 keyring
-rwxr-xr-x   1 runner  staff   255B Oct 13 23:02 markdown-it
-rwxr-xr-x   1 runner  staff   256B Oct 13 23:02 pip
-rwxr-xr-x   1 runner  staff   256B Oct 13 23:02 pip3
-rwxr-xr-x   1 runner  staff   256B Oct 13 23:02 pip3.12
lrwxr-xr-x   1 runner  staff     9B Jan  1  2024 pydoc3 -> pydoc3.12
-rwxr-xr-x   1 runner  staff   111B Oct 13 23:02 pydoc3.12
-rwxr-xr-x   1 runner  staff   250B Oct 13 23:02 pygmentize
lrwxr-xr-x   1 runner  staff    10B Jan  1  2024 python -> python3.12
lrwxr-xr-x   1 runner  staff    10B Jan  1  2024 python3 -> python3.12
lrwxr-xr-x   1 runner  staff    17B Jan  1  2024 python3-config -> python3.12-config
-rwxr-xr-x   1 runner  staff    49K Jan  1  2024 python3.12
-rwxr-xr-x   1 runner  staff   2.0K Oct 13 23:02 python3.12-config
-rwxr-xr-x   1 runner  staff   254B Oct 13 23:02 userpath
-rwxr-xr-x   1 runner  staff    24M Oct 13 23:02 uv
-rwxr-xr-x   1 runner  staff   328K Oct 13 23:02 uvx
-rwxr-xr-x   1 runner  staff   273B Oct 13 23:02 virtualenv

[Midnighter]

Could it be the space in the filepath, and you need to have the following line instead?

"exec" "$(dirname "$0")/python3.12" "$0" "$@"

[Midnighter]

At least, this short test makes a big difference:

#!/usr/bin/env sh

path='/some/fake/path with/a/space'

echo "$(dirname $path)"
echo "$(dirname "$path")"

output

/some/fake
with/a
/some/fake/path with/a

but I do not see really see this outcome in the GitHub error message.

[Midnighter]

Hi @ofek,

I have tried a few more things like completely remove the uv installer also from the default environment and remove the pip-compile plugin, but I still get the same error. I have now made the repo public and you can inspect all the failed runs on the debug-hatch-ci branch.

https://github.com/Midnighter/knowledge-chat/actions