Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.12 fails to build on M1 - 'Modules/_ssl.cpython-312-darwin.so': No such file or directory #990

Closed
kyhyco opened this issue Nov 10, 2023 · 9 comments · Fixed by jdx/mise-docs#11
Labels
bug Something isn't working

Comments

@kyhyco
Copy link

kyhyco commented Nov 10, 2023

Describe the bug
There are similar issues for pyenv - pyenv/pyenv#2805 and pyenv/pyenv#2823

Can't install python 3.12. Full error log

Error snippet

 ...
/opt/homebrew/bin/ginstall -c -m 755 Modules/_sha3.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_sha3.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_blake2.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_blake2.cpython-312-darwin.so
ginstall: cannot stat 'Modules/_blake2.cpython-312-darwin.so': No such file or directory
make: *** [sharedinstall] Error 1
BUILD FAILED (OS X 14.0 using python-build 20180424)
Inspect or clean up the working tree at /var/folders/kh/b7d8n69d25j56c1nw5tb0p800000gn/T/python-build.20231110142528.37566
Results logged to /var/folders/kh/b7d8n69d25j56c1nw5tb0p800000gn/T/python-build.20231110142528.37566.log
Last 10 log lines:
/opt/homebrew/bin/ginstall -c -m 755 Modules/_dbm.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_dbm.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_gdbm.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_gdbm.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/readline.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/readline.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_md5.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_md5.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_sha1.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_sha1.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_sha2.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_sha2.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_sha3.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_sha3.cpython-312-darwin.so
/opt/homebrew/bin/ginstall -c -m 755 Modules/_blake2.cpython-312-darwin.so /Users/spacefuture/.local/share/rtx/installs/python/3.12.0/lib/python3.12/lib-dynload/_blake2.cpython-312-darwin.so
ginstall: cannot stat 'Modules/_blake2.cpython-312-darwin.so': No such file or directory
make: *** [sharedinstall] Error 1
Error:
   0: /Users/spacefuture/Library/Caches/rtx/python/pyenv/plugins/python-build/bin/python-build exited with non-zero status: exit code 1

Location:
   src/cmd.rs:265

Version:
   2023.11.2 macos-arm64 (2023-11-09)

OS: macOS Sonama 14.0
Arch: arm64 (M1)
Shell: zsh

To Reproduce

  1. Install rtx - brew install rtx
  2. Add eval "$(rtx activate zsh)" to .zshrc
  3. Restart terminal and run rtx install [email protected]

Expected behavior
Python 3.12 should install without issues.

rtx doctor output

rtx version:
  2023.11.2 macos-arm64 (2023-11-09)

build:
  Target: aarch64-apple-darwin
  Features: BREW, DEFAULT, NATIVE_TLS
  Built: Thu, 9 Nov 2023 19:04:17 +0000
  Rust Version: rustc 1.72.1 (d5c2e9c34 2023-09-13) (Homebrew)
  Profile: release

shell:
  /bin/zsh
  zsh 5.9 (x86_64-apple-darwin23.0)

rtx data directory:
  /Users/spacefuture/.local/share/rtx

rtx environment variables:
  RTX_SHELL=zsh

settings:
  {"always_keep_download": "false", "always_keep_install": "false", "asdf_compat": "false", "disable_default_shorthands": "false", "disable_tools": "[]", "experimental": "false", "jobs": "4", "legacy_version_file": "true", "legacy_version_file_disable_tools": "[]", "log_level": "INFO", "missing_runtime_behavior": "warn", "plugin_autoupdate_last_check_duration": "10080", "raw": "false", "trusted_config_paths": "[]", "verbose": "false", "yes": "false"}

config files:

plugins:
  go       (core)
  java     (core)
  node     (core)
  python   (core)
  ruby     (core)
  rust     https://github.com/code-lever/asdf-rust.git#95acf4f

toolset:


No problems found

Dependency Info
Xcode info:

❯ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 15.0.0.0.1.1694021235
volume: /
location: /
install-time: 1697401497
@kyhyco kyhyco added the bug Something isn't working label Nov 10, 2023
@kyhyco
Copy link
Author

kyhyco commented Nov 10, 2023

As reference rtx install [email protected] works

@kyhyco
Copy link
Author

kyhyco commented Nov 10, 2023

Updated to macOS Sonoma 14.1.1 and Xcode 15.0.1 and get the same error.

@kyhyco
Copy link
Author

kyhyco commented Nov 20, 2023

Only one solution worked out of all the recommendations from similar threads:

brew unlink pkg-config
rtx install [email protected]
brew link pkg-config

Check with this command to see which packages has dependencies to pkg-config

brew uses --installed pkg-config 

@kyhyco
Copy link
Author

kyhyco commented Nov 20, 2023

Here is the explanation - pyenv/pyenv#2823 (comment)

Homebrew installs pkg-config under a name that shadows the system's one.

This stops one from being able to build software using the system toolchain -- because the flags it produces only work out of the box if using a Homebrew-provided compiler.

Pyenv doesn't use packages from Homebrew in Linux by default. That's because Homebrew is more problematic to use than distro-provided packages. Unlike most distros, Homebrew is rolling-release -- and as such, its dependency package upgrades break existing Pyenv-provided installations whenever their .so names change.

To build with the system's toolchain,

uninstall or at least brew unlink Homebrew-provided pkg-config; or
remove Homebrew from PATH
To build with Homebrew's toolchain,

install a Homebrew-provided compiler and
direct any builds to use it with export CC=<compiler executable to use>

@BoscoDomingo
Copy link

This happened to me too on Ubuntu 22.04 (WSL) due to using Homebrew. The solution was to run:

brew unlink pkg-config && \
CFLAGS="-I$(brew --prefix openssl)/include" \
LDFLAGS="-L$(brew --prefix openssl)/lib" \
rtx install python@latest; \
brew link pkg-config

This will temporarily unlink pkg-config and undo it after the installation (even if it fails). The CFLAGS and LDFLAGS might not be necessary in your case, but otherwise I get openssl errors and whatnot. It took about 2 minutes, so be patient

@BoscoDomingo
Copy link

@jdx Might want to include this in the docs as a workaround for common errors. This may happen often with Python 3.12, idk

@jdx
Copy link
Owner

jdx commented Nov 25, 2023

you're welcome to make some modifications to https://github.com/jdx/rtx/blob/main/docs/python.md

@BoscoDomingo
Copy link

BoscoDomingo commented Jan 8, 2024

you're welcome to make some modifications to https://github.com/jdx/rtx/blob/main/docs/python.md

Done in the mise-docs repo 👍

@jdx jdx closed this as completed Jan 8, 2024
@jdx
Copy link
Owner

jdx commented Jan 8, 2024

thanks!

roele pushed a commit to roele/mise-docs that referenced this issue Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants