Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv not detecting python in termux #7373

Closed
Noth1ngLol opened this issue Sep 13, 2024 · 15 comments · Fixed by #9005
Closed

uv not detecting python in termux #7373

Noth1ngLol opened this issue Sep 13, 2024 · 15 comments · Fixed by #9005
Labels
bug Something isn't working uv python Related to the uv python interface

Comments

@Noth1ngLol
Copy link

Noth1ngLol commented Sep 13, 2024

Disclaimer: I used AI to help write this report because English is not my first language, I apologize if anything is unclear or too detailed. Let me know if you need any clarifications

I'm encountering an issue when trying to use uv on Termux. When attempting to install packages using uv pip install, I receive an error indicating that no virtual environment or system Python installation is found, despite Python being installed and accessible.

Environment

  • OS: Termux (Android)
  • Python version: 3.11.9
  • UV version: 0.4.10
  • Termux version: 0.119.0-b1+monet36
  • Android version: 14
  • Device: vivo V2206
  • CPU architecture: aarch64
  • Kernel: Linux 4.19.157-perf+

Detailed Termux Information

Termux Variables:
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_APP__APK_FILE=/data/app/~~NkwmZsImblhcEj3x0C3ZBQ==/com.termux-9tzZyDv4vcJKhizjsUGNtQ==/base.apk
TERMUX_APP__APK_RELEASE=GITHUB
TERMUX_APP__APP_VERSION_CODE=1020
TERMUX_APP__APP_VERSION_NAME=0.119.0-b1+monet36
TERMUX_APP__DATA_DIR=/data/user/0/com.termux
TERMUX_APP__IS_DEBUGGABLE_BUILD=true
TERMUX_APP__IS_INSTALLED_ON_EXTERNAL_STORAGE=false
TERMUX_APP__PACKAGE_NAME=com.termux
TERMUX_APP__PID=31641
TERMUX_APP__TARGET_SDK=28
TERMUX_VERSION=0.119.0-b1+monet36
TERMUX__SE_FILE_CONTEXT=u:object_r:app_data_file:s0:c218,c258,c512,c768
TERMUX__SE_INFO=default:targetSdkVersion=28:complete
TERMUX__SE_PROCESS_CONTEXT=u:r:untrusted_app_27:s0:c218,c258,c512,c768
TERMUX__UID=10730
TERMUX__USER_ID=0

LD Variables:
LD_LIBRARY_PATH=
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so

Installed termux plugins:
com.termux.api versionCode:51

Steps to Reproduce

  1. Install uv on Termux
  2. Run uv pip install scapy
  3. Attempt to create a virtual environment using uv venv
  4. Try installing with --system flag: uv pip install scapy --system
  5. Create a virtual environment using python -m venv uvenv
  6. Activate the virtual environment and repeat steps 2-4

Expected Behavior

The package should install successfully, either in a virtual environment or system-wide.

Actual Behavior

Receive errors indicating that no virtual environment or system Python installation is found, despite Python being correctly installed and accessible.

Detailed Debug Output

Attempt 1: Installing scapy with UV (outside virtual environment)

~ $ uv -v pip install scapy
DEBUG uv 0.4.10
DEBUG Searching for Python interpreter in system path
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3: Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python: Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3.11: Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
error: No virtual environment found; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment

Attempt 2: Installing scapy with UV using --system flag

~ $ uv -v pip install scapy --system
DEBUG uv 0.4.10
DEBUG Searching for Python interpreter in system path
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3: Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python: Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3.11: Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
error: No system Python installation found

Attempt 3: Creating a virtual environment with UV

~ $ uv -v venv
DEBUG uv 0.4.10
DEBUG Searching for Python interpreter in managed installations or system path
DEBUG Searching for managed installations at `.local/share/uv/python`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3: Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python: Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3.11: Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
DEBUG Requested Python not found, checking for available download...
DEBUG Acquired lock for `.local/share/uv/python`
DEBUG Released lock at `/data/data/com.termux/files/home/.local/share/uv/python/.lock`
  × No interpreter found in managed installations or
  │ system path

Attempt 4: Installing scapy with UV (inside virtual environment)

(uvenv) ~ $ uv -v pip install scapy
DEBUG uv 0.4.10
DEBUG Searching for Python interpreter in system path
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python3: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python3: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python3.11: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3.11`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3: Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python: Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3.11: Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
error: No virtual environment found; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment

Attempt 5: Installing scapy with UV using --system flag (inside virtual environment)

(uvenv) ~ $ uv -v pip install scapy --system
DEBUG uv 0.4.10
DEBUG Searching for Python interpreter in system path
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python3: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/home/uvenv/bin/python3.11: Can't use Python at `/data/data/com.termux/files/home/uvenv/bin/python3.11`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3: Can't use Python at `/data/data/com.termux/files/usr/bin/python3`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python: Can't use Python at `/data/data/com.termux/files/usr/bin/python`
DEBUG Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
DEBUG Skipping bad interpreter at /data/data/com.termux/files/usr/bin/python3.11: Can't use Python at `/data/data/com.termux/files/usr/bin/python3.11`
error: No system Python installation found

Python and UV versions

(uvenv) ~ $ which python
/data/data/com.termux/files/home/uvenv/bin/python
(uvenv) ~ $ python --version
Python 3.11.9
(uvenv) ~ $ uv --version
uv 0.4.10

Additional Observations

  1. UV consistently fails to detect or use any Python installation, whether system-wide or in a virtual environment.
  2. The error messages suggest that UV is unable to use the Python interpreters it finds, but doesn't provide a specific reason why.
  3. The issue persists even after confirming that Python is correctly installed and accessible.
  4. Attempts to create a virtual environment with UV also fail due to the same interpreter detection issue.
  5. The problem occurs consistently across different scenarios and command variations, suggesting a fundamental issue with how UV interacts with the Python installation on Termux.

Attempted Solutions

  1. Upgraded Python:
    $ pkg upgrade python
    python is already the newest version (3.11.9-5).
    
  2. Tried both system-wide and virtual environment installations.
  3. Verified Python installation and accessibility.

Additional Notes

  • The issue occurs both in and out of a virtual environment.
  • Python is correctly installed and accessible, but UV seems unable to detect or use it.
  • Termux is using the apt package manager.
  • The Termux app is installed from GitHub (TERMUX_APP__APK_RELEASE=GITHUB).

Any assistance in resolving this issue would be greatly appreciated. Let me know if you need any additional information.

@samypr100
Copy link
Collaborator

samypr100 commented Sep 14, 2024

Interesting, seems like uv is not detecting python in Android (installed as part of the Termux APK).
How did you install uv within Termux? Downloading an aarch64 binary directly?

I decided to give it a quick try, and check if get_interpreter_info.py even works, but it was definitely going the branch of

print(json.dumps({"result": "error", "kind": "libc_not_found"}))
sys.exit(0)

which is likely the main culprit here to some degree.

I modified it to return unknown for OS name intead and got below:

{
  "result": "success",
  "markers": {
    "implementation_name": "cpython",
    "implementation_version": "3.11.9",
    "os_name": "posix",
    "platform_machine": "aarch64",
    "platform_python_implementation": "CPython",
    "platform_release": "[Redacted Android Release ID]",
    "platform_system": "Linux",
    "platform_version": "[Redacted Android Version ID]",
    "python_full_version": "3.11.9",
    "python_version": "3.11",
    "sys_platform": "linux"
  },
  "sys_base_prefix": "/data/data/com.termux/files/usr",
  "sys_base_exec_prefix": "/data/data/com.termux/files/usr",
  "sys_prefix": "/data/data/com.termux/files/usr",
  "sys_base_executable": "/data/data/com.termux/files/usr/bin/python",
  "sys_executable": "/data/data/com.termux/files/usr/bin/python",
  "sys_path": [
    "/data/data/com.termux/files/home",
    "/data/data/com.termux/files/usr/lib/python311.zip",
    "/data/data/com.termux/files/usr/lib/python3.11",
    "/data/data/com.termux/files/usr/lib/python3.11/lib-dynload",
    "/data/data/com.termux/files/usr/lib/python3.11/site-packages"
  ],
  "stdlib": "/data/data/com.termux/files/usr/lib/python3.11",
  "scheme": {
    "platlib": "/data/data/com.termux/files/usr/lib/python3.11/site-packages",
    "purelib": "/data/data/com.termux/files/usr/lib/python3.11/site-packages",
    "include": "/data/data/com.termux/files/usr/include/python3.11",
    "scripts": "/data/data/com.termux/files/usr/bin",
    "data": "/data/data/com.termux/files/usr"
  },
  "virtualenv": {
    "purelib": "lib/python3.11/site-packages",
    "platlib": "lib/python3.11/site-packages",
    "include": "include/site/python3.11",
    "scripts": "bin",
    "data": ""
  },
  "platform": {
    "os": {
      "name": "unknown",
      "major": -1,
      "minor": -1
    },
    "arch": "aarch64"
  },
  "manylinux_compatible": true,
  "gil_disabled": false,
  "pointer_size": "64"
}

@Noth1ngLol
Copy link
Author

Noth1ngLol commented Sep 14, 2024

Interesting, seems like uv is not detecting python in Android (installed as part of the Termux APK).
How did you install uv within Termux? Downloading an aarch64 binary directly?

I installed uv normally using pkg in Termux. The exact command I used was:

pkg install uv

I didn't download any binaries directly or use any special installation methods.

I decided to give it a quick try, and check if get_interpreter_info.py even works, but it was definitely going the branch of

print(json.dumps({"result": "error", "kind": "libc_not_found"}))
sys.exit(0)
which is likely the main culprit here to some degree.

Your findings about the get_interpreter_info.py script are interesting. Is there a way for me to modify this script on my end to help diagnose the issue further? I'm willing to run any tests or provide any additional information that might be helpful in resolving this problem.

Also, given that the modified script you ran was able to detect the Python installation correctly, do you have any ideas about potential fixes or workarounds I could try?

@ilotoki0804
Copy link

ilotoki0804 commented Sep 25, 2024

@Noth1ngLol Did you solve the problem? If so, can you share your solution?

@mgedmin
Copy link

mgedmin commented Oct 4, 2024

Came here because of the same problem. I think the issue was closed prematurely?

It's probably a duplicate of #2408 anyway.

@zanieb zanieb reopened this Oct 4, 2024
@zanieb
Copy link
Member

zanieb commented Oct 4, 2024

Let's use this issue to track the Python interpreter query bug and #2408 to track broader support.

@zanieb zanieb added bug Something isn't working uv python Related to the uv python interface labels Oct 4, 2024
@Neurovert
Copy link

Interesting, seems like uv is not detecting python in Android (installed as part of the Termux APK). How did you install uv within Termux? Downloading an aarch64 binary directly?

I decided to give it a quick try, and check if get_interpreter_info.py even works, but it was definitely going the branch of

print(json.dumps({"result": "error", "kind": "libc_not_found"}))
sys.exit(0)

which is likely the main culprit here to some degree.

I modified it to return unknown for OS name intead and got below:

{
  "result": "success",
  "markers": {
    "implementation_name": "cpython",
    "implementation_version": "3.11.9",
    "os_name": "posix",
    "platform_machine": "aarch64",
    "platform_python_implementation": "CPython",
    "platform_release": "[Redacted Android Release ID]",
    "platform_system": "Linux",
    "platform_version": "[Redacted Android Version ID]",
    "python_full_version": "3.11.9",
    "python_version": "3.11",
    "sys_platform": "linux"
  },
  "sys_base_prefix": "/data/data/com.termux/files/usr",
  "sys_base_exec_prefix": "/data/data/com.termux/files/usr",
  "sys_prefix": "/data/data/com.termux/files/usr",
  "sys_base_executable": "/data/data/com.termux/files/usr/bin/python",
  "sys_executable": "/data/data/com.termux/files/usr/bin/python",
  "sys_path": [
    "/data/data/com.termux/files/home",
    "/data/data/com.termux/files/usr/lib/python311.zip",
    "/data/data/com.termux/files/usr/lib/python3.11",
    "/data/data/com.termux/files/usr/lib/python3.11/lib-dynload",
    "/data/data/com.termux/files/usr/lib/python3.11/site-packages"
  ],
  "stdlib": "/data/data/com.termux/files/usr/lib/python3.11",
  "scheme": {
    "platlib": "/data/data/com.termux/files/usr/lib/python3.11/site-packages",
    "purelib": "/data/data/com.termux/files/usr/lib/python3.11/site-packages",
    "include": "/data/data/com.termux/files/usr/include/python3.11",
    "scripts": "/data/data/com.termux/files/usr/bin",
    "data": "/data/data/com.termux/files/usr"
  },
  "virtualenv": {
    "purelib": "lib/python3.11/site-packages",
    "platlib": "lib/python3.11/site-packages",
    "include": "include/site/python3.11",
    "scripts": "bin",
    "data": ""
  },
  "platform": {
    "os": {
      "name": "unknown",
      "major": -1,
      "minor": -1
    },
    "arch": "aarch64"
  },
  "manylinux_compatible": true,
  "gil_disabled": false,
  "pointer_size": "64"
}

@samypr100 so does uv venv work properly for you? The json result you're showing is probably result of running just the get_interpreter_info.py script, right? Or did you edit it successfully and now uv detects the venv correctly?

@samypr100
Copy link
Collaborator

@samypr100 so does uv venv work properly for you? The json result you're showing is probably result of running just the get_interpreter_info.py script, right? Or did you edit it successfully and now uv detects the venv correctly?

Sorry, I did not recompile uv with the changes on Android to be able to test that.

@MatiasHiltunen
Copy link

I have encountered similar errors with 0.51 version installed using pkg.

I tried multiple other installation methods with no success, this was first issue while building with cargo install:

error: subprocess-exited-with-error
warning: [email protected]: c/linux.c:96:11: error: call to undeclared library function 'index' with type 'char *(const char *, int)';
ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
error: failed to run custom build command for `sys-info v0.9.1`

For the Python-based installation using pipx:

pip failed to build package:
    uv

Some possibly relevant errors from pip install: 
error: subprocess-exited-with-error           warning: [email protected]: c/linux.c:96:11: error: call to undeclared library function 'index' with type 'char *(const char *, int)'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    error: failed to run custom build command for `sys-info v0.9.1`
    cargo:warning=c/linux.c:96:11: error: call to undeclared library function 'index' with type 'char *(const char *, int)'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]            Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/data/data/com.termux/files/home/.local/share/pipx/venvs/uv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1                                         ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (uv)

This ssue appears to stem from the sys-info crate, specifically in c/linux.c. The code calls the index function, which is deprecated and not supported in ISO C99 or later. Android's Bionic libc (used by Termux) is stricter and does not include this deprecated function.

I'm going to keep looking more into this as I suspect the issue is more related to external dependencies or env-variables that termux does differently.

@dead10ck
Copy link
Contributor

I tried my hand at this in #9005. Not sure if this is a good solution, but it seems to work.

@MatiasHiltunen
Copy link

I tried my hand at this in #9005. Not sure if this is a good solution, but it seems to work.

Good to know that there is a solution! Have you tried building the project in termux with these changes?

MatiasHiltunen added a commit to MatiasHiltunen/sys-info-rs that referenced this issue Nov 13, 2024
Replaced deprecated index() functions with strchr(). Not sure if this is possible to do as drop-in replacement but this fixed my error, more about it here: astral-sh/uv#7373 (comment)
@MatiasHiltunen
Copy link

About the build error that I posted, there is a pending PR that worked for me to fix the issue above that I got during build process: FillZpp/sys-info-rs#118

Not sure how actively maintained the sys-info crate is as last commit has been 2 years ago

konstin pushed a commit that referenced this issue Nov 21, 2024
## Summary

On Termux, uv currently fails to find any interpreter because it can't
find a glibc version, because there isn't one. But the Python
interpreter is still functional nonetheless.

So, when glibc cannot be found, simply return 0 for the version numbers
and mark the interpreter as being incompatible with manylinux

I really don't know if this is the right way to address this, but I can
attest that manual testing shows uv appears to be fully functional, at
least for pip and virtualenvs.

Fixes #7373

## Test Plan

I tried running the test suite, and after some tweaks, a good portion of
the test suite passes as well. A significant number of tests fail, but
this appears to be due to minor differences in output, like warnings
about hard links not working (hard links are completely disallowed on
Android), differences in the number of files removed, etc. The test
suite seems to be very sensitive to minor variations in output.
@ChristianF88
Copy link

ChristianF88 commented Nov 26, 2024

Fyi: I just tried and uv is still not detecting my python interpreter in termux:

Here is what I did:

pkg install uv
uv --version 
# uv 0.5.4
python --version
# Python 3.12.7
which python
# /data/data/com.termux/files/usr/bin/python
uv python find 3.12
# error: No interpreter found for Python 3.12 in virtual environments, managed installations, or search path 

Installing a new version does also not work:

uv python install 3.11
# error: No download found for request: cpython-3.11-linux-aarch64-none

Cheers

@MatiasHiltunen
Copy link

MatiasHiltunen commented Nov 26, 2024

Also as long as the sys-info (v.0.9.1) crate is there it is not possible to use other installation methods that involves build process due to android 14(?) compiler restrictions with deprecated C-code. I think I saw somewhere discussion about replacing sys-info crate with more up-to-date crate, couldn't find it anymore though. With rust I have noticed that many crates that target android as a platform can not be compiled to termux because platform is detected to be aarch64-linux-android and those packages expect (correctly) to bind to android activity as they would if we were building android application, even though termux does not need the program to bind to android activity to work. When activity is not found, compile can not go trough. In some cases, I have modified the platform detection code so that compiler skips the activity check and build goes nicely through natively to aarch64-linux-android. This is not a good practise though. Perhaps this is a discussion that should be taken to Termux (or rust) community and see if there is something that could be done to help rust compiler to detect termux better as emulated linux environment and prevent the rust from expecting the android activity to be available when its not needed?

@samypr100
Copy link
Collaborator

samypr100 commented Nov 26, 2024

Has there been a release yet that includes the fix from the PR? We'd have to wait for pkg to also have the update on termux repos.

@dead10ck
Copy link
Contributor

No I don't believe there has been a release yet that includes the fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working uv python Related to the uv python interface
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants