pipx list prints psutil error on android #1084

purarue · 2023-10-12T16:02:59Z

Describe the bug

On android, running many different commands pipx list/pipx install print an error like this:

$ pipx  list --verbose
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 399, in wrapper
    return cache[key]
           ~~~~~^^^^^
KeyError: (('/proc',), frozenset())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 285, in <module>
    set_scputimes_ntuple("/proc")
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 401, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 728, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/stat'
pipx >(setup:757): pipx version is 1.1.0
pipx >(setup:758): Default python interpreter is '/data/data/com.termux/files/usr/bin/python3'
pipx >(needs_upgrade:69): Time since last upgrade of shared libs, in seconds: 128. Upgrade will be run by pipx if greater than 2592000.
venvs are in /data/data/com.termux/files/home/.local/pipx/venvs
apps are exposed on your $PATH at /data/data/com.termux/files/home/.local/bin
   package currencyconverter 0.17.11, installed using Python 3.11.6
    - currency_converter
    -

similarly with any pipx install:

$ pipx install flake8            
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 399, in wrapper
    return cache[key]
           ~~~~~^^^^^
KeyError: (('/proc',), frozenset())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 285, in <module>
    set_scputimes_ntuple("/proc")
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 401, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 728, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/stat'
⚠️  File exists at /data/data/com.termux/files/home/.local/bin/flake8 and points to /data/data/com.termux/files/home/.local/bin/flake8,
    not /data/data/com.termux/files/home/.local/pipx/venvs/flake8/bin/flake8. Not modifying.
  installed package flake8 6.1.0, installed using Python 3.11.6

Im not sure if this actually impacts functionality, i.e. I can still list/install packages, it just probably shouldnt print this error or do something custom on android

I had thought this might be because on android the typical linux directories may not be present. but /proc is present on android:

$ lls /proc 2>/dev/null
19494	   consoles	kmsg	      softirqs
19498	   cpuinfo	kpagecgroup   stat
19500	   crypto	kpagecount    swaps
19501	   debugdriver  kpageflags    sys
26163	   device-tree  loadavg       sysrq-trigger
29593	   devices	locks	      thread-self
30569	   diskstats	meminfo       timer_list
30605	   driver	misc	      tty
31448	   execdomains  modules       uid_concurrent_active_time
31459	   filesystems  mounts        uid_concurrent_policy_time
asound	   fs		net	      uid_cputime
ath_pktlog  fts		pagetypeinfo  uid_io
bldrlog     interrupts	partitions    uid_procstat
buddyinfo   iomem	pressure      uid_time_in_state
bus	   ioports	sched_debug   uptime
cgroups     irq		schedstat     version
cld	   kallsyms	scsi	      vmallocinfo
cmdline     key-users	self	      vmstat
config.gz   keys	slabinfo      zoneinfo

How to reproduce

Believe I'm on the latest versions:

$ pip show psutil pipx      
Name: psutil
Version: 5.9.5
Summary: Cross-platform lib for process and system monitoring in Python.
Home-page: https://github.com/giampaolo/psutil
Author: Giampaolo Rodola
Author-email: [email protected]
License: BSD-3-Clause
Location: /data/data/com.termux/files/usr/lib/python3.11/site-packages
Requires: 
Required-by:
---
Name: pipx
Version: 1.2.0
Summary: Install and Run Python Applications in Isolated Environments
Home-page: 
Author: 
Author-email: Chad Smith <[email protected]>
License: 
Location: /data/data/com.termux/files/usr/lib/python3.11/site-packages
Requires: argcomplete, packaging, userpath
Required-by:

Running pipx list would print the error

I dont think this is related to this other issue

The text was updated successfully, but these errors were encountered:

dukecat0 · 2023-10-12T23:38:45Z

I guess it's related to giampaolo/psutil#2195.

purarue · 2023-10-12T23:54:17Z

Ah, thanks

Seems that it does not impact functionality, from the comment here:

The error originates from here and occurs at import time. It's just printed, meaning it does not crash the app.

Otherwise seems like pipx works fine, it just is a bit annoying to parse through the output sometimes and tell if what Im looking at actually errored or if its just the psutil traceback.

Not sure what the status on that is... it does look like they renamed the error its meant to show this to the user and I do see the new error text that was updated in version 5.9.5, but that issue is not closed yet...

dukecat0 · 2023-10-13T12:04:43Z

Do you have any idea why this error was outputted? psutil isn't one of the dependencies of pipx.

purarue · 2023-10-13T14:38:27Z

Oh, that is even more strange

I will try creating a venv and installing pipx there to see if I can debug

purarue · 2023-10-13T15:01:25Z

Followed the import chain:

# code object from '/data/data/com.termux/files/usr/lib/python3.11/site-packages/userpath/__pycache__/utils.cpython-311.pyc'
# /data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/__pycache__/__init__.cpython-311.pyc matches /data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/__init__.py
# code object from '/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/__pycache__/__init__.cpython-311.pyc'
# /data/data/com.termux/files/usr/lib/python3.11/__pycache__/__future__.cpython-311.pyc matches /data/data/com.termux/files/usr/lib/python3.11/__future__.py
# code object from '/data/data/com.termux/files/usr/lib/python3.11/__pycache__/__future__.cpython-311.pyc'
import '__future__' # <_frozen_importlib_external.SourceFileLoader object at 0x73e1c73710>
import 'pwd' # <class '_frozen_importlib.BuiltinImporter'>

It looks to be a transient, optional dependency from userpath:

https://github.com/ofek/userpath/blob/44d835120825265aa5402673e9e907302d5e7bbb/userpath/utils.py#L5-L8

Can confirm if I uninstall psutil the error is gone and pipx continues to work fine

I think will create an issue in userpath, this probably shouldnt print an error just on import if psutils fails that check...

Though, I dont think psutil is incredibly broken on android or anything, it is just that it prints this stack trace if a check fails. typical functionality works fine:

In [1]: import psutil
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 285, in <module>
    set_scputimes_ntuple("/proc")
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 432, in wrapper
    raise raise_from(err, None)
          ^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 3, in raise_from
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 430, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 765, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/stat'

In [2]: pid = os.getpid()

In [3]: process = psutil.Process(pid)

In [4]: process
Out[4]: psutil.Process(pid=20599, name='ipython', status='running')

dukecat0 · 2023-10-13T17:36:28Z

Thanks for your investigation, and I will close this issue since there's nothing we can do here.

giampaolo · 2023-10-13T18:42:30Z

I have fixed this in giampaolo/psutil@becbe86. The error/exception is no longer printed on screen. It is only shown by using PSUTIL_DEBUG env var.

purarue changed the title ~~pipx list prints~~ pipx list prints error on android Oct 12, 2023

purarue changed the title ~~pipx list prints error on android~~ pipx list prints psutil error on android Oct 12, 2023

dukecat0 closed this as completed Oct 13, 2023

purarue mentioned this issue Oct 13, 2023

[Termux] PermissionError: [Errno 13] Permission denied: '/proc/stat' giampaolo/psutil#2195

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipx list prints psutil error on android #1084

pipx list prints psutil error on android #1084

purarue commented Oct 12, 2023 •

edited

Loading

dukecat0 commented Oct 12, 2023

purarue commented Oct 12, 2023

dukecat0 commented Oct 13, 2023

purarue commented Oct 13, 2023

purarue commented Oct 13, 2023 •

edited

Loading

dukecat0 commented Oct 13, 2023

giampaolo commented Oct 13, 2023

pipx list prints psutil error on android #1084

pipx list prints psutil error on android #1084

Comments

purarue commented Oct 12, 2023 • edited Loading

dukecat0 commented Oct 12, 2023

purarue commented Oct 12, 2023

dukecat0 commented Oct 13, 2023

purarue commented Oct 13, 2023

purarue commented Oct 13, 2023 • edited Loading

dukecat0 commented Oct 13, 2023

giampaolo commented Oct 13, 2023

purarue commented Oct 12, 2023 •

edited

Loading

purarue commented Oct 13, 2023 •

edited

Loading