Process.cmdline returns an empty list on OS X 10.6 #83

giampaolo · 2014-05-23T14:31:52Z

From [email protected] on March 20, 2010 22:01:33

The `cmdline` attribute of a `Process` instance always returns an empty list.  If I loop over all 
processes and print out the value of `cmdline`, every process return `[]`.  For example:

import psutil
for p in psutil.get_process_list():
    print p.cmdline


Prints a '[]' for every process running.  I am running OS X 10.6

Original issue: http://code.google.com/p/psutil/issues/detail?id=83

The text was updated successfully, but these errors were encountered:

giampaolo · 2014-05-23T14:31:54Z

From [email protected] on March 20, 2010 14:16:13

Unfortunately I only have access to OS X 10.4, and it works here so I can't reproduce
the issue. 

Is the behavior the same when running as root using sudo? If you don't have
sufficient privileges, the default is to return an empty list for the cmdline, and on
OS X you can't read other user processes' information except when running as root.

Status: Accepted

giampaolo · 2014-05-23T14:31:55Z

From [email protected] on March 20, 2010 14:24:34

I just tried the same procedure, but as root, and I get the same results--just a bunch of empty lists.

giampaolo · 2014-05-23T14:31:56Z

From [email protected] on March 20, 2010 14:32:44

hmmm... well, until I can afford a new mac or an upgrade to 10.6 I can't reproduce
the problem to debug so we may be stuck for now. If it's not an access problem then
it means the actual cmdline reading code in getcmdargs() is failing to find null
terminated strings representing the cmdline arguments. Why that might be I don't
really know, perhaps the sysctl() interface has changed in 10.6 or something along
those lines. 

Did you get any errors when compiling with setup.py ? Can you go back and run
"setup.py clean" then run "setup.py build" again to see if there's any warnings or
compiler errors?

Labels: OpSys-OSX

giampaolo · 2014-05-23T14:31:56Z

From [email protected] on March 20, 2010 15:02:24

I ran "setup.py build" again after cleaning and there were no issues.

I installed PSI, as I noticed some of your code was based off it, tested it and it behaved as I expected:

p = psi.process.Process(pid=37209)

`p.args` returns a list of arguments.  PSI is nice, but I like that psutil has support for Windows as well.  I'd like to 
help fix this bug--let me know how I can help.

giampaolo · 2014-05-23T14:31:57Z

From [email protected] on March 20, 2010 15:16:11

odd, the PSI code is very similar, I'm not sure why it would be failing for you...
are you using the latest SVN release of psutil or one of the older packages?

giampaolo · 2014-05-23T14:31:58Z

From [email protected] on March 20, 2010 15:22:51

Here's what I suggest, go to psutil/arch/osx/process_info.c and go down to line 217,
in the function getcmdargs(). You'll see the ERROR_RETURN label, go to the line
"PyErr_SetString(PyExc_SystemError, err);" and change it like so (add the printf line):

-----
ERROR_RETURN:
    // Clean up.
    if (NULL != procargs) {
        free(procargs);
    }
    if (NULL != err) {
        printf("getcmdargs() err: %s", err);
        PyErr_SetString(PyExc_SystemError, err);
        return -1;
-----


Don't change anything else, just add that printf line and it should print out an
error message if getcmdargs() has one available. That may help us determine the problem. 


Thanks, 

-Jay

giampaolo · 2014-05-23T14:31:58Z

From [email protected] on March 20, 2010 22:10:35

I've made the modification, cleaned, recompiled and reinstalled the 
code and ran through the same procedure, but there is no difference.  No error is printed.

As a reference, here is the output from the build command (there are some warnings, but no errors):

running build
running build_py
creating build
creating build/lib.macosx-10.6-universal-2.6
creating build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/__init__.py -> build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/_psbsd.py -> build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/_pslinux.py -> build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/_psmswindows.py -> build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/_psosx.py -> build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/error.py -> build/lib.macosx-10.6-universal-2.6/psutil
copying psutil/wmi.py -> build/lib.macosx-10.6-universal-2.6/psutil
running build_ext
building '_psutil_osx' extension
creating build/temp.macosx-10.6-universal-2.6
creating build/temp.macosx-10.6-universal-2.6/psutil
creating build/temp.macosx-10.6-universal-2.6/psutil/arch
creating build/temp.macosx-10.6-universal-2.6/psutil/arch/osx
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv 
-Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -
I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6
 -c psutil/_psutil_osx.c -o build/temp.macosx-10.6-universal-2.6/psutil/_psutil_osx.o
In file included from psutil/_psutil_osx.c:23:
/usr/include/mach/shared_memory_server.h:48:2: warning: #warning 
"<mach/shared_memory_server.h> is deprecated.  Please use <mach/shared_region.h> instead."
psutil/_psutil_osx.c: In function ‘get_system_cpu_times’:
psutil/_psutil_osx.c:568: warning: passing argument 3 of ‘host_statistics’ from incompatible pointer type
In file included from psutil/_psutil_osx.c:23:
/usr/include/mach/shared_memory_server.h:48:2: warning: #warning 
"<mach/shared_memory_server.h> is deprecated.  Please use <mach/shared_region.h> instead."
psutil/_psutil_osx.c: In function ‘get_system_cpu_times’:
psutil/_psutil_osx.c:568: warning: passing argument 3 of ‘host_statistics’ from incompatible pointer type
In file included from psutil/_psutil_osx.c:23:
/usr/include/mach/shared_memory_server.h:48:2: warning: #warning 
"<mach/shared_memory_server.h> is deprecated.  Please use <mach/shared_region.h> instead."
psutil/_psutil_osx.c: In function ‘get_process_cpu_times’:
psutil/_psutil_osx.c:301: warning: cast from pointer to integer of different size
psutil/_psutil_osx.c: In function ‘get_memory_info’:
psutil/_psutil_osx.c:410: warning: cast from pointer to integer of different size
psutil/_psutil_osx.c: In function ‘get_system_cpu_times’:
psutil/_psutil_osx.c:568: warning: passing argument 3 of ‘host_statistics’ from incompatible pointer type
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv 
-Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -
I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6
 -c psutil/arch/osx/process_info.c -o build/temp.macosx-10.6-universal-
2.6/psutil/arch/osx/process_info.o
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch 
ppc -arch x86_64 
build/temp.macosx-10.6-universal-2.6/psutil/_psutil_osx.o build/temp.macosx-
10.6-universal-2.6/psutil/arch/osx/process_info.o -o build/lib.macosx-10.6-universal-2.6/_psutil_osx.so

giampaolo · 2014-05-23T14:31:59Z

From [email protected] on March 21, 2010 11:22:14

I did a little more digging around, and it looks like your initial 
hunch was pretty close--it appears to be failing due to some sort of permission issue (or at least, the code 
thinks it is a permission issue).  The code get executed up to, in process_info.c:

===

        if (EINVAL == errno) { // invalid == access denied for some reason
            free(procargs);
            return ARGS_ACCESS_DENIED;  /* Insufficient privileges */
        }

===

However, if I run python as sudo, the permission issue doesn't go away.  Even I login as root, the problem is the same.

giampaolo · 2014-05-23T14:32:00Z

From [email protected] on March 22, 2010 06:14:21

Interesting... that means sysctl is returning EINVAL which doesn't necessarily mean
permissions problems. It could also be invalid parameters to sysctl. Can you try
making the following change: 

    mib[0] = CTL_KERN;
    mib[1] = KERN_PROCARGS2;
    mib[2] = (int)pid;

To: 

    mib[0] = CTL_KERN;
    mib[1] = KERN_PROCARGS2;
    mib[2] = pid;

It might be the PID being cast to an int that's causing a problem. Give that a try
and let me know if that makes any difference.

giampaolo · 2014-05-23T14:32:01Z

From [email protected] on March 22, 2010 09:30:54

Unfortunately that still doesn't solve the problem.  I am trying to compare psutil to PSI to see where there might 
be a difference.

giampaolo · 2014-05-23T14:32:02Z

From [email protected] on March 25, 2010 17:23:29

Let's try cutting this down to a minimal app where we can reproduce the problem.
Please download the attached arguments.c and compile it on your system (simple 'gcc
arguments.c' should do it), then run it like so: 

  [user@host ]$ sudo ./a.out   
  getting PID for 8094
  sysctl returned successfully
  arg[151572]: ./a.out

  [user@host ]$ sudo ./a.out 62
  getting PID for 62
  sysctl returned successfully
  arg[151601]: /usr/sbin/coreaudiod


Note the use of 'sudo' both times, and the first time you're running it with no
arguments, second time around pass it the name of a known valid PID (the default is
to use it's own PID, otherwise it uses the first argument). 

Let me know what you get for output, hopefully it reproduces the problem and you get
an EINVAL error on all processes. At least that way we can work in a smaller codebase. 

Thanks!

-Jay

Attachment: arguments.c

giampaolo · 2014-05-23T14:32:02Z

From [email protected] on March 25, 2010 18:52:43

Looks like this is actually working.  Here's what I get:

wnielson$ sudo ./a.out 78376
getting PID for 78376
sysctl returned successfully
arg[135194]: login
arg[135198]: -pf
arg[135207]: wnielson

giampaolo · 2014-05-23T14:32:03Z

From [email protected] on March 26, 2010 17:31:58

Hah! Of course it works, that'd be too easy if it showed the same problem in a small
reproducible test case. I am not sure what to make of that. I copied and pasted the
code from psutil so it ought to be (not) working the same. I have a couple of C
programmer friends who have access to 10.6 so I'll continue trying to work this as
best I can remotely.

Status: Started

giampaolo · 2014-05-23T14:32:04Z

From [email protected] on April 03, 2010 19:40:54

Just to keep this issue updated... I finally gave in and ordered a copy of 10.6 and
will try upgrading my macbook so I can debug this problem. I'll keep everyone posted
with any progress.

giampaolo · 2014-05-23T14:32:04Z

From [email protected] on April 12, 2010 11:55:48

Ok, so of course now that I've updated to Snow Leopard and tried to reproduce this, it works perfectly fine for me 
and there's no issue with retrieving the cmdline (using current trunk version). I'll keep digging around - can you 
let me know what version of Python you're using, and whether it was installed via an installer or from source etc? 

Thanks

giampaolo · 2014-05-23T14:32:05Z

From [email protected] on April 12, 2010 12:31:20

Could you also tell if it's a 32 or 64 bit system?

giampaolo · 2014-05-23T14:32:06Z

From [email protected] on August 05, 2010 17:07:10

I'm marking this PostPoned until I have access to a 64bit OS X system 
where I can reproduce this, since it works as designed on my machine and I can't debug it.

Status: PostPoned

giampaolo · 2014-05-23T14:32:06Z

From [email protected] on August 30, 2010 12:59:31

Issue 106 has been merged into this issue.

giampaolo · 2014-05-23T14:32:08Z

From [email protected] on August 31, 2010 06:15:55

Status: ReOpened

giampaolo · 2014-05-23T14:32:08Z

From [email protected] on September 13, 2010 10:40:22

I have a 10.6 Snow Leopard 64 bit Macbook pro and I'd like to voluneer to help debug. Let me know what needs to be done.

giampaolo · 2014-05-23T14:32:09Z

From [email protected] on September 13, 2010 11:03:01

have moved over to psi (still willing to help)...any reason why these 
are 2 different projects...sounds to me like there should be a unified library to do this

giampaolo · 2014-05-23T14:32:10Z

From g.rodola on September 13, 2010 11:31:21

Despite similar, psi and psutil are two different projects maintained by different teams.
Aside from the API, differences include support for different 
platforms (e.g. psutil works on Windows and FreeBSD while psi works on Solaris and some other platforms which are not covered by psutil).
Also, psutil is centered around processes only while PSI also provides various information about the system.

giampaolo · 2014-05-23T14:32:10Z

From g.rodola on September 20, 2010 15:35:59

Labels: 64bit

giampaolo · 2014-05-23T14:32:11Z

From g.rodola on November 11, 2010 10:28:17

Confirmed this does not affect OSX 10.5.

giampaolo · 2014-05-23T14:32:12Z

From g.rodola on November 11, 2010 11:27:33

Funnily enough, I've just got access against a 10.6 64-bit box and everything is fine there:

bash-3.2$ python2.7
Python 2.7 ( r27 :82508, Jul  3 2010, 21:12:11) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil, os
>>> psutil.Process(os.getpid()).cmdline
['/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python']

>>> ^D
bash-3.2$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.6.4
BuildVersion:   10F569
bash-3.2$

giampaolo · 2014-05-23T14:32:12Z

From g.rodola on November 11, 2010 12:09:20

[email protected], if you're still reading, could you please try this against python 2.7?

giampaolo · 2014-05-23T14:32:13Z

From [email protected] on November 16, 2010 14:35:04

r823 has completely rewritten cmdline code for OS X, I'd be curious to know if this issue still occurs with the latest trunk as well.

giampaolo · 2014-05-23T14:32:14Z

From [email protected] on November 16, 2010 14:38:54

Status: WaitingForReview

giampaolo · 2014-05-23T14:32:14Z

From [email protected] on November 26, 2010 17:40:53

Had a chance to test this on 64bit OS X 10.6 and confirmed the 
rewrite of the code resolved the problem. With Dave's help I was able 
to check 0.1.3 and observed the issue (blank cmdline). Then we tested the current trunk and the cmdline was populated properly.

Status: FixedInSVN

giampaolo · 2014-05-23T14:32:15Z

From g.rodola on November 27, 2010 11:55:09

Labels: Milestone-0.2.1

giampaolo · 2014-05-23T14:32:16Z

From [email protected] on December 09, 2010 04:16:26

I think the problem was caused by KERN_ARGMAX systcl returning 4GB when called from 64bit code.

giampaolo · 2014-05-23T14:32:16Z

From g.rodola on March 20, 2011 14:55:37

Status: Fixed

giampaolo · 2014-05-23T14:32:17Z

From g.rodola on March 01, 2013 14:15:49

Issue 106 has been merged into this issue.

giampaolo · 2014-05-23T14:32:18Z

From g.rodola on March 02, 2013 03:52:29

Updated csets after the SVN -> Mercurial migration: r27 == revision 446c22c12039 r823 == revision 31161e959cff

giampaolo · 2014-05-23T14:32:18Z

From g.rodola on March 02, 2013 03:54:10

Issue 106 has been merged into this issue.

giampaolo self-assigned this May 23, 2014

giampaolo closed this as completed May 23, 2014

This was referenced May 23, 2014

psutil Does Not Return path or cmdline on os x #106

Closed

Cached properties ignoring terminated processes and code refactoring #126

Closed

Refactor OS X cmdline code #128

Closed

psutil cannot create Process object, OSX #135

Closed

pyup-bot mentioned this issue Jun 2, 2017

Pin psutil to latest version 5.2.2 crowddynamics/crowddynamics#123

Closed

pyup-bot mentioned this issue Jul 21, 2017

Pin psutil to latest version 5.2.2 mozilla-services/autopush#975

Closed

pyup-bot mentioned this issue Aug 16, 2017

Pin psutil to latest version 5.2.2 springermac/pyinstaller#37

Closed

giampaolo added the macos label Nov 15, 2020

Process.cmdline returns an empty list on OS X 10.6 #83

Process.cmdline returns an empty list on OS X 10.6 #83

Comments

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014