Could not import psutil on FreeBSD 9 #325

giampaolo · 2014-05-23T15:04:03Z

From [email protected] on September 13, 2012 18:34:39

What steps will reproduce the problem?  
1. Install with pip on FreeBSD 9.0-RELEASE-p3, Python 2.7.3, in virtualenv.
2. Open python shell.
3. import psutil 

What is the expected output?  
The module is imported. 

What do you see instead?  
Python 2.7.3 (default, Sep 13 2012, 11:46:06)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File 
"/usr/home/voodoo_people/python/lib/python2.7/site-packages/psutil/__init__.py",
 line 88, in <module>
    import psutil._psbsd as _psplatform
  File 
"/usr/home/voodoo_people/python/lib/python2.7/site-packages/psutil/_psbsd.py", 
line 28, in <module>
    TOTAL_PHYMEM = _psutil_bsd.get_virtual_mem()[0]
SystemError: error return without exception set 

What version of psutil are you using? What Python version?  
psutil 0.6.1, Python 2.7.3. 

On what operating system? Is it 32bit or 64bit version?  
Please provide any additional information below.

Original issue: http://code.google.com/p/psutil/issues/detail?id=325

The text was updated successfully, but these errors were encountered:

giampaolo · 2014-05-23T15:04:04Z

From g.rodola on September 13, 2012 09:36:14

Have you tried latest SVN revision?

giampaolo · 2014-05-23T15:04:04Z

From [email protected] on September 13, 2012 10:05:42

The last SVN version:

Python 2.7.3 (default, Sep 13 2012, 11:46:06)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
/usr/home/voodoo_people/python/lib/python2.7/site-packages/psutil-0.6.2-py2.7-freebsd-9.0-RELEASE-p3-amd64.egg/_psutil_bsd.py:3:
 UserWarning: Module _psutil_bsd was already imported from 
/usr/home/voodoo_people/python/lib/python2.7/site-packages/psutil-0.6.2-py2.7-freebsd-9.0-RELEASE-p3-amd64.egg/_psutil_bsd.pyc,
 but /usr/home/wyse/build/psutil-read-only is being added to sys.path
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "psutil/__init__.py", line 88, in <module>
    import psutil._psbsd as _psplatform
  File "psutil/_psbsd.py", line 28, in <module>
    TOTAL_PHYMEM = _psutil_bsd.get_virtual_mem()[0]
SystemError: error return without exception set
>>>

giampaolo · 2014-05-23T15:04:05Z

From g.rodola on September 13, 2012 10:07:18

Ok, something is definitively wrong then.
I won't be able to look into this any time soon though.

giampaolo · 2014-05-23T15:04:06Z

From [email protected] on September 13, 2012 10:19:59

Just got it running with 0.4.1 version.

giampaolo · 2014-05-23T15:04:06Z

From g.rodola on December 13, 2012 06:31:53

Getting back to this.
Unfortunately I'm not able to reproduce the issue.

Would you be able to try to debug the problem yourself in order to figure out 
what's the function which is causing the error?

You should look here: 
https://code.google.com/p/psutil/source/browse/trunk/psutil/_psutil_bsd.c?spec=svn1550&r=1550#581
 Basically, what's going on in there is that one of those sysctl() or 
sysctlbyname() calls is failing and the first thing to do is figure out which one.
The weird thing is that contrarily to what stated in the man pages errno is not set.

Status: Started
Labels: OpSys-FreeBSD Crash

giampaolo · 2014-05-23T15:04:07Z

From [email protected] on January 28, 2013 05:12:55

Does psutil work fine with GENERIC kernel? This could be 
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/172803

giampaolo · 2014-05-23T15:04:08Z

From g.rodola on January 28, 2013 05:29:15

Not sure what you mean by GENERIC kernel but we might try the patch proposed in 
that ticket.
Can someone who's able to reproduce this issue try this patch (save it into a 
file and apply with "patch -p0 < file.patch")?


Index: psutil/_psutil_bsd.c
===================================================================
--- psutil/_psutil_bsd.c    (revisione 1560)
+++ psutil/_psutil_bsd.c    (copia locale)
@@ -584,11 +584,16 @@
 static PyObject*
 get_virtual_mem(PyObject* self, PyObject* args)
 {
-    unsigned int   total, active, inactive, wired, cached, free, buffers;
+    unsigned int   total, active, inactive, wired, cached, free;
     size_t        size = sizeof(total);
    struct vmtotal vm;
    int            mib[] = {CTL_VM, VM_METER};
    long           pagesize = getpagesize();
+#if __FreeBSD_version > 702101
+    long buffers;
+#else
+    int buffers;
+#endif

     if (sysctlbyname("vm.stats.vm.v_page_count", &total, &size, NULL, 0))
         goto error;
@@ -602,7 +607,7 @@
         goto error;
     if (sysctlbyname("vm.stats.vm.v_free_count", &free, &size, NULL, 0))
         goto error;
-    if (sysctlbyname("vfs.bufspace", &buffers, &size, NULL, 0))
+    if (sysctlbyname("vfs.bufspace", &buffers, sizeof(buffers), NULL, 0))
         goto error;

     size = sizeof(vm);

giampaolo · 2014-05-23T15:04:08Z

From g.rodola on January 28, 2013 05:31:48

Sorry, there's a typo.
Here's the correct patch (not tested):


Index: psutil/_psutil_bsd.c
===================================================================
--- psutil/_psutil_bsd.c    (revisione 1560)
+++ psutil/_psutil_bsd.c    (copia locale)
@@ -584,11 +584,16 @@
 static PyObject*
 get_virtual_mem(PyObject* self, PyObject* args)
 {
-    unsigned int   total, active, inactive, wired, cached, free, buffers;
+    unsigned int   total, active, inactive, wired, cached, free;
     size_t        size = sizeof(total);
    struct vmtotal vm;
    int            mib[] = {CTL_VM, VM_METER};
    long           pagesize = getpagesize();
+#if __FreeBSD_version > 702101
+    long buffers;
+#else
+    int buffers;
+#endif
+size_t buffers_size = sizeof(buffers);

     if (sysctlbyname("vm.stats.vm.v_page_count", &total, &size, NULL, 0))
         goto error;
@@ -602,7 +607,7 @@
         goto error;
     if (sysctlbyname("vm.stats.vm.v_free_count", &free, &size, NULL, 0))
         goto error;
-    if (sysctlbyname("vfs.bufspace", &buffers, &size, NULL, 0))
+    if (sysctlbyname("vfs.bufspace", &buffers, &buffers_size, NULL, 0))
         goto error;

     size = sizeof(vm);

giampaolo · 2014-05-23T15:04:09Z

From [email protected] on February 28, 2013 01:21:51

Hi again, guys. I was unable to reproduce this issue today.

[sunrize@PRWC013 ~]$ uname -a
FreeBSD PRWC013.local 9.0-RELEASE-p3 FreeBSD 9.0-RELEASE-p3 #0: Tue Jun 12 
02:52:29 UTC 2012     
[email protected]:/usr/obj/usr/src/sys/GENERIC  amd64
[sunrize@PRWC013 ~]$ source psutils_test/bin/activate
(psutils_test)[sunrize@PRWC013 ~]$ pip install psutil
Downloading/unpacking psutil
  Downloading psutil-0.6.1.tar.gz (138kB): 138kB downloaded
  Running setup.py egg_info for package psutil

Installing collected packages: psutil
  Running setup.py install for psutil
    building '_psutil_bsd' extension
    cc -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -O2 -pipe 
-fno-strict-aliasing -fPIC -I/usr/local/include/python2.7 -c 
psutil/_psutil_bsd.c -o 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/_psutil_bsd.o
    cc -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -O2 -pipe 
-fno-strict-aliasing -fPIC -I/usr/local/include/python2.7 -c 
psutil/_psutil_common.c -o 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/_psutil_common.o
    cc -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -O2 -pipe 
-fno-strict-aliasing -fPIC -I/usr/local/include/python2.7 -c 
psutil/arch/bsd/process_info.c -o 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/arch/bsd/process_info.o
    cc -shared -pthread 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/_psutil_bsd.o 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/_psutil_common.o 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/arch/bsd/process_info.o 
-ldevstat -o build/lib.freebsd-9.0-RELEASE-p3-amd64-2.7/_psutil_bsd.so
    building '_psutil_posix' extension
    cc -fno-strict-aliasing -O2 -pipe -fno-strict-aliasing -DNDEBUG -O2 -pipe 
-fno-strict-aliasing -fPIC -I/usr/local/include/python2.7 -c 
psutil/_psutil_posix.c -o 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/_psutil_posix.o
    cc -shared -pthread 
build/temp.freebsd-9.0-RELEASE-p3-amd64-2.7/psutil/_psutil_posix.o -o 
build/lib.freebsd-9.0-RELEASE-p3-amd64-2.7/_psutil_posix.so

Successfully installed psutil
Cleaning up...
(psutils_test)[sunrize@PRWC013 ~]$ python
Python 2.7.3 (default, Oct 24 2012, 15:09:17) 
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.__version__
'0.6.1'
>>>

giampaolo · 2014-05-23T15:04:10Z

From [email protected] on February 28, 2013 01:26:42

Should i test this patch anyway?

giampaolo · 2014-05-23T15:04:10Z

From g.rodola on February 28, 2013 01:44:59

> I was unable to reproduce this issue today.

Damn. Is it the exact same system you were using in the first place? Is 
something changed? Point is if we're not able to reproduce the original issue 
there's no point in committing the patch.

giampaolo · 2014-05-23T15:04:11Z

From [email protected] on February 28, 2013 01:54:52

Yes, everything was the same: same os, same psutil and python version. But it 
was not the same machine. Ok, i'll try to figure out, whether that server is 
available now.

giampaolo · 2014-05-23T15:04:12Z

From g.rodola on March 05, 2013 13:58:41

This is now mitigated by revision d6cf2f8ba8c0 which makes psutil raise an 
exception later than sooner, so that at least we don't crash at import time.
Instead we'll get a TypeError on get_memory_percent().
Did you get the chance to put hands on that machine?

giampaolo · 2014-05-23T15:04:12Z

From [email protected] on April 13, 2013 04:55:50

I just experienced this, and so did another user.

I've committed the patch to the FreeBSD ports tree, because it works well. 
http://www.freebsd.org/cgi/query-pr.cgi?pr=172803 Chris

giampaolo · 2014-05-23T15:04:13Z

From g.rodola on April 13, 2013 05:29:04

Oh nice! Can you confirm the attached patch fixes the issue?

Attachment: 325.patch

giampaolo · 2014-05-23T15:04:14Z

From g.rodola on May 02, 2013 07:45:50

I cannot reproduce the issue but it seems I'm not going to receive any 
confirmation about the effectiveness of the patch any time soon either, so 
given this is a high priority issue I just committed the patch in revision 
9b6e780ea6b5 .

I'm going to consider this fixed for now.

Status: FixedInHG
Labels: Milestone-0.7.1

giampaolo · 2014-05-23T15:04:14Z

From [email protected] on May 02, 2013 12:01:32

Hi, I'm using psutil 0.7.0 on FreeBSD 8.2 (amd64) and Python 2.7.1. 
Getting same problem on import:

>>> import psutil
python2.7/site-packages/psutil-0.7.0-py2.7-freebsd-8.2-RELEASE-amd64.egg/psutil/_psbsd.py:38:
 RuntimeWarning: couldn't determine platform's TOTAL_PHYMEM
  warnings.warn("couldn't determine platform's TOTAL_PHYMEM", RuntimeWarning)

Most things work though (psutil.swap_memory() for example). But not 
psutil.virtual_memory():

>>> psutil.swap_memory()
swap(total=34359607296, used=0, free=34359607296, percent=0.0, sin=0, 
sout=328114176)

>>> psutil.virtual_memory()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File 
"/AppDocs/local/lib/python2.7/site-packages/psutil-0.7.0-py2.7-freebsd-8.2-RELEASE-amd64.egg/psutil/__init__.py",
 line 1154, in virtual_memory
    return _psplatform.virtual_memory()
  File 
"/AppDocs/local/lib/python2.7/site-packages/psutil-0.7.0-py2.7-freebsd-8.2-RELEASE-amd64.egg/psutil/_psbsd.py",
 line 66, in virtual_memory
    mem =  _psutil_bsd.get_virtual_mem()
SystemError: error return without exception set

giampaolo · 2014-05-23T15:04:15Z

From g.rodola on May 03, 2013 03:15:56

Please try latest repository version and confirm it's fixed:

hg clone https://[email protected]/p/psutil/ 
cd psutil
python setup.py install
python -c "import psutil; psutil.virtual_memory()"

giampaolo · 2014-05-23T15:04:15Z

From [email protected] on May 03, 2013 06:48:39

thanks, it does work. 
Let me know if you ever need help with FreeBSD 8 and Python2.7, I'll be glad to test.

giampaolo · 2014-05-23T15:04:16Z

From g.rodola on May 03, 2013 08:43:00

Status: Fixed

giampaolo closed this as completed May 23, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not import psutil on FreeBSD 9 #325

Could not import psutil on FreeBSD 9 #325

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

Could not import psutil on FreeBSD 9 #325

Could not import psutil on FreeBSD 9 #325

Comments

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014