Merge remote-tracking branch 'giampaolo/master'

* giampaolo/master: (34 commits)
  pre-release updates
  try to fix appveyor failure
  refactor setup.py + fix Makefile
  fix OSX test failure
  linux / setup.py: do not print warnings
  giampaolo#675: [Linux] net_connections(); UnicodeDecodeError may occur when listing UNIX sockets
  move unicode tests so that they are executed on all platforms
  skip failing test on appveyor
  try to fix appveyor failure
  test refactoring+
  try to fix appveyor failures
  open files by using sys.getfilesystemencoding() + refactor stuff
  giampaolo#675: try to fix encoding issue
  fix python 3 compilation error
  giampaolo#659: [Linux] compilation error on Suse 10. (patch by maozguttman)
  make flake8 happy
  fix giampaolo#644: add support for CTRL_* signals on Windows
  str-encode NIC names
  always return encoded strings instead of unicode
  giampaolo#650: make cmdline() handle unicode on python 2
  ...

CREDITS

@@ -313,3 +313,15 @@ I: 634
 N: Bart van Kleef
 W: https://github.com/bkleef
 I: 664
+
+N: Steven Winfield
+W: https://github.com/stevenwinfield
+I: 672
+
+N: sk6249
+W: https://github.com/sk6249
+I: 670
+
+N: maozguttman
+W: https://github.com/maozguttman
+I: 659

HISTORY.rst

@@ -1,19 +1,41 @@
 Bug tracker at https://github.com/giampaolo/psutil/issues
 
-3.1.2 - XXXX-XX-XX
+3.2.0 - 2015-09-02
 ==================
 
 **Enhancements**
 
+- #644: [Windows] added support for CTRL_C_EVENT and CTRL_BREAK_EVENT signals
+  to use with Process.send_signal().
 - #648: CI test integration for OSX. (patch by Jeff Tang)
-- #663: net_if_addrs() now returns point-to-point addresses (for VPNs).
+- #663: [UNIX] net_if_addrs() now returns point-to-point (VPNs) addresses.
+- #655: [Windows] different issues regarding unicode handling were fixed. On
+  Python 2 all APIs returning a string will now return an encoded version of it
+  by using sys.getfilesystemencoding() codec. The APIs involved are:
+  - psutil.net_if_addrs()
+  - psutil.net_if_stats()
+  - psutil.net_io_counters()
+  - psutil.Process.cmdline()
+  - psutil.Process.name()
+  - psutil.Process.username()
+  - psutil.users()
 
 **Bug fixes**
 
 - #513: [Linux] fixed integer overflow for RLIM_INFINITY.
 - #641: [Windows] fixed many compilation warnings.  (patch by Jeff Tang)
-- #659: [Linux] compilation error on Suse 10.
+- #652: [Windows] net_if_addrs() UnicodeDecodeError in case of non-ASCII NIC
+  names.
+- #655: [Windows] net_if_stats() UnicodeDecodeError in case of non-ASCII NIC
+  names.
+- #659: [Linux] compilation error on Suse 10. (patch by maozguttman)
 - #664: [Linux] compilation error on Alpine Linux. (patch by Bart van Kleef)
+- #670: [Windows] segfgault of net_if_addrs() in case of non-ASCII NIC names.
+  (patch by sk6249)
+- #672: [Windows] compilation fails if using Windows SDK v8.0. (patch by
+  Steven Winfield)
+- #675: [Linux] net_connections(); UnicodeDecodeError may occur when listing
+  UNIX sockets.
 
 
 3.1.1 - 2015-07-15

INSTALL.rst

@@ -30,11 +30,11 @@ Compiling on Windows using Visual Studio
 ========================================
 
 In order to compile psutil on Windows you'll need Visual Studio (Mingw32 is
-no longer supported). You must have the same version of Visual Studio used to compile
-your installation of Python, that is::
+no longer supported). You must have the same version of Visual Studio used to
+compile your installation of Python, that is::
 
-* Python 2.6:  VS 2008
-* Python 2.7:  VS 2008
+* Python 2.6:  VS 2008 (download it from `here <http://www.microsoft.com/en-us/download/details.aspx?id=44266>`_)
+* Python 2.7:  VS 2008 (download it from `here <http://www.microsoft.com/en-us/download/details.aspx?id=44266>`_)
 * Python 3.3, 3.4: VS 2010 (you can download it from `MS website <http://www.visualstudio.com/downloads/download-visual-studio-vs#d-2010-express>`_)
 * Python 3.5: `VS 2015 UP <http://www.visualstudio.com/en-au/news/vs2015-preview-vs>`_
 

Makefile

@@ -35,9 +35,11 @@ build: clean
 
 # useful deps which are nice to have while developing / testing
 setup-dev-env: install-git-hooks
-	python -c "import urllib2; \
-			   r = urllib2.urlopen('https://bootstrap.pypa.io/get-pip.py'); \
-			   open('/tmp/get-pip.py', 'w').write(r.read());"
+	python -c  "import urllib2, ssl; \
+				context = ssl._create_unverified_context() if hasattr(ssl, '_create_unverified_context') else None; \
+				kw = dict(context=context) if context else {}; \
+				r = urllib2.urlopen('https://bootstrap.pypa.io/get-pip.py', **kw); \
+				open('/tmp/get-pip.py', 'w').write(r.read());"
 	$(PYTHON) /tmp/get-pip.py --user
 	rm /tmp/get-pip.py
 	$(PYTHON) -m pip install --user --upgrade pip

README.rst

@@ -350,6 +350,7 @@ http://groups.google.com/group/psutil/
 Timeline
 ========
 
+- 2015-09-02: `psutil-3.1.1.tar.gz <https://pypi.python.org/packages/source/p/psutil/psutil-3.2.0.tar.gz>`_
 - 2015-07-15: `psutil-3.1.1.tar.gz <https://pypi.python.org/packages/source/p/psutil/psutil-3.1.1.tar.gz>`_
 - 2015-07-15: `psutil-3.1.0.tar.gz <https://pypi.python.org/packages/source/p/psutil/psutil-3.1.0.tar.gz>`_
 - 2015-06-18: `psutil-3.0.1.tar.gz <https://pypi.python.org/packages/source/p/psutil/psutil-3.0.1.tar.gz>`_

docs/index.rst

@@ -457,7 +457,7 @@ Network
 
   *New in 3.0.0*
 
-  *Changed in 3.1.2:* *ptp* field was added.
+  *Changed in 3.2.0:* *ptp* field was added.
 
 .. function:: net_if_stats()
 
@@ -1183,10 +1183,14 @@ Process class
      Send a signal to process (see
      `signal module <http://docs.python.org//library/signal.html>`__
      constants) pre-emptively checking whether PID has been reused.
-     This is the same as ``os.kill(pid, sig)``.
-     On Windows only **SIGTERM** is valid and is treated as an alias for
+     On UNIX this is the same as ``os.kill(pid, sig)``.
+     On Windows only **SIGTERM**, **CTRL_C_EVENT ** and **CTRL_BREAK_EVENT**
+     signals are supported and **SIGTERM** is treated as an alias for
      :meth:`kill()`.
 
+     *Changed in 3.2.0:* support for CTRL_C_EVENT and CTRL_BREAK_EVENT signals
+     was added.
+
   .. method:: suspend()
 
      Suspend process execution with **SIGSTOP** signal pre-emptively checking

psutil/__init__.py

@@ -158,7 +158,7 @@
 ]
 __all__.extend(_psplatform.__extra__all__)
 __author__ = "Giampaolo Rodola'"
-__version__ = "3.1.2"
+__version__ = "3.2.0"
 version_info = tuple([int(num) for num in __version__.split('.')])
 AF_LINK = _psplatform.AF_LINK
 _TOTAL_PHYMEM = None
@@ -1039,8 +1039,14 @@ def send_signal(self, sig):
         else:
             if sig == signal.SIGTERM:
                 self._proc.kill()
+            # py >= 2.7
+            elif sig in (getattr(signal, "CTRL_C_EVENT", object()),
+                         getattr(signal, "CTRL_BREAK_EVENT", object())):
+                self._proc.send_signal(sig)
             else:
-                raise ValueError("only SIGTERM is supported on Windows")
+                raise ValueError(
+                    "only SIGTERM, CTRL_C_EVENT and CTRL_BREAK_EVENT signals "
+                    "are supported on Windows")
 
     @_assert_pid_not_reused
     def suspend(self):

psutil/_pslinux.py

@@ -56,7 +56,8 @@
 CLOCK_TICKS = os.sysconf("SC_CLK_TCK")
 PAGESIZE = os.sysconf("SC_PAGE_SIZE")
 BOOT_TIME = None  # set later
-DEFAULT_ENCODING = sys.getdefaultencoding()
+if PY3:
+    FS_ENCODING = sys.getfilesystemencoding()
 if enum is None:
     AF_LINK = socket.AF_PACKET
 else:
@@ -115,6 +116,16 @@ class IOPriority(enum.IntEnum):
 TimeoutExpired = None
 
 
+# --- utils
+
+def open_text(fname):
+    """On Python 3 opens a file in text mode by using fs encoding.
+    On Python 2 this is just an alias for open(name, 'rt').
+    """
+    kw = dict(encoding=FS_ENCODING) if PY3 else dict()
+    return open(fname, "rt", **kw)
+
+
 # --- named tuples
 
 def _get_cputimes_fields():
@@ -263,7 +274,7 @@ def cpu_count_logical():
         # try to parse /proc/stat as a last resort
         if num == 0:
             search = re.compile('cpu\d')
-            with open('/proc/stat', 'rt') as f:
+            with open_text('/proc/stat') as f:
                 for line in f:
                     line = line.split(' ')[0]
                     if search.match(line):
@@ -471,7 +482,7 @@ def process_inet(self, file, family, type_, inodes, filter_pid=None):
         if file.endswith('6') and not os.path.exists(file):
             # IPv6 not supported
             return
-        with open(file, 'rt') as f:
+        with open_text(file) as f:
             f.readline()  # skip the first line
             for line in f:
                 try:
@@ -504,7 +515,9 @@ def process_inet(self, file, family, type_, inodes, filter_pid=None):
 
     def process_unix(self, file, family, inodes, filter_pid=None):
         """Parse /proc/net/unix files."""
-        with open(file, 'rt') as f:
+        # see: https://github.com/giampaolo/psutil/issues/675
+        kw = dict(encoding=FS_ENCODING, errors='replace') if PY3 else dict()
+        with open(file, 'rt', **kw) as f:
             f.readline()  # skip the first line
             for line in f:
                 tokens = line.split()
@@ -575,7 +588,7 @@ def net_io_counters():
     """Return network I/O statistics for every network interface
     installed on the system as a dict of raw tuples.
     """
-    with open("/proc/net/dev", "rt") as f:
+    with open_text("/proc/net/dev") as f:
         lines = f.readlines()
     retdict = {}
     for line in lines[2:]:
@@ -626,7 +639,7 @@ def disk_io_counters():
 
     # determine partitions we want to look for
     partitions = []
-    with open("/proc/partitions", "rt") as f:
+    with open_text("/proc/partitions") as f:
         lines = f.readlines()[2:]
     for line in reversed(lines):
         _, _, _, name = line.split()
@@ -643,7 +656,7 @@ def disk_io_counters():
                 partitions.append(name)
     #
     retdict = {}
-    with open("/proc/diskstats", "rt") as f:
+    with open_text("/proc/diskstats") as f:
         lines = f.readlines()
     for line in lines:
         # http://www.mjmwired.net/kernel/Documentation/iostats.txt
@@ -669,7 +682,7 @@ def disk_io_counters():
 def disk_partitions(all=False):
     """Return mounted disk partitions as a list of namedtuples"""
     fstypes = set()
-    with open("/proc/filesystems", "r") as f:
+    with open_text("/proc/filesystems") as f:
         for line in f:
             line = line.strip()
             if not line.startswith("nodev"):
@@ -748,9 +761,7 @@ def __init__(self, pid):
 
     @wrap_exceptions
     def name(self):
-        fname = "/proc/%s/stat" % self.pid
-        kw = dict(encoding=DEFAULT_ENCODING) if PY3 else dict()
-        with open(fname, "rt", **kw) as f:
+        with open_text("/proc/%s/stat" % self.pid) as f:
             data = f.read()
         # XXX - gets changed later and probably needs refactoring
         return data[data.find('(') + 1:data.rfind(')')]
@@ -786,9 +797,7 @@ def exe(self):
 
     @wrap_exceptions
     def cmdline(self):
-        fname = "/proc/%s/cmdline" % self.pid
-        kw = dict(encoding=DEFAULT_ENCODING) if PY3 else dict()
-        with open(fname, "rt", **kw) as f:
+        with open_text("/proc/%s/cmdline" % self.pid) as f:
             data = f.read()
         if data.endswith('\x00'):
             data = data[:-1]
@@ -898,7 +907,7 @@ def memory_maps(self):
             Fields are explained in 'man proc'; here is an updated (Apr 2012)
             version: http://goo.gl/fmebo
             """
-            with open("/proc/%s/smaps" % self.pid, "rt") as f:
+            with open_text("/proc/%s/smaps" % self.pid) as f:
                 first_line = f.readline()
                 current_block = [first_line]
 
@@ -1021,7 +1030,7 @@ def threads(self):
 
     @wrap_exceptions
     def nice_get(self):
-        # with open('/proc/%s/stat' % self.pid, 'r') as f:
+        # with open_text('/proc/%s/stat' % self.pid) as f:
         #   data = f.read()
         #   return int(data.split()[18])
 

psutil/_psutil_linux.c

@@ -23,7 +23,15 @@
 #include <sys/socket.h>
 #include <linux/sockios.h>
 #include <linux/if.h>
-#include <linux/types.h>
+
+// see: https://github.com/giampaolo/psutil/issues/659
+#ifdef PSUTIL_ETHTOOL_MISSING_TYPES
+    #include <linux/types.h>
+    typedef __u64 u64;
+    typedef __u32 u32;
+    typedef __u16 u16;
+    typedef __u8 u8;
+#endif
 #include <linux/ethtool.h>
 
 #include "_psutil_linux.h"
@@ -629,13 +637,13 @@ PyMODINIT_FUNC PyInit__psutil_linux(void)
 void init_psutil_linux(void)
 #endif
 {
+    PyObject *v;
 #if PY_MAJOR_VERSION >= 3
     PyObject *module = PyModule_Create(&moduledef);
 #else
     PyObject *module = Py_InitModule("_psutil_linux", PsutilMethods);
 #endif
 
-
     PyModule_AddIntConstant(module, "version", PSUTIL_VERSION);
 #if PSUTIL_HAVE_PRLIMIT
     PyModule_AddIntConstant(module, "RLIMIT_AS", RLIMIT_AS);
@@ -650,7 +658,6 @@ void init_psutil_linux(void)
     PyModule_AddIntConstant(module, "RLIMIT_RSS", RLIMIT_RSS);
     PyModule_AddIntConstant(module, "RLIMIT_STACK", RLIMIT_STACK);
 
-    PyObject *v;
 #if defined(HAVE_LONG_LONG)
     if (sizeof(RLIM_INFINITY) > sizeof(long)) {
         v = PyLong_FromLongLong((PY_LONG_LONG) RLIM_INFINITY);