Implement process CPU / memory / files / ... limits on Windows

[giampaolo]

Existing resource limits capabilities

Right now it is possible to limit the resources of a certain process in many ways on both UNIX and Windows:

• CPU priority (Process.nice())
• IO priority (Process.ionice())
• CPU affinity (Process.cpu_affinity())

Linux (UPDATE: and FreeBSD) has a fourth option, Process.rlimit(), see here. It appears the same can be done on (recent verions of) Windows, see python-ideas post. Here's some findings.

CPU

In here it is suggested to use NtSetInformationProcess in order to set CPU percent limit (!) but it's an undocumented call. A modern alternative (Windows 8.1 +) is SetInformationJobObject, which appears to be the key to all resource limit functionalities.
E.g. SetInformationJobObject + JobObjectCpuRateControlInformation lets you set a CPU percent limit, which is quite impressive.
Here is a C++ cmdline tool using it. This is important as it shows it accepts a PID, meaning this can be done on a per-process basis and fit into the psutil.Process(pid) model.

Memory

Looks like JOBOBJECT_EXTENDED_LIMIT_INFORMATION lets you a generic virtual memory limit for the process. JOBOBJECT_BASIC_LIMIT_INFORMATION lets you set min/max working set size (Process.memory_info().rss in psutil terms). See SetProcessWorkingSetSize.

Network

JOBOBJECT_NET_RATE_CONTROL_INFORMATION limits the outgoing network bandwidth (max bytes - what about incoming?).

Disk IO

JOBOBJECT_LIMIT_VIOLATION_INFORMATION limits bytes read and written from/to disk.

Others

There is a command line tool by daemontools which also gives an idea on what capabilities are:
https://cr.yp.to/daemontools/softlimit.html
It shows shows other interesting things can be done amongst which:

• limit number of open files
• limit number of processes per user

API

I suppose that we can probably reuse Linux's Process.rlimit method and its signature, which is rlimit(resource, limits=None) and serves both "get" and "set" usage. Hypothetical idea:

>>> p = psutil.Process()
>>> # limit process memory to 50 MB 
>>> p.rlimit(psutil.WIN_RLIMIT_MEMORY, 50 * 1024 * 1024)  # set
>>> p.rlimit(psutil.WIN_RLIMIT_MEMORY)  # get
52428800
>>>
>>> # limit CPU usage to 50%
>>> p.win_rusage(psutil.WIN_RLIMIT_CPU_PERCENT, 50)  # set

When more than one argument is supposed to be passed we can use a tuple as in (still hypothetical):

# limit disk read/writes to 50.000 bytes
>>> p.rlimit(psutil.WIN_RLIMIT_DISK_IO, (50000, 50000))

[dbwiddis]

Not sure if it's relevant to this issue (Windows only?) or related to a different one as this is a Linux-based comment, but there are additional limitations in cgroup on Linux which are used by Docker containers. The relevant file is /sys/fs/cgroup/memory/memory.limit_in_bytes.