Skip to content

Latest commit

 

History

History
176 lines (160 loc) · 6.18 KB

cgroups.md

File metadata and controls

176 lines (160 loc) · 6.18 KB
Raw

cgroups

Contents

1. What's cgroups?
2. How to use cgroups?
3. References

1. What's cgroups?

cgroups stands for Control Groups, it's a feature of Linux kernel that allocates and isolates resources: CPU, memory, disk I/O and networking of one or more processes.
The below figure shows the configurations of cgroups in Linux kernel:

cgroups

The design of cgroup aims to provide a unified interface to manage processes or OS-level virtualization, including Linux Containers (LXC):

  • Resource limiting: a group can be configured not to exceed a specified memory limit or use more than the desired amount of processors or be limited to specific peripheral devices.
  • Prioritization: one or more groups may be configured to utilize fewer or more CPUs or disk I/O throughput.
  • Accounting: a group's resource usage is monitored and measured.
  • Control: groups of processes can be frozen or stopped and restarted.

2. How to use cgroups?

The user can access and manage cgroups directly and indirectly (with LXC, libvirt or Docker).
Install the necessary packages:

$ sudo apt-get install libcgroup1 cgroup-tools

Now, the enabled cgroups can be seen via proc filesystem or sysfs:

$ cat /proc/cgroups

#subsys_name    hierarchy       num_cgroups     enabled
cpuset  9       2       1
cpu     4       134     1
cpuacct 4       134     1
blkio   7       134     1
memory  5       163     1
devices 11      134     1
freezer 2       2       1
net_cls 3       2       1
perf_event      10      2       1
net_prio        3       2       1
hugetlb 8       2       1
pids    6       136     1

$ ls -l /sys/fs/cgroup/

total 0
dr-xr-xr-x 6 root root  0 Nov 13 00:55 blkio
drwxr-xr-x 2 root root 60 Nov 13 01:00 cgmanager
lrwxrwxrwx 1 root root 11 Nov 13 00:55 cpu -> cpu,cpuacct
lrwxrwxrwx 1 root root 11 Nov 13 00:55 cpuacct -> cpu,cpuacct
dr-xr-xr-x 6 root root  0 Nov 13 00:55 cpu,cpuacct
dr-xr-xr-x 3 root root  0 Nov 13 00:55 cpuset
dr-xr-xr-x 6 root root  0 Nov 13 00:55 devices
dr-xr-xr-x 3 root root  0 Nov 13 00:55 freezer
dr-xr-xr-x 3 root root  0 Nov 13 00:55 hugetlb
dr-xr-xr-x 6 root root  0 Nov 13 00:55 memory
lrwxrwxrwx 1 root root 16 Nov 13 00:55 net_cls -> net_cls,net_prio
dr-xr-xr-x 3 root root  0 Nov 13 00:55 net_cls,net_prio
lrwxrwxrwx 1 root root 16 Nov 13 00:55 net_prio -> net_cls,net_prio
dr-xr-xr-x 3 root root  0 Nov 13 00:55 perf_event
dr-xr-xr-x 6 root root  0 Nov 13 00:55 pids
dr-xr-xr-x 6 root root  0 Nov 13 00:55 systemd

cgroups can be configured directly via the sysfs. For example, let's create a small bash script named test_cgroups.sh for demonstration:

#!/bin/bash

while :
do
    echo "Print line" > /dev/tty
    sleep 5
done

Run above script:

$ chmod +x test_cgroups.sh
$ ./test_cgroups.sh
Print line
Print line
Print line
...
...

Change directory to /sys/fs/cgroup/devices where devices represents kind of resources that allows or denies access to devices by tasks in a cgroup:

$ cd sys/fs/cgroup/devices

Then, create a directory cgroups_test_group:

# mkdir cgroups_test_group

After creation of the cgroups_test_group directory, the following files will be generated:

$ ls -l /sys/fs/cgroup/devices/cgroups_test_group

total 0
-rw-r--r-- 1 root root 0 Nov 16 02:05 cgroup.clone_children
-rw-r--r-- 1 root root 0 Nov 16 02:05 cgroup.procs
--w------- 1 root root 0 Nov 16 02:05 devices.allow
--w------- 1 root root 0 Nov 16 02:05 devices.deny
-r--r--r-- 1 root root 0 Nov 16 02:05 devices.list
-rw-r--r-- 1 root root 0 Nov 16 02:05 notify_on_release
-rw-r--r-- 1 root root 0 Nov 16 02:05 tasks

The tasks file contains PIDs (Process ID) of processes which will be attached to the cgroups_test_group, the devices.deny file contains list of denied devices. By default, a newly created group has no any limits for devices access. In order to forbid a device (in this case, it's /dev/tty), the devices.deny file should be modified:

# echo "c 5:0 w" > devices.deny

In the above command, the c indicates that /dev/tty is a character device, 5:0 is major and minor numbers of the device. The last w is write permission, so the above command forbids tasks to write to the /dev/tty.

$ ls -l /dev/tty

crw-rw-rw- 1 root tty 5, 0 Nov 18 17:02 /dev/tty

After that, re-run the script test_cgroups.sh:

$ ./test_cgroups.sh
Print line
Print line
Print line
...
...

then add the PID of this process to the tasks file:

# echo $(pidof -x test_cgroups.sh) > /sys/fs/cgroup/devices/cgroups_test_group/tasks

The result will be as expected:

$ ./test_cgroups.sh
Print line
Print line
Print line
./test_cgroups.sh: line 5: /dev/tty: Operation not permitted
./test_cgroups.sh: line 5: /dev/tty: Operation not permitted
...
...

An other example when running docker container

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS
98225055fa39        ubuntu              "/bin/bash"         47 seconds ago      Up 30 seconds

$ cat /sys/fs/cgroup/device/docker/98225055fa394b388e988b067b77dda61e53027ee944e4e0fd7887e19cdcf341/tasks
13556

During starting up of a docker container, docker creates a cgroup for processes in this container:

$ docker run -it ubuntu
$ top
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
   1  root      20   0   18508   1848   1444 S   0.0  0.0   0:00.01 bash
   12 root      20   0   36628   1924   1420 R   0.0  0.0   0:00.01 top

Now, the cgroup of above process will be seen on host machine:

$ systemd-cgls
Control group /:
-.slice
├─1429 /sbin/cgmanager -m name=systemd
├─docker
│ └─98225055fa394b388e988b067b77dda61e53027ee944e4e0fd7887e19cdcf341
│   └─13556 /bin/bash

3. References

[1] Linux Control Groups