Mastering Linux Cgroups: The Core of Container Resource Management

Cgroups (control groups) is a Linux kernel mechanism that can limit, record, and isolate the physical resources (CPU, memory, I/O, etc.) used by groups of processes, providing a framework for resource management and a basic guarantee for container virtualization such as Docker.

1. Why Understand Cgroups

Since the rise of Docker (2013) and Kubernetes (2014), container technology has become a mainstream cloud‑native foundation, relying on Linux Cgroups and Namespace for resource limitation and isolation.

Cgroups mainly manages resource allocation and limits; Namespace mainly encapsulates and isolates resources, giving processes their own global resources.

2. Cgroups Overview

Cgroups (control groups) allow the Linux kernel to restrict, record, and isolate the physical resources used by a group of processes.

Administrators can finely control allocation, ordering, denial, management, and monitoring of system resources, improving overall efficiency. Cgroups originated from Google engineers and were merged into the Linux kernel, becoming a basis for lightweight virtualization such as Linux Containers (LXC).

Unlike Namespace, which isolates resources between groups, Cgroups focuses on unified monitoring and limiting of a group’s resources.

Cgroups have v1 and v2 versions. v1 offers many features but is fragmented; v2 consolidates functionality and is production‑ready in newer kernels (e.g., 4.5), though still limited in scope.

3. What Is a Cgroup?

A Cgroup is a tree of process groups; each node (process group) can be associated with one or more subsystems (resource controllers). The tree’s hierarchy defines how resources are applied.

Cgroups consist of two main parts:

subsystem : a kernel module attached to a cgroup tree that performs specific operations (e.g., CPU, memory). It is often called a resource controller.

hierarchy : a cgroup tree where each node is a process group; a hierarchy can be linked to multiple subsystems.

Linux supports up to 12 subsystems, allowing up to 12 independent cgroup trees.

4. Why Need Cgroups?

Traditional process grouping (session, progress groups) cannot provide fine‑grained tracking of memory or I/O. Cgroups unify process grouping and enable monitoring and resource control, solving issues such as antivirus software consuming excessive resources or a single Docker container monopolizing CPU.

5. How Cgroups Are Implemented (CentOS 7)

Systemd binds cgroup hierarchies to its unit tree, creating slices, scopes, and services automatically. By default three top‑level slices are created: system.slice, user.slice, and machine.slice.

Each slice receives an equal share of CPU time when the CPU is busy. For example, user.slice can be configured to receive 100% CPU if the system is idle.

system.slice : default location for all system services.

user.slice : default location for all user sessions; each user gets a sub‑slice.

machine.slice : default location for virtual machines and containers.

CPU shares are expressed as a relative weight (default 1024). Example calculations illustrate how each slice receives roughly one‑third of CPU time.

6. Functions of Cgroups

Resource limiting (e.g., memory limits trigger OOM).

Prioritization (e.g., CPU shares).

Accounting (e.g., cpuacct records CPU time).

Isolation (e.g., ns subsystem provides separate namespaces).

Control (e.g., freezer can suspend/resume a group).

7. Cgroup Concepts and Relationships

7.1 Related Concepts

Task : a single process in a cgroup.

Control group : a set of processes grouped by a standard; resources are managed per control group.

Hierarchy : the tree structure of control groups; child groups inherit attributes from parents.

Subsystem : a resource controller (e.g., cpu, memory) that must be attached to a hierarchy to take effect.

7.2 Inter‑relationships

When a new hierarchy is created, all tasks belong to the root cgroup of that hierarchy.

A subsystem can be attached to only one hierarchy.

A hierarchy can have multiple subsystems.

A task may belong to multiple cgroups, but only one per hierarchy.

Child processes inherit their parent’s cgroup unless moved manually.

8. Subsystem Introduction

Under /sys/fs/cgroup you can find directories such as cpu, memory, etc., each representing a subsystem.

net_cls : classifies network packets for traffic control.

net_prio : sets network traffic priority.

memory : controls memory usage.

cpuset : assigns specific CPUs and memory nodes.

freezer : suspends or resumes tasks.

blkio : limits block device I/O.

cpu : controls CPU time allocation.

cpuacct : accounts CPU usage.

devices : controls device access.

8.1 How to View Supported Subsystems

#subsys_name hierarchy num_cgroups enabled
cpuset 11 1 1
cpu 3 64 1
cpuacct 3 64 1
blkio 8 64 1
memory 9 104 1
de vices 5 64 1
freezer 10 4 1
net_cls 6 1 1
perf_event 7 1 1
net_prio 6 1 1
hugetlb 4 1 1
pids 2 68 1

8.2 CPU Subsystem Details

The CPU subsystem uses five main interfaces: cpu.cfs_period_us: length of a scheduling period (µs). cpu.cfs_quota_us: total CPU time allowed within the period. cpu.shares: relative weight for proportional sharing. cpu.rt_period_us: period for real‑time bandwidth. cpu.rt_runtime_us: max continuous real‑time CPU usage within the period.

For example, setting cpu.cfs_period_us=10000 and cpu.cfs_quota_us=2000 limits a cgroup to 2 CPU cores.

8.3 Installing Cgroups on CentOS

# Install if not present
yum install libcgroup

# Check status and start service
service cgconfig status
service cgconfig start

# Verify installation
grep cgroup /proc/filesystems

8.4 Viewing Service Cgroup

systemctl status [pid] | grep CGroup
cat /proc/[pid]/cgroup
cd /sys/fs/ && find * -name "*.procs" -exec grep [pid] {} /dev/null \; 2>/dev/null

Quick Bash script to find a process’s cgroup by name:

#!/bin/bash
THISPID=`ps -eo pid,comm | grep $1 | awk '{print $1}'`
cat /proc/$THISPID/cgroup

9. How to Use Cgroups

9.1 Set Cgroup via systemctl

Use systemctl set-property with tab‑completion to adjust properties such as CPU, Memory, and Block I/O.

$ systemctl set-property user-1000.slice CPUQuota=20%

9.2 Limit CPU Usage

Configure cpu.cfs_period_us (period) and cpu.cfs_quota_us (allowed time) to enforce strict CPU caps.

# Example: limit to 20% of a CPU
cpu.cfs_period_us=100000
cpu.cfs_quota_us=20000

9.3 Configure via /etc/cgconfig.conf

mount {
  cpuset = /cgroup/cpuset;
  cpu = /cgroup/cpu;
  cpuacct = /cgroup/cpuacct;
  memory = /cgroup/memory;
  devices = /cgroup/devices;
  freezer = /cgroup/freezer;
  net_cls = /cgroup/net_cls;
  blkio = /cgroup/blkio;
}

Persist settings in /etc/cgconfig.conf and /etc/cgrules.conf for reboot‑stable control.

10. Viewing Cgroups

10.1 Using systemd

systemd-cgls

displays the full cgroup hierarchy.

$ systemd-cgls --no-page
├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
├─user.slice
│  ├─user-1000.slice
│  │  └─session-11.scope
│  │     ├─9507 sshd: tom [priv]
│  │     ├─9509 sshd: tom@pts/3
│  │     └─9510 -bash
│  └─user-0.slice
│     └─session-1.scope
│        ├─6239 sshd: root@pts/0
│        ├─6241 -zsh
│        └─11537 systemd-cgls --no-page
└─system.slice
   ├─rsyslog.service
   │  └─5831 /usr/sbin/rsyslogd -n
   ├─sshd.service
   │  └─5828 /usr/sbin/sshd -D
   └─...

systemd-cgtop

provides dynamic resource usage similar to top.

$ systemd-cgtop
Path               Tasks %CPU Memory Input/s Output/s
/                  161   1.2 161.0M -       -
/system.slice      -     0.1 -       -       -
...

10.2 Using /proc

Inspect a process’s cgroup membership via /proc/[pid]/cgroup:

$ cat /proc/777/cgroup
11:cpuset:/
10:freezer:/
9:memory:/system.slice/cron.service
8:blkio:/system.slice/cron.service
7:perf_event:/
6:net_cls,net_prio:/
5:devices:/system.slice/cron.service
4:hugetlb:/
3:cpu,cpuacct:/system.slice/cron.service
2:pids:/system.slice/cron.service
1:name=systemd:/system.slice/cron.service

10.3 Using /sys

Check CPU limits directly:

$ cat /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice/cpu.cfs_period_us
100000
$ cat /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice/cpu.cfs_quota_us
20000

This shows that user tom can use 20 ms of CPU time in each 100 ms period, regardless of overall CPU load.