Understanding and Using Linux Cgroups for Container Resource Management

When developing software, monitoring the released application is essential for extending its lifecycle, and container technology makes system customization and packaging easier; cgroups provide the underlying mechanism for container resource monitoring and limitation.

Cgroups (control groups) are a Linux kernel feature that can limit, record, and isolate the physical resources used by a group of processes, forming the foundation for container virtualization tools such as Docker and LXC.

Cgroups Functions

Resource limiting – e.g., setting memory usage limits that trigger OOM when exceeded.

Prioritization – allocating CPU time slices and I/O bandwidth to control process priority.

Accounting – tracking CPU time, memory usage, etc., useful for billing.

Process control – suspending or resuming tasks.

Cgroups Components

Cgroups consist of task, cgroup, subsystem, and hierarchy. A task is a Linux process; a cgroup groups tasks and attaches one or more subsystems; a subsystem is a resource controller (e.g., cpu, memory); a hierarchy arranges cgroups in a tree.

Component Relationships

1. A hierarchy can attach multiple subsystems.

2. A subsystem can attach to only one hierarchy.

3. When a new hierarchy is created, all existing tasks belong to its root cgroup; a task can belong to only one cgroup per hierarchy but may belong to multiple hierarchies.

4. Child tasks inherit the parent’s cgroup but can be moved independently.

Using Cgroups

Creating a hierarchy and attaching subsystems can be done with shell commands (e.g., mkdir to create a cgroup directory, mount -t cgroup -o cpu,memory none /sys/fs/cgroup/myhier), or using libcgroup tools on CentOS 6.

Setting parameters, such as limiting a cgroup to CPUs 0‑1, is done by echoing values into control files (e.g., echo 0-1 > cpuset.cpus).

Moving a process into a cgroup involves writing its PID to the cgroup’s tasks file.

Subsystem Overview

blkio – I/O bandwidth control.

cpu – CPU usage limiting.

cpuacct – CPU usage accounting.

cpuset – Assign specific CPUs and memory nodes.

memory – Memory usage limiting and accounting.

devices – Device access control.

net_cls – Tag network packets for traffic control.

freezer – Suspend or resume tasks.

ns – Assign different namespaces.

Containers and Cgroups

Both Docker run and Kubernetes create containers whose resource limits are enforced via cgroups.

Docker Example

Creating containers with different --cpu-shares values results in separate cgroup directories under /sys/fs/cgroup/cpu/docker/, and the cpu.shares files reflect the configured shares.

Memory limits can be set similarly; exceeding the limit causes the container to be killed.

Kubernetes Example

Kubernetes places container cgroups under /sys/fs/cgroup/cpu/kubepods/; inspecting a pod’s cgroup shows cpu.shares and memory.limit_in_bytes values derived from the pod’s resource specifications.

These examples illustrate how cgroups provide fine‑grained control over container resources, making them indispensable for modern container orchestration and system administration.