Master Linux Resource Limits: ulimit, cgroups, and Docker Compose Guide

When running many tasks in production, a single process may consume excessive resources, causing system hangs and affecting service availability.

1. Environment Preparation and Execution Location

System : AlmaLinux 8.x

Execution Location : All commands run in Shell (bash) as normal user or root via SSH to target servers (master, node1, node2).

Deployment Method : For containerized setups, place docker-compose.yml in project root and run docker-compose up -d.

2. ulimit: User-level Resource Limits

1. Basic Concept

ulimit

is a built‑in Linux command to set per‑user process resource limits such as open files, max threads, etc. Exceeding the limit causes the system to deny allocation, preventing uncontrolled growth.

2. View Current Limits

# Show all limits
ulimit -a

In DataGrip terminal the same command can be run and the output captured as shown.

ulimit -n

3. Official Example – Limiting Open Files with ulimit

# Temporarily set max open files to 1024
ulimit -n 1024
# Verify
ulimit -n

If a process tries to open more than 1024 files, it fails with “Too many open files”.

bash: <program>: Too many open files

4. cgroups: System‑level Fine‑grained Limits

Control Groups (cgroups) provide resource limits for groups of processes, allowing detailed management of CPU, memory, block I/O, etc.

1. Install and Start

# Install cgroup tools
yum install -y libcgroup-tools
# Enable and start service
systemctl enable --now cgconfig

2. Sample Configuration (/etc/cgconfig.conf)

mount {
  cpu = /sys/fs/cgroup/cpu;
  memory = /sys/fs/cgroup/memory;
}
# Define group mydb
group mydb {
  cpu {
    cpu.shares = 512;   # CPU weight
  }
  memory {
    memory.limit_in_bytes = 1G;  # Max 1G
  }
}

3. Assign Existing Process to cgroup

# Add process with PID 1234 to mydb
cgclassify -g cpu,memory:mydb 1234

5. Docker Compose Integration Example

In container environments you can set cgroup limits via a Compose file:

version: '3.8'
services:
  db:
    image: mysql:8.0
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
    environment:
      - MYSQL_ROOT_PASSWORD=secret

Save as docker-compose.yml and run: docker-compose up -d This restricts the database instance’s resources at the container level.

6. Component Relationship Summary

ulimit (user‑level) : Quick command‑line effect; resets on reboot.

cgroups (system‑level) : Unified management of process groups; persistent configuration; usable inside containers.

Docker Compose : Leverages Docker’s underlying cgroups to isolate resources in containerized deployments.

Combined, they enable comprehensive resource control from user space to system and container layers.

For production, combine with monitoring (Prometheus + Grafana) to regularly check resource usage and avoid setting limits too low.

To make ulimit permanent, add settings to /etc/security/limits.conf for new sessions. cgroups v2 offers a more unified management approach, selectable per distribution.

Original content by IT咸鱼, please credit the public account when reproducing.