Mastering CPU Performance: Monitoring and Tuning Techniques for Linux

Performance optimization is a common topic; typical issues include slow page response, interface timeout, high server load, low concurrency, and frequent database deadlocks. In today’s fast‑paced internet development, increasing traffic and code bloat bring many performance problems.

At the system level, three factors affect application performance: CPU, memory, and I/O. This article focuses on CPU performance monitoring and tuning.

CPU Performance Monitoring

When a program becomes slow, first use commands such as top, vmstat, and ps to check CPU usage and determine whether a CPU bottleneck exists.

The us (user space) percentage is the key indicator. When us approaches or exceeds 100%, the CPU is busy. Common causes of CPU busy include infinite loops, heavy regular‑expression matching, intensive calculations, frequent GC, and frequent thread context switches.

top command

For multi‑core CPUs, top shows the combined usage. Press 1 inside top to display per‑core usage.

us – user‑process CPU usage sy – kernel‑process CPU usage ni – nice‑adjusted tasks id – idle CPU wa – I/O wait hi – hardware interrupt si – software interrupt st – stolen time (virtual CPU waiting for real CPU)

vmstat

vmstat

provides metrics such as:

in – CPU interrupts per second cs – context switches per second (lower is better) us – user CPU time sy – system CPU time (high value may indicate frequent I/O) id – idle CPU time (id + us + sy ≈ 100%) wt – I/O wait time

gstat -gcutil

For Java processes with high CPU usage, gstat -gcutil shows whether the JVM is performing frequent garbage collection.

S0, S1 – Survivor space usage E – Eden space usage O – Old space usage P – Perm space usage YGC/YGCT – Young GC count and time FGC/FGCT – Full GC count and time GCT – Total GC time

Problem Analysis

After locating the issue with the above commands, analyze the root cause. CPU bottlenecks appear as user‑mode or kernel‑mode bottlenecks. A normal user‑to‑system CPU time ratio is 3:1 to 4:1; higher ratios indicate a problem.

High us

When us is high, the application consumes most CPU. For Java, identify the thread consuming CPU, check GC frequency with gstat -gcutil, and use profiling methods described in “What the CPU is doing when it spikes”.

High sy

When sy is high, use vmstat to examine context switches. Excessive thread switching often results from many threads blocked on locks or I/O. Dump Java thread stacks with kill -3 <pid> or jstack -l <pid> to find waiting or lock‑contended threads.

CPU Tuning

Adjust process priority

Use nice and renice to set priority. Syntax: nice [-n value] command. Values range from –20 (highest priority) to 19 (lowest). Negative values require root.

Limit CPU time with ulimit

ulimit

limits CPU time for the current shell and its children. Example: limit tar to 100 seconds; exceeding the limit terminates the process.

Program‑specific CPU settings

Some programs provide their own CPU tuning. For example, Nginx can bind worker processes to specific CPU cores using a mask (e.g., 0001 0010 0100 1000 for four cores), achieving balanced CPU usage.