How to Diagnose and Fix High CPU Context Switches on Linux

Understanding CPU Context Switches

CPU context switches are essential for Linux to switch between processes, threads, and interrupt handlers. Excessive switches consume CPU time to save and restore registers, program counters, kernel stacks, and virtual memory, leading to noticeable performance degradation.

Checking Context Switches with vmstat

The vmstat utility can display overall system metrics, including context switches. Running vmstat 5 (5‑second interval) produces output where: cs – number of context switches per second. in – interrupts per second. r – length of the run‑queue (processes ready to run). b – processes blocked in uninterruptible sleep.

Example output (truncated):

In this snapshot the system shows 33 context switches, 25 interrupts, and a run‑queue length of 0, indicating an idle system.

Detailed Process Analysis with pidstat

While vmstat gives a system‑wide view, pidstat -w reveals per‑process switch counts. Important columns are: cswch/s – voluntary context switches per second. nvcswch/s – involuntary (forced) context switches per second.

Voluntary switches occur when a process waits for resources (e.g., I/O), whereas involuntary switches happen when a time slice expires and the scheduler preempts the process.

# Output interval is 5
$ pidstat -w 5
Linux 4.15.0 (ubuntu)  09/23/18  _x86_64_  (2 CPU)
08:18:26      UID       PID   cswch/s nvcswch/s  Command
08:18:31        0         1      0.20      0.00  systemd
08:18:31        0         8      5.40      0.00  rcu_sched
...

Case Study: Simulating Load with sysbench

Using sysbench (https://github.com/akopytov/sysbench) we generate a multithreaded workload to stress the scheduler. First, capture a baseline with vmstat (≈35 cs, 19 in, 0 r, 0 b). Then run:

$ sysbench --threads=10 --max-time=300 threads run

After the load starts, vmstat shows a sharp increase to 1.39 million context switches, a run‑queue length of 8, CPU usage at 100 % (84 % kernel), and interrupts rising to ~10 000 /s.

Analysis shows the ready‑queue is overloaded, causing many involuntary switches, while the high interrupt count points to additional bottlenecks.

Analyzing Interrupt Sources

Inspect /proc/interrupts (e.g., watch -d cat /proc/interrupts) to identify which interrupt types dominate. In the example, the RES (rescheduling) interrupt spikes, confirming that excessive task scheduling is the root cause.

# -d: Highlight the change area
$ watch -d cat /proc/interrupts
           CPU0       CPU1
... 
RES:    2450431    5279697   Rescheduling interrupts
...

When Are Switch Rates Normal?

Typical idle systems show a few hundred to a few thousand switches per second. Values consistently above 10 000 /s, or a rapid increase, usually indicate a performance problem such as CPU contention, I/O saturation, or interrupt storms.

Conclusion

By correlating vmstat, pidstat, and /proc/interrupts data you can pinpoint whether high context‑switch rates stem from voluntary waits, involuntary preemptions, or interrupt overload, and then take targeted actions (e.g., tuning scheduler, reducing I/O pressure, or fixing misbehaving drivers).