How to Diagnose High Context Switches on Linux with vmstat and pidstat

When performance problems arise due to frequent context switches, understanding and monitoring these switches is essential. This article explains the concept of system context switches, shows how to use vmstat and pidstat for observation, and walks through a concrete example that generates heavy switching.

What is a system context switch

A system context switch is the process by which the operating system saves the state of one process or thread and restores the state of another, allowing multiple programs to share CPU time. It is a core kernel responsibility that ensures fair resource distribution and overall system efficiency.

Monitoring context switches with vmstat

In Linux, vmstat 1 prints performance statistics every second, including the number of context switches (cs). The output columns are:

cs : total context switches (both voluntary and non‑voluntary)

in : total interrupts since boot

r : length of the run queue (processes ready to run)

b : length of the blocked queue (processes waiting for resources)

vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 1956380  62880 877460    0    0   3692   312  572 1251 28 50 21  1  0
 ... (subsequent lines omitted for brevity)

Example: stress‑ng generates massive context switches

Running stress‑ng -i 100 to simulate I/O load while monitoring with vmstat 1 shows the cs column rising from around 500 to over 20 000. The following screenshot illustrates the spike:

When a high number of context switches is confirmed, the next step is to identify the responsible process using pidstat.

Drilling down with pidstat

Execute pidstat -w 3 to display per‑process context‑switch statistics every three seconds. Sample output:

pidstat -w 3
12:04:17 UID   PID   cswch/s nvcswch/s Command
12:04:20   0    14    1.33      0.00 ksoftirqd/0
12:04:20   0    15   29.33      0.00 rcu_preempt
... (many stress‑ng processes omitted) ...
12:04:20   0  7475    6.33      6.33 stress‑ng
12:04:20   0  7476    6.33      7.00 stress‑ng

The output reveals that the stress‑ng processes dominate the context‑switch count, with a notable kernel thread kworker/u256:29-flush-8:0 contributing 707 switches.

Understanding the kworker/u256:29-flush-8:0 thread

This kernel worker thread handles I/O flush operations, writing cached data to storage to maintain consistency and durability. Its name breaks down as follows:

kworker : generic kernel worker thread prefix

/u256 : identifier for the worker pool

29 : thread number within the pool

flush-8:0 : indicates a flush operation, possibly tied to a specific device

Such threads run in the background to ensure data integrity during I/O activity, which explains why the stress‑ng I/O load triggers many switches.

Voluntary vs. non‑voluntary context switches

pidstat

reports two metrics: cswch/s (voluntary) and nvcswch/s (non‑voluntary). Their meanings are:

Voluntary context switches : initiated by the process itself, e.g., when it yields the CPU or waits for I/O.

Non‑voluntary context switches : forced by the scheduler, e.g., when a time slice expires or the process blocks for a resource.

Understanding both types helps diagnose whether the workload is cooperating with the scheduler or being pre‑empted due to resource contention.

Conclusion

System context switches are a fundamental aspect of multitasking operating systems. By using vmstat to observe overall switch rates and pidstat to pinpoint offending processes or kernel threads, administrators can identify performance bottlenecks and take targeted actions to improve system efficiency.