Why Page Faults Kill Performance and How to Measure Them on Linux

When a service shows high CPU kernel usage, persistent disk I/O wait, and memory constantly below the working set, frequent alerts often indicate excessive page faults.

What is a page fault?

A page fault is a hardware exception triggered when the CPU accesses a virtual memory page that is not currently mapped to physical memory. The MMU detects an invalid page‑table entry, the kernel allocates a physical page, loads data from swap or a file, updates the page table, and resumes execution.

How to quantify page faults

On Linux you can use several tools: perf stat -e page-faults – counts page faults for a specific process. Example output shows 44,331 page faults for PID 6770 over 13.27 seconds. vmstat – monitors system‑wide page‑fault rate (pgfault/s) to assess overall performance. ftrace – kernel tracing tool for deep analysis of page‑fault handling, function calls, and stack traces.

Combining these tools gives a complete view of fault frequency, distribution, and impact.

Why frequent page faults degrade performance

Each fault incurs a context switch and interrupt handling, raising CPU usage; swapping data from disk adds hundreds of milliseconds of latency; and TLB misses force repeated page‑table walks, creating a vicious cycle that especially hurts memory‑intensive workloads.

Causes and solutions

Insufficient memory (major page fault) : Physical RAM cannot hold the program’s working set, causing pages to be swapped out and later swapped in. The straightforward fix is to add more RAM.

Enough memory but poor access patterns (minor page fault) : Random large‑range accesses, pointer‑chasing structures, or lack of huge pages lead to frequent TLB misses and minor faults. Optimise data locality, use huge pages, and reduce cache‑line bouncing.

System‑level policies can also trigger faults even with ample memory, e.g., aggressive swappiness causing idle pages to be swapped out. Mitigation steps:

Lower the swappiness value (e.g., to 10) to reduce unnecessary swapping.

Lock critical pages with mlock() to prevent them from being swapped out (useful for real‑time workloads).

Disable swap entirely when memory is abundant using swapoff -a, noting the risk of OOM kills if memory runs out.