Why Linux Skips OOM at High Memory? Meet FastOOM to Stop Near‑OOM Jitters

Background

Users reported sudden spikes in CPU utilization and system load when memory usage neared 90‑95%, causing short‑term stalls that impact latency‑sensitive services. They questioned why the kernel does not trigger OOM to free memory at such high watermarks.

Kernel Memory Reclamation

Linux defines three memory watermarks: high, low, and min. When free memory falls below the low watermark, the kswapd daemon is awakened to perform asynchronous reclamation, which has little impact. If memory drops below the min watermark, the kernel blocks allocating processes and aggressively reclaims file caches and kernel structure caches, which can increase load and cause temporary blocking. If reclamation cannot satisfy the request, OOM is not immediately triggered, leading to a Near‑OOM livelock where the kernel continuously tries to reclaim while applications keep generating cache pressure, causing sustained high load and possible system hangs.

Limitations of Existing User‑Space OOM (oomd)

The systemd‑integrated oomd uses cgroup v2 and PSI (Pressure Stall Information) to predict OOM, but it is tightly coupled to these features. Many cloud environments still run cgroup v1, and PSI may be disabled due to performance overhead. Additionally, oomd only supports killing at the cgroup level, limiting flexibility.

FastOOM Solution

Alibaba Cloud introduced FastOOM , a user‑space OOM mechanism that proactively kills processes before the kernel reaches OOM. It consists of two modules:

Prediction module : collects memory‑pressure metrics from the Alibaba Cloud OS, applies statistical methods to estimate the probability of an OOM event, and determines if a node or pod is entering Near‑OOM.

Kill module : follows user‑defined policies to select and terminate specific processes.

All kill actions are reported to the console for visibility.

Case Study 1: Node‑Level FastOOM

An automotive customer experienced long‑lasting unresponsiveness on a VM when memory usage approached the total limit. Monitoring showed high CPU sys and iowait, and kswapd consuming CPU. Enabling FastOOM at the node level with a policy that protects Python processes while allowing log‑collection processes to be killed prevented the Near‑OOM state and eliminated the stalls.

Case Study 2: Pod‑Level FastOOM

In a Kubernetes cluster, a big‑data workload ran latency‑sensitive pods that occasionally exhibited long tail latency despite normal network metrics. The custom metric for container memory‑reclaim delay indicated prolonged reclamation during Near‑OOM. Configuring FastOOM at the pod level to pre‑emptively kill non‑critical processes (e.g., log collectors) removed the reclamation delay and stopped the jitter.

Summary

Linux’s conservative OOM policy can cause Near‑OOM livelocks that degrade latency‑sensitive applications. FastOOM adds precise, user‑space memory‑pressure prediction and configurable process‑killing at node and pod levels, effectively mitigating Near‑OOM induced jitter and system hangs.