Understanding Linux Memory Management: Allocation, OOM, and Cache

Preface: Inspired by an OOM talk, the author shares a detailed overview of Linux kernel memory management.

1. Process Memory Allocation

When a program starts, exec loads the executable and maps the code, data, BSS, and stack segments via mmap. The heap is created on the first malloc call, using brk or an anonymous mmap. The kernel builds VMA structures and the memory allocator (ptmalloc, tcmalloc, jemalloc) returns the requested memory; large allocations use direct mmap. Freeing memory either calls munmap for mmap regions or returns pages to the allocator.

2. Out‑of‑Memory (OOM) Handling

When memory is exhausted, the OOM killer selects a victim based on factors such as memory usage, runtime, priority, user, child count and oom_adj. Each process receives an oom_score; the highest score is killed. Adjusting /proc/<pid>/oom_adj can protect a process (e.g., setting -17 makes it immune). The kernel parameter /proc/sys/vm/overcommit_memory controls over‑commit behavior: 0 = heuristic, 1 = always allow, 2 = strict limit.

3. Where Allocated Memory Resides

Memory can be in page cache or ordinary RAM. File‑backed mappings (code, data, shared libraries) are cached; shared file mappings increase buff/cache. Private file mappings use copy‑on‑write, also increasing cache. Anonymous private mappings allocate pages directly in RAM without using cache. Shared anonymous mappings use the cache so that parent and child share the same physical pages.

4. Memory Reclamation

Manual reclamation can be triggered with echo 1 > /proc/sys/vm/drop_caches, which frees page cache, dentries, and inodes. The kernel’s kswapd thread periodically scans LRU lists, evicts inactive pages, and may swap out anonymous pages. Parameters such as vm.swappiness control the preference for swapping versus cache reclamation.

Typical commands:

echo 1 >> /proc/sys/vm/drop_caches

The kernel also uses tmpfs for shared memory, POSIX shared memory, and other IPC mechanisms; pages belonging to tmpfs are not reclaimable by drop_caches because they are referenced by files.

5. Summary

The article reviews the process address space, explains how Linux allocates and frees memory, describes OOM selection, shows where different kinds of mappings are stored, and outlines both manual and automatic memory reclamation mechanisms.