Mastering Linux Memory Management: Structures, Allocation Algorithms, and Common Pitfalls

Linux memory is a critical resource for backend developers; understanding its organization, allocation mechanisms, and common pitfalls can greatly improve system performance and stability.

1. Introduction to Memory

Memory, also called main memory or RAM, is the storage directly addressable by the CPU. It offers fast access speeds and temporarily holds data needed for computation.

Key functions of memory include:

Temporarily storing CPU computation data.

Exchanging data with external storage such as disks.

Ensuring stable and high‑performance CPU operation.

2. Linux Memory Address Space

Linux separates memory into user space and kernel space. User‑mode code runs in Ring 3, while kernel‑mode code runs in Ring 0. Transitions between the two occur via system calls, exceptions, or hardware interrupts.

Each process has an isolated virtual address space; the kernel’s address space is shared among all processes and is protected from user‑mode access.

2.1 MMU Address Translation

The Memory Management Unit (MMU) performs two stages of translation:

Segmentation: converts a logical address to a linear address.

Paging: converts the linear address to a physical address.

2.2 Segmentation Details

Segment selectors are cached in six segment registers (cs, ss, ds, es, fs, gs).

Each segment has a base address (starting point in linear space) and a limit (maximum offset).

2.3 Paging (32‑bit)

Page directory (10 bits), page table (10 bits), and page offset (12 bits).

Page size is 4 KB.

2.4 User‑Mode Address Layout

TEXT : executable code and read‑only data.

DATA : initialized global variables.

BSS : uninitialized global variables.

HEAP : dynamically allocated memory via malloc, calloc, etc.

MMAP : memory‑mapped files and shared libraries.

STACK : per‑process call stack.

2.5 Kernel‑Mode Address Layout

Direct‑map region: a contiguous range starting at 3 GB for physical‑to‑virtual mapping.

Vmalloc region: non‑contiguous memory allocated by vmalloc.

Permanent‑map region: accesses high memory.

Fixed‑map region: small fixed‑size mappings for special purposes (e.g., ACPI).

3. Linux Memory Allocation Algorithms

3.1 Memory Fragmentation

Fragmentation occurs when many small allocations leave unusable gaps. It reduces performance and memory utilization.

3.2 Buddy System

The buddy allocator groups free pages into 11 free‑list bins (sizes 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024 pages). Allocation requests of 2^i pages are satisfied from the corresponding bin; if empty, larger bins are split recursively. Deallocation merges adjacent free blocks (buddies) when possible.

Typical usage scenarios for allocating >4 MB include:

Increasing MAX_ORDER and recompiling the kernel.

Passing a mem= parameter at boot (ARM/PowerPC only).

Calling alloc_bootmem before mem_init (requires recompilation).

Using vmalloc for virtually contiguous but physically scattered memory.

3.3 Slab Allocator

Derived from the SunOS slab algorithm, the Linux slab allocator caches frequently used kernel objects to reduce allocation overhead and internal fragmentation.

Objects are allocated from slab caches via kmem_cache_alloc and freed with kmem_cache_free.

Common caches are managed in slabs_empty for reclamation.

Specialized caches can be created with kmem_cache_create.

3.4 Kernel and User Memory Pools

Kernel pools pre‑allocate a set of equal‑sized blocks to avoid fragmentation during runtime. User‑space pools (e.g., C++ memory‑pool implementations) provide similar benefits for application code.

4. Memory Usage Scenarios

Page management.

Slab (kmalloc, memory pools).

User‑mode allocation ( malloc, realloc, file mapping, shared memory).

Process memory map (text, data, heap, stack, mmap).

Data transfer between kernel and user ( copy_from_user, copy_to_user).

Memory‑mapped hardware registers and reserved regions.

DMA buffers.

5. Common Memory‑Related Pitfalls

5.1 C Memory Leaks

Mismatched new / delete in constructors/destructors.

Missing deletion of nested pointers.

Non‑virtual base destructors causing incomplete cleanup.

Absent copy constructors or overloaded assignment operators.

5.2 C Wild Pointers

Uninitialized pointers.

Use‑after‑free without resetting to NULL.

Returning pointers to stack memory.

Dereferencing null pointers.

5.3 Resource Conflicts

Missing volatile on shared variables.

Unprotected global variables in multithreaded code.

Unsynchronized writes to shared memory.

5.4 STL Iterator Invalidation

Erasing an element invalidates the iterator; store the next iterator before erasing.

5.5 Modern C++ Practices

Replace auto_ptr with unique_ptr.

Initialize shared_ptr via std::make_shared.

Use weak_ptr to break reference cycles (methods: lock(), expired(), get()).

5.6 C++11 Improvements

std::atomic

for lock‑free thread safety. std::array for fixed‑size arrays. std::vector::shrink_to_fit() to reduce capacity. std::forward_list for memory‑efficient singly linked lists. std::unordered_map/set for O(1) average lookup.

6. Observing and Managing Memory

System memory info: /proc/meminfo.

Process memory status: /proc/<pid>/status.

Overall usage: free, top, vmstat, ps aux --sort=-rss.

Drop caches: echo 1 > /proc/sys/vm/drop_caches (pagecache), echo 2 > /proc/sys/vm/drop_caches (dentries & inodes), echo 3 > /proc/sys/vm/drop_caches (both).

By mastering these concepts and tools, developers can diagnose memory‑related bugs, optimize allocation strategies, and maintain robust Linux systems.