Unlocking Linux Memory: From Address Spaces to Allocation Algorithms

1. Introduction to Linux Memory

Linux memory is a critical resource for backend developers; proper usage improves system performance and stability. The article introduces Linux memory organization, page layout, causes of fragmentation, optimization algorithms, kernel memory‑management methods, usage scenarios, and common pitfalls.

2. Linux Memory Address Space

Linux separates memory into user space and kernel space. User‑mode code runs in Ring 3, while the kernel runs in Ring 0 with protected access. The address translation involves the MMU, which first converts a logical address to a linear address via segmentation, then maps the linear address to a physical address via paging.

User space: isolated virtual address space for each process.

Kernel space: shared among all processes, includes direct‑mapping, vmalloc, and fixed‑mapping regions.

3. Linux Memory Allocation Algorithms

3.1 Memory Fragmentation

Fragmentation occurs when many small allocations with long lifetimes leave unusable gaps, reducing overall utilization. It can be mitigated by allocating larger blocks, using power‑of‑two sizes, or employing specialized allocators.

3.2 Buddy System

The buddy allocator groups free pages into 11 size‑ordered lists (1, 2, 4 … 1024 pages). Allocation requests 2^i pages; if the corresponding list is empty, larger blocks are split recursively. Freeing merges adjacent buddies of the same order.

3.3 Slab Allocator

Derived from the SunOS slab algorithm, it caches frequently used kernel objects (e.g., task structs) to reduce allocation overhead and internal fragmentation. Objects are allocated from slabs; slabs are kept in caches (kmem_cache_create, kmem_cache_alloc, kmem_cache_free).

3.4 Kernel and User‑Space Memory Pools

Kernel pools pre‑allocate equal‑sized blocks (mem‑pool) to avoid fragmentation. User‑space pools use standard libc functions (malloc, calloc, realloc) and system calls (brk, mmap) for dynamic allocation.

3.5 DMA Memory

Direct Memory Access allows peripherals to read/write main memory without CPU intervention. The DMA controller uses signals such as DREQ, DACK, HRQ, and HLDA to coordinate transfers.

4. Memory Usage Scenarios

Page management (kmalloc, slab, memory pools).

User‑space allocation (malloc, calloc, realloc, mmap, shared memory).

Kernel‑user data transfer (copy_from_user, copy_to_user).

Memory‑mapped I/O and reserved regions.

DMA buffers.

5. Common Pitfalls

5.1 C Memory Leaks

Mismatched new/delete or malloc/free.

Missing virtual destructors in base classes.

Improper handling of pointer arrays.

5.2 Wild Pointers

Uninitialized pointers, use‑after‑free, out‑of‑scope pointers.

5.3 Resource Conflicts

Unsynchronized access to global variables in multithreaded code.

Improper use of shared memory without locking.

5.4 STL Iterator Invalidations

Erasing elements invalidates iterators; store the next iterator before erasing.

5.5 Modern C++ Practices

Replace deprecated auto_ptr with unique_ptr.

Use make_shared for shared_ptr creation.

Employ weak_ptr to break reference cycles.

6. Inspecting Memory Usage

System memory: /proc/meminfo Process memory: /proc/<pid>/status Overall usage: free Top processes: top Virtual memory stats: vmstat Sort processes by RSS: ps aux --sort=-rss Drop caches: echo 1 > /proc/sys/vm/drop_caches (pagecache), echo 2 > /proc/sys/vm/drop_caches (dentries & inodes), echo 3 > /proc/sys/vm/drop_caches (both).