Unlocking Linux Memory: From Basics to Advanced Allocation Strategies

1. Introduction to Linux Memory

Linux memory is a critical resource for backend developers; proper usage improves system performance and stability. The article outlines the memory organization, page layout, causes of fragmentation, optimization algorithms, kernel memory‑management methods, usage scenarios, and common pitfalls.

2. Linux Memory Address Space

2.1 Overall Layout

The Linux memory address space is divided into user‑mode and kernel‑mode regions, each with independent virtual address spaces. User processes cannot directly access kernel addresses, providing security isolation.

2.2 User‑Mode vs. Kernel‑Mode

User‑mode (Ring 3) code runs with limited privileges.

Kernel‑mode (Ring 0) code runs with full privileges.

Transitions occur via system calls, exceptions, or hardware interrupts.

2.3 MMU Address Translation

The MMU consists of segmentation and paging units.

Segmentation converts a logical address to a linear address.

Paging converts a linear address to a physical address.

2.4 Segmentation Mechanism

Segment selectors are cached in six segment registers (cs, ss, ds, es, fs, gs).

Each segment has a base address and a limit defining the usable offset range.

2.5 Paging Mechanism (32‑bit)

Page directory (10 bits) → page table (10 bits) → page offset (12 bits).

Page size is 4 KB.

2.6 User‑Mode Address Space Layout

TEXT : executable code, read‑only literals.

DATA : initialized global variables.

BSS : uninitialized global variables.

HEAP : dynamic allocations via malloc, realloc, etc.

MMAP : memory‑mapped files and shared libraries.

STACK : process stack.

2.7 Kernel‑Mode Address Space

Direct‑map region (≈896 MB starting at 3 GB).

Dynamic vmalloc region.

Permanent high‑memory mapping.

Fixed‑mapping region (e.g., ACPI_BASE).

2.8 Process Memory Space

User processes can only access their own virtual address space.

Kernel space is shared among all processes and has its own page tables.

3. Linux Memory Allocation Algorithms

3.1 Fragmentation

External fragmentation occurs when many small allocations leave unusable gaps.

Internal fragmentation wastes memory within allocated blocks.

3.2 Avoiding Fragmentation

Prefer stack allocation over dynamic allocation.

Allocate and free memory within the same function when possible.

Request larger blocks instead of many tiny ones.

Allocate sizes that are powers of two.

Use the buddy system for external fragmentation.

Use the slab allocator for internal fragmentation.

Design custom memory pools for specific workloads.

3.3 Buddy System

Divides free pages into 11 linked lists, each representing block sizes from 1 to 1024 pages (max 4 MB).

Allocation: find the smallest suitable block; if unavailable, split larger blocks recursively.

Freeing: coalesce adjacent free blocks of the same order.

3.4 Allocating >4 MB

Increase MAX_ORDER and recompile the kernel.

Pass mem= parameter at boot (ARM/PowerPC only).

Reserve memory with request_mem_region and map via ioremap_nocache.

Allocate early in start_kernel using alloc_boot_mem.

Use vmalloc for virtually contiguous but physically non‑contiguous memory.

3.5 Slab Allocator

Originated in SunOS (Jeff Bonwick).

Caches frequently used kernel objects to reduce allocation overhead.

Provides kmem_cache_create, kmem_cache_alloc, and kmem_cache_free APIs.

3.6 Kernel Memory Pools

Create pools with mempool_create, allocate with mempool_alloc, free with mempool_free, and destroy with mempool_destroy.

3.7 User‑Space Memory Pools (C++ Example)

Illustrates a custom memory pool implementation for C++ applications (image omitted).

3.8 DMA Memory

Direct Memory Access allows peripherals to read/write main memory without CPU intervention.

DMA controller signals: DREQ (request), DACK (acknowledge), HRQ (host request), HLDA (host acknowledge).

4. Memory Usage Scenarios

Page management, slab/kmalloc, user‑space allocations (malloc, mmap, shared memory).

Process memory mapping (text, data, bss, heap, stack, mmap).

Kernel‑user data transfer ( copy_from_user, copy_to_user).

Memory‑mapped hardware registers and reserved regions.

DMA buffers.

4.1 User‑Space Allocation Functions

alloca

: stack allocation (no free needed). malloc: uninitialized heap memory. calloc: zero‑initialized heap memory. realloc: resize existing allocation, possibly moving it.

4.2 Kernel‑Space Allocation Functions

get_free_pages

: allocate one or more physical pages. kmalloc / kfree: slab‑based allocation for sizes < 128 KB. vmalloc: virtually contiguous memory, not physically contiguous. dma_alloc_coherent: DMA‑compatible memory (typically 4 MB). ioremap: map known physical addresses into kernel virtual space.

5. Common Memory Pitfalls

5.1 C Memory Leaks

Mismatched new / delete in constructors/destructors.

Missing deletion of nested pointers.

Non‑virtual base destructors causing incomplete cleanup.

Absent copy constructors leading to shallow copies.

Improper handling of pointer arrays versus object arrays.

Missing overloaded assignment operators.

5.2 C Wild Pointers

Uninitialized pointers.

Use‑after‑free without resetting to NULL.

Returning pointers to stack memory.

Dereferencing null pointers.

Incorrect sizeof usage on arrays.

5.3 C Resource Conflicts

Missing volatile on shared variables.

Unprotected global variable access in multithreaded code.

Shared memory writes without synchronization.

Unsafe mmap usage across processes.

5.4 STL Iterator Invalidations

Erasing an iterator invalidates it.

Insertions or deletions can invalidate iterators of sequential containers.

5.5 C++11 Smart Pointers

Replace auto_ptr with unique_ptr.

Use make_shared to create shared_ptr. weak_ptr provides non‑owning references; use lock(), expired(), and get() to manage them.

5.6 Modern C++ Utilities

std::atomic

for lock‑free thread safety. std::array for fixed‑size arrays. std::vector::shrink_to_fit() to reduce capacity. std::forward_list (singly linked list) saves memory compared to std::list. std::unordered_map / std::unordered_set provide O(1) average lookup.

5.7 Monitoring Memory

System memory info: /proc/meminfo.

Process memory status: /proc/<pid>/status.

Overall usage: free, top, vmstat, ps aux --sort -rss.

Drop caches: echo 1 > /proc/sys/vm/drop_caches (pagecache), echo 2 (dentries/inodes), echo 3 (both).