Unlock Linux Memory: From Basics to Advanced Allocation Strategies

1. Introduction to Linux Memory

Linux memory is a critical resource for backend developers; proper usage improves system performance and stability. This article introduces Linux memory organization, page layout, causes of fragmentation, optimization algorithms, kernel memory‑management methods, usage scenarios, and common pitfalls.

2. Linux Memory Address Space

2.1 What is Memory?

Memory (main memory) is the storage directly addressable by the CPU, built from semiconductor devices.

It features fast access speed.

2.2 Functions of Memory

Temporarily stores CPU computation data.

Buffers data exchanged with external storage such as disks.

Ensures CPU computation stability and high performance.

2.3 Address Spaces

User mode (Ring 3) runs user‑space code and is protected by the processor.

Kernel mode (Ring 0) runs privileged code and shares a common address space among kernel threads.

Switching from user to kernel can occur via system calls, exceptions, or hardware interrupts.

Each process has an isolated user address space; user programs cannot arbitrarily access kernel space, providing security.

2.4 MMU Address Translation

The MMU contains segmentation and paging units.

Segmentation converts a logical address to a linear address.

Paging converts a linear address to a physical address.

2.5 Segmentation Mechanism

Segment selectors are cached in six segment registers: cs, ss, ds, es, fs, gs.

Base address defines the start of a segment in linear space.

Limit defines the maximum offset within the segment.

2.6 Paging Mechanism (32‑bit)

Page directory: 10 bits, Page table: 10 bits, Offset: 12 bits.

Page size is 4 KB.

2.7 User‑Space Layout

TEXT: executable code, string literals, read‑only data.

DATA: initialized global variables.

BSS: uninitialized global variables.

HEAP: dynamically allocated memory (malloc, new).

MMAP: memory‑mapped files and shared libraries.

STACK: process stack.

2.8 Kernel‑Space Layout

Direct‑mapped region: starts at 3 GB, up to 896 MB.

Dynamic vmalloc region.

Permanent high‑memory mapping.

Fixed mapping (e.g., ACPI_BASE) with a 4 KB guard.

2.9 Process Memory Space

User processes can only access their own virtual address space.

Kernel space is managed by the kernel and is consistent across processes.

3. Linux Memory Allocation Algorithms

3.1 Memory Fragmentation

Fragmentation occurs when many small allocations with long lifetimes leave unusable gaps, reducing memory utilization and slowing the system.

3.2 Avoiding Fragmentation

Prefer stack allocation over dynamic allocation.

Allocate and free memory within the same function when possible.

Request larger blocks instead of many tiny ones.

Allocate sizes that are powers of two.

Use the buddy system to reduce external fragmentation.

Use the slab allocator to reduce internal fragmentation.

Design custom memory pools for specific workloads.

3.3 Buddy System

The buddy system groups free pages into 11 lists (1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024 pages). Allocation requests 2^i pages; if none are free, higher‑order lists are split until a suitable block is found. On free, adjacent buddies are merged.

3.4 Slab Allocator

Derived from SunOS’s algorithm, the slab allocator caches frequently used kernel objects (e.g., task_struct) to reduce allocation overhead. It supports generic caches (kmem_cache_create, kmem_cache_alloc/free) and specialized caches for particular object types.

3.5 Kernel Memory Pools

Memory pools pre‑allocate a number of equal‑sized blocks for fast reuse, minimizing fragmentation. APIs: mempool_create, mempool_alloc, mempool_free, mempool_destroy.

3.6 User‑Space Memory Pools (C++)

Example implementations use custom allocators to manage large contiguous blocks, reducing fragmentation and allocation overhead.

3.7 DMA Memory

Direct Memory Access allows peripherals to transfer data to/from main memory without CPU intervention. DMA controller signals include DREQ (request), DACK (acknowledge), HRQ (request bus), and HLDA (grant).

4. Memory Usage Scenarios

Page management.

Slab (kmalloc, memory pools).

User‑space allocation (malloc, realloc, mmap, shared memory).

Process memory map (stack, heap, code, data).

Data transfer between kernel and user (copy_from_user, copy_to_user).

Memory‑mapped hardware registers and reserved memory.

DMA buffers.

4.1 User‑Space Allocation Functions

alloca – allocates on the stack, no free needed.

malloc – allocates uninitialized memory; may cause issues after reuse.

calloc – allocates zero‑initialized memory.

realloc – expands or moves an existing allocation.

4.2 Kernel‑Space Allocation Functions

get_free_pages – allocates one or more physical pages.

kmalloc / kmem_cache_alloc – slab‑based allocation for up to 128 KB.

vmalloc – creates a virtually contiguous mapping for large, non‑contiguous physical memory.

dma_alloc_coherent – allocates DMA‑capable memory (typically 4 MB).

ioremap – maps a known physical address into kernel virtual space.

alloc_bootmem – reserves memory early in boot.

4.3 malloc Implementation Details

malloc searches the free chunk list for a suitable block; if none is found, it expands the heap via the brk system call.

4.4 Page Fault Handling

When a user‑space address lacks a physical mapping, a page‑fault exception triggers kernel allocation of a physical page and updates the page table.

4.5 Shared Memory

shmget – creates a shared memory segment.

shmat – attaches the segment to the process address space.

shmdt – detaches the segment.

5. Common Memory Pitfalls

5.1 C Memory Leaks

Missing matching new/delete in constructors/destructors.

Failing to delete nested object pointers.

Non‑virtual base destructors causing derived resources to leak.

Absent copy constructors leading to shallow copies.

Confusing arrays of object pointers with arrays of objects.

Missing overloaded assignment operators for variable‑size classes.

5.2 C Wild Pointers

Uninitialized pointer variables.

Using freed pointers without resetting to NULL.

Returning pointers to stack memory.

Dereferencing null pointers without checks.

Incorrect use of sizeof on arrays.

5.3 Resource Conflicts

Missing volatile on shared variables in multithreaded code.

Unprotected global variable access.

Global variables ineffective across processes.

Lack of synchronization when multiple processes write shared memory.

Unsafe mmap usage across processes.

5.4 STL Iterator Invalidation

Erasing an element invalidates its iterator.

Insertions or deletions may invalidate iterators of sequential containers.

5.5 C++11 Smart Pointers

Replace auto_ptr with unique_ptr.

Initialize shared_ptr via make_shared.

Use weak_ptr to break reference cycles (lock, expired, get).

5.6 Modern C++ Features

std::atomic for lock‑free multithreading.

std::array for fixed‑size arrays.

std::vector::shrink_to_fit to reduce capacity.

std::forward_list for memory‑efficient singly linked lists.

std::unordered_map / std::unordered_set for O(1) average lookup.

6. How to Inspect Memory

System memory info: /proc/meminfo Process memory status: /proc/<pid>/status Overall memory usage: free Process CPU/memory usage: top Virtual memory statistics: vmstat Process memory ranking: ps aux --sort -rss Drop caches: echo 1 > /proc/sys/vm/drop_caches (pagecache), echo 2 > /proc/sys/vm/drop_caches (dentries/inodes), echo 3 > /proc/sys/vm/drop_caches (both).