Unlocking Linux Memory: Architecture, Allocation Algorithms, and Common Pitfalls

Linux memory is a critical resource for backend developers; understanding its structure and management mechanisms can greatly improve system performance and stability. This article covers the organization of Linux memory, address spaces, allocation algorithms, usage scenarios, and common pitfalls.

1. Introduction to Linux Memory

Memory, also called main memory, is the directly addressable storage for the CPU, built from semiconductor devices. Its fast access speed makes it essential for storing temporary computation data, swapping data with external storage, and ensuring CPU stability.

2. Linux Memory Address Space

2.1 Overall Layout

The Linux memory map includes user space and kernel space, each with distinct regions.

User mode (Ring3) : Code runs with limited privileges.

Kernel mode (Ring0) : Core code runs with full privileges.

Transition methods: system calls, exceptions, and hardware interrupts.

2.2 MMU Address Translation

The MMU consists of segmentation and paging units.

Segmentation converts a logical address to a linear address.

Paging converts a linear address to a physical address.

2.3 Segmentation Details

Segment selectors are cached in six registers: cs, ss, ds, es, fs, gs.

Each segment has a base address and a limit defining the usable offset range.

2.4 Paging (32‑bit)

Page directory: 10 bits, page table: 10 bits, offset: 12 bits.

Page size: 4 KB.

2.5 User‑Space Layout

TEXT: executable code, string literals, read‑only data.

DATA: initialized global variables.

BSS: uninitialized global variables.

HEAP: dynamic memory allocated via malloc etc.

MMAP: shared libraries and anonymous mappings.

STACK: process stack.

2.6 Kernel‑Space Layout

Direct‑map area: ~896 MB starting at 3 GB.

vmalloc area: dynamically allocated by vmalloc.

High‑memory area: for devices requiring high physical addresses.

Fixed‑map area: small reserved regions for specific hardware (e.g., ACPI).

3. Linux Memory Allocation Algorithms

3.1 Memory Fragmentation

External fragmentation occurs when many small allocations leave unusable gaps.

Internal fragmentation occurs when allocated blocks are larger than needed.

To mitigate fragmentation:

Prefer stack allocation or large contiguous allocations.

Allocate and free memory within the same function when possible.

Use power‑of‑two sized blocks.

Apply the buddy system for external fragmentation.

Use the slab allocator for internal fragmentation.

Consider custom memory pools.

3.2 Buddy System

Organizes free pages into 11 linked lists for block sizes of 1, 2, 4, …, 1024 pages (up to 4 MB).

Allocation: find the smallest list with a free block; split larger blocks if necessary.

Freeing: coalesce adjacent free blocks of the same order.

Conditions for merging: same size, physically contiguous, same order.

3.3 Allocating >4 MB

Increase MAX_ORDER and recompile the kernel.

Pass mem= parameters at boot (e.g., mem=80M) for ARM/PowerPC.

Use request_mem_region and ioremap_nocache for reserved regions.

Call alloc_boot_mem early in start_kernel.

Use vmalloc for virtually contiguous but physically non‑contiguous memory.

3.4 Slab Allocator

Originated by Jeff Bonwick for SunOS; caches frequently used kernel objects.

Reduces internal fragmentation and allocation overhead.

Structures: slab lists, slabs_empty for reclaiming, and per‑object constructors.

APIs: kmem_cache_create, kmem_cache_alloc, kmem_cache_free, kmalloc / kfree (built on slab).

3.5 Kernel Memory Pools

Create pools with mempool_create, allocate with mempool_alloc, free with mempool_free, and destroy with mempool_destroy.

4. User‑Space Memory Allocation Functions

alloca

: stack allocation, no explicit free. malloc: uninitialized heap allocation. calloc: zero‑initialized heap allocation. realloc: resize existing allocation; may move memory. mmap: map files or devices into memory, shared across processes.

5. DMA Memory

Direct Memory Access allows peripherals to transfer data to/from main memory without CPU intervention. Key signals include DREQ (request), DACK (acknowledge), HRQ (host request), and HLDA (host acknowledge).

6. Memory Usage Scenarios

Page management, slab/kmalloc, user‑space allocations, process memory maps (stack, heap, code, data).

Data transfer between kernel and user space ( copy_from_user, copy_to_user).

Memory‑mapped I/O, reserved memory, DMA buffers.

7. Common Pitfalls

C Memory Leaks

Mismatched new / delete in constructors/destructors.

Missing virtual destructors in base classes.

Improper handling of pointer arrays and copy constructors.

Wild Pointers

Uninitialized pointers, use‑after‑free, out‑of‑scope pointers.

Resource Conflicts

Missing volatile for shared variables, lack of locking in multithreaded code, unsafe mmap usage.

STL Iterator Invalidations

Erasing elements invalidates iterators; store the next iterator before erasing.

C++11 Smart Pointers

Replace auto_ptr with unique_ptr.

Use make_shared for shared_ptr creation.

Manage weak_ptr via lock(), expired(), and get().

8. Monitoring and Releasing Memory

System-wide info: /proc/meminfo, free, top, vmstat, ps aux --sort -rss.

Per‑process info: /proc/<pid>/status.

Drop caches: echo 1 > /proc/sys/vm/drop_caches (pagecache), echo 2 (dentries & inodes), echo 3 (both).

Images illustrating the concepts are retained throughout the article.