Unlocking Linux Memory: How Virtual Memory, MMU, and Heap Really Work

Do you really know how much memory your Linux program "eats"? The numbers shown by top or free (VSZ and RSS) are constantly changing, but where does the memory actually hide? Even a few kilobytes of code can show several megabytes of virtual memory. In fact, each Linux process does not store memory in a chaotic heap; it follows a clear layered architecture: code segment, data segment, dynamically allocated heap, thread‑private stack, and kernel‑mapped shared memory.

Understanding this layout helps you interpret the numbers in top, locate memory leaks (usually in the heap), and reduce stack overflow or heap fragmentation. The following sections dissect the memory architecture layer by layer.

1. Process “Exclusive Memory Space”: Virtual Memory Mechanism

1.1 Each Process Has Its Own Memory Palace

Under OS management, every process appears to have its own "memory palace"—the virtual address space. On a 32‑bit system each process theoretically owns 4 GB of virtual addresses (0x00000000‑0xFFFFFFFF), like a private warehouse of 4 GB that the process can allocate freely without colliding with other processes. This virtual memory mechanism is key to isolation and sharing, allowing many processes to run safely on the same physical machine.

Think of a process as a hotel guest with a unique room number (virtual address). The OS front desk allocates physical rooms (physical memory) and maps them to the guest’s virtual number. Guests can request new rooms, and the OS assigns them without interfering with other guests, maximizing resource utilization.

A running program is always executed segment by segment; only the active segment resides in memory while others stay on disk.

When a new segment needs to run, the OS copies the required segment from auxiliary storage into memory, leaving the rest on disk.

If another segment (e.g., segment 2) must run, the OS swaps out segment 1 back to disk, frees memory, and swaps in segment 2.

Swapping out copies a program segment back to auxiliary storage; swapping in maps a segment from storage into memory. Repeated swapping lets the processor run programs larger than physical RAM, creating a virtual memory space that appears larger than actual physical memory.

The size of a real computer’s virtual memory space is determined by the address width of the program counter. For a 32‑bit processor the virtual address space is 4 GB.

Thus a system with virtual memory has two spaces: virtual memory (addresses called "virtual addresses") and physical memory (addresses called "physical addresses"). The processor and programmers see only virtual addresses; the hardware sees only physical addresses.

Mapping from virtual to physical addresses occurs twice: first the OS creates a virtual address space, then the hardware (MMU) and OS memory manager map it to physical memory. The hardware component is the Memory Management Unit (MMU); the software component is the OS memory‑management module.

The OS maintains a table indexed by virtual address that records the corresponding physical address. This table is the basis for the MMU’s address translation.

In summary, virtual memory works because programs can be divided into segments and only a small portion of the total address space is needed at any moment. The technique uses auxiliary storage to simulate RAM, allowing the address space to exceed physical memory.

From the processor and programmer’s perspective, they see a virtual memory space encapsulated by the MMU, mapping tables, and physical memory; its size depends on the processor’s address width.

The system gives each program a virtual memory space equal to the processor’s address space.

At any moment the processor works with only one program’s mapping table, ensuring isolation.

The mapping table makes physical memory sharing easy.

1.2 The "Translator": MMU and Page Tables

The MMU enables multiple tasks to run in their own private virtual memory spaces without needing to know the actual physical addresses.

Think of memory as a large warehouse and programs as customers. The MMU is the warehouse manager that registers each request (virtual address) and translates it to the actual storage location (physical address), ensuring safety and order.

When a process accesses a virtual address, the MMU looks up the page table, which is like a dictionary mapping virtual pages to physical pages. Modern OSes use multi‑level page tables; for example, x86‑64 uses a four‑level hierarchy.

Linux uses two key structures to describe a process’s memory layout: mm_struct (describes the whole virtual address space) and vm_area_struct (describes each continuous virtual region with its permissions and optional file mapping). Together they allow precise memory management.

1.3 Page‑Fault Handling: Real Memory Is Allocated on First Access

When malloc reserves virtual address space, no physical memory is allocated yet. The first access triggers a page‑fault; the kernel then allocates a physical page and creates the mapping.

The flow: the CPU detects an invalid page‑table entry, raises a page‑fault, the kernel’s do_user_addr_fault finds the corresponding vm_area_struct, __handle_mm_fault creates a new page‑table entry, and for anonymous memory do_anonymous_page allocates a physical page. This is analogous to booking a hotel room (virtual memory) and only having the room prepared (physical memory) when you actually check‑in.

2. Process Memory Lifecycle: From Startup to Runtime Layout

2.1 Startup: How a Program Is Loaded into Memory

When you run an executable (e.g., ./a.out) Linux creates a new process, allocates an mm_struct, and loads the ELF file into memory. The ELF loader reads the program header table and maps code (.text) and data (.data, .bss) segments into the process’s virtual address space using mmap. Code segments get executable permissions; data segments get read‑write permissions.

The function __bprm_mm_init initializes the process’s memory, allocating an initial 4 KB stack. For dynamically linked programs, the loader also maps shared libraries (.so) via elf_map and resolves symbols.

2.2 Runtime: Dynamic Expansion of Heap and Stack

The stack grows downward; each function call creates a stack frame. The following C program demonstrates stack growth:

#include <stdio.h>

void recursive_function(int depth) {
    int localVar = 0;
    printf("Depth: %d, localVar address: %p
", depth, &localVar);
    if (depth < 10) {
        recursive_function(depth + 1);
    }
}

int main() {
    recursive_function(0);
    return 0;
}

Each recursive call creates a new frame, and the address of localVar moves to lower addresses, illustrating stack growth. If the stack exceeds its limit, a stack overflow occurs.

The heap grows upward; malloc, calloc, etc., request memory. Internally the kernel uses brk (or sbrk) to move the program break and enlarge the data segment, or mmap for large or non‑contiguous allocations.

When malloc is called, the kernel function do_brk_flags creates a new vm_area_struct for the heap region and adds it to the process’s address space. Subsequent allocations and frees cause the heap size to expand or shrink like an elastic container.

2.3 High‑Speed Path: TLB and the Principle of Locality

Repeated address translation via page tables would be slow. The Translation Lookaside Buffer (TLB) caches recent page‑table entries. On a memory access the CPU first checks the TLB; a hit yields the physical address instantly, while a miss requires a page‑table walk.

The locality principle (temporal and spatial) explains why most accesses hit the TLB: recently accessed data is likely to be accessed again soon, and data near each other in memory tends to be accessed together.

CPU accesses a virtual address and checks the TLB.

If the entry is present (TLB hit), the physical address is obtained directly.

If not (TLB miss), the CPU walks the page table in memory.

The OS loads the correct page‑table entry into the TLB.

Subsequent accesses to the same address hit the TLB, improving performance.

TLB misses incur extra memory accesses; the OS manages TLB flushing and replacement policies (fully associative, set‑associative, direct‑mapped).

3. Dynamic Memory Management: From malloc to Kernel System Calls

3.1 User‑Space Interface: How malloc “Tricks” Programmers

malloc

first reserves virtual address space; it does not allocate physical memory immediately. For requests < 128 KB it typically uses brk to extend the heap; for larger requests it uses mmap to map an anonymous region.

Physical memory is only allocated on the first write to the reserved pages (page‑fault). This is also where Copy‑On‑Write (COW) comes into play: after fork, parent and child share the same physical pages until one writes, at which point the kernel copies the page for the writer.

3.2 Kernel Implementation: Differences Between brk and mmap

brk

moves the end of the data segment to grow or shrink the heap. It is simple and efficient for small allocations but can cause fragmentation. mmap creates a new virtual region (often between heap and stack) and can map files or anonymous memory. It supports non‑contiguous large allocations and avoids fragmentation, making it suitable for allocations > 1 MB.

3.3 Memory Metrics: Understanding Process Memory Usage

Key metrics:

Virtual Size (VSZ) : total virtual address space used, including unmapped pages and shared libraries.

Resident Set Size (RSS) : actual physical memory currently resident (working set).

Private Bytes : physical memory used exclusively by the process (not shared).

VSZ gives an overview of required address space but may over‑state physical usage. RSS reflects real memory pressure; if total RSS approaches physical RAM, the system may start swapping. Private Bytes help isolate a process’s own memory consumption.

4. Performance and Security: The Double‑Edged Sword of Process Memory Management

4.1 Performance Optimization Key Points

Reducing page‑faults improves performance because each fault involves slow disk I/O. Strategies include pre‑allocating memory with posix_memalign (ensuring alignment and reducing fragmentation) and using huge pages (2 MB or larger) to lower page‑table overhead.

Applying the locality principle—placing frequently accessed data together and iterating arrays sequentially—also reduces cache and TLB misses.

4.2 Common Issues and Debugging

Memory leaks occur when dynamically allocated memory is never freed, eventually exhausting RAM. Tools like valgrind can detect leaks, uninitialized reads, and out‑of‑bounds accesses.

Wild pointer accesses happen when a freed pointer is used. Example:

#include <stdio.h>
#include <stdlib.h>

int main() {
    int *ptr = (int *)malloc(sizeof(int));
    *ptr = 10;
    free(ptr);
    // ptr is now a wild pointer
    *ptr = 20; // undefined behavior, may cause segmentation fault
    return 0;
}

AddressSanitizer (ASan) can catch such illegal accesses at runtime when compiled with -fsanitize=address.

4.3 Security Mechanisms: From Isolation to Permission Control

Address Space Layout Randomization (ASLR) randomizes the base addresses of the stack, heap, and shared libraries, making it harder for attackers to predict where to inject code.

Page‑permission control sets read/write/execute bits per page. For example, code segments are marked read‑execute only, preventing accidental or malicious writes. The mprotect system call can change permissions dynamically, useful for JIT compilers that need to make generated code executable.

5. Practical Tools for Inspecting Process Memory

5.1 Command‑Line: /proc Filesystem

The /proc filesystem provides insight into a process’s memory layout. cat /proc/[pid]/maps shows each virtual region with its address range, permissions, offset, device, inode, and mapped file. Example output:

555555554000-555555557000 r-xp 00000000 08:01 10000000000000000000 /usr/bin/your_program
555555756000-555555757000 r--p 00002000 08:01 10000000000000000000 /usr/bin/your_program
555555757000-555555758000 rw-p 00003000 08:01 10000000000000000000 /usr/bin/your_program
7ffff7fad000-7ffff7faf000 rw-p 00000000 00:00 0
... (additional mappings) ...
7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]

cat /proc/[pid]/smaps

provides detailed statistics per region (Size, Rss, Pss, Shared_Clean, Private_Clean, etc.), useful for deep memory analysis.

555555554000-555555557000 r-xp 00000000 08:01 10000000000000000000 /usr/bin/your_program
Size:               12 kB
Rss:                4 kB
Pss:                4 kB
Shared_Clean:       0 kB
Private_Clean:      4 kB
... (more fields) ...
VmFlags: rd ex mr mw me ac sd

5.2 Programming Interfaces: sysinfo and malloc_info

sysinfo()

(declared in <sys/sysinfo.h>) returns system‑wide memory and swap statistics via a struct sysinfo:

#include <sys/sysinfo.h>
struct sysinfo info;
if (sysinfo(&info) == -1) perror("sysinfo");
printf("Total RAM: %lu bytes
", info.totalram);
printf("Free RAM: %lu bytes
", info.freeram);
printf("Total Swap: %lu bytes
", info.totalswap);
printf("Free Swap: %lu bytes
", info.freeswap);

malloc_info()

(glibc) prints the current heap state, showing allocated and free chunks and fragmentation. Example usage:

#include <stdio.h>
#include <stdlib.h>
#include <malloc/malloc.h>
int main() {
    int *p1 = malloc(1024 * sizeof(int));
    int *p2 = malloc(2048 * sizeof(int));
    malloc_info(0, stdout);
    free(p1);
    free(p2);
    malloc_info(0, stdout);
    return 0;
}

5.3 Visual Tools: GDB and Memory Visualization

GDB can inspect memory with the x command, e.g., (gdb) x/10xw 0x7fffffffde40 displays ten 4‑byte words at the given address.

The vmmap GDB plugin visualizes the process’s memory map:

(gdb) vmmap
Start               End                 Offset   Perm Pathname
0x00400000          0x00401000          0x0      r-xp /path/to/your_program
0x00600000          0x00601000          0x0      r--p /path/to/your_program
0x00601000          0x00602000          0x1000   rw-p /path/to/your_program
0x7ffff7a0d000      0x7ffff7a2e000      0x0      r-xp /lib/x86_64-linux-gnu/libc-2.31.so
... (more mappings) ...
0x7ffffffde000      0x7ffffffff000      0x0      rw-p [stack]
ffffffffff600000    ffffffff601000      0x0      --xp [vsyscall]

On Windows, Process Explorer provides similar visualizations, showing private bytes, working set, and other memory details per process.