Master Linux Memory Management: From Physical Pages to Virtual Mapping

Overview of Linux Memory Management

Linux memory management controls allocation, release, mapping, swapping and compression of system memory. It separates memory into kernel space, user space, caches and swap partitions, aiming to maximise usable memory while keeping the system stable.

Key Components

Virtual memory – maps physical pages to independent process address spaces.

Physical memory – manages RAM pages, allocation and reclamation.

Page‑replacement algorithms – decide which pages to evict when RAM is scarce.

Process address‑space management – handles code, data and stack segments.

Memory protection – sets page attributes and access permissions.

Statistics & monitoring – provides usage data for tuning and troubleshooting.

Physical Memory Management

Physical memory is divided into fixed‑size pages (typically 4 KB or 8 KB). Linux uses two allocation strategies:

Contiguous allocation (Buddy System) – splits memory into power‑of‑two blocks and merges buddies to reduce fragmentation.

Non‑contiguous allocation – uses paging to map virtual pages to any free physical page.

Core memblock API

memblock_init()

– initialise the memblock allocator. memblock_reserve() – reserve regions that must not be allocated. memblock_alloc() – allocate a physical block. memblock_free() – free a previously allocated block. memblock_find_in_range() – locate a free block within a given address range.

Example kernel module that allocates a 4 KB block:

#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/mm.h>

static int __init test_init(void)
{
    unsigned long size = 4096;
    unsigned long *ptr = memblock_alloc(size, PAGE_SIZE);
    if (!ptr) {
        pr_err("Failed to allocate memory
");
        return -ENOMEM;
    }
    pr_info("Allocated %lu bytes at %p
", size, ptr);
    return 0;
}

static void __exit test_exit(void)
{
    pr_info("Exiting test module
");
}

module_init(test_init);
module_exit(test_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Author");
MODULE_DESCRIPTION("Simple memblock demo");

Virtual Memory Management

Virtual memory abstracts RAM by providing each process with a contiguous address space backed by RAM, swap or files. When a process accesses an unmapped virtual address, the kernel allocates a physical page and updates the page table. The MMU translates virtual to physical addresses.

Common System Calls

mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset)

– map a file or anonymous memory into a process. munmap(void *addr, size_t length) – remove a mapping. mlock(const void *addr, size_t len) – pin pages in RAM. mprotect(void *addr, size_t len, int prot) – change protection flags.

File‑mapping example:

#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>

int main(int argc, char *argv[])
{
    int fd = open(argv[1], O_RDONLY);
    if (fd < 0) { perror("open"); exit(1); }
    struct stat sb;
    if (fstat(fd, &sb) == -1) { perror("fstat"); exit(1); }
    char *addr = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(1); }
    printf("%s", addr);
    munmap(addr, sb.st_size);
    close(fd);
    return 0;
}

Locking 1 MiB of heap memory:

#include <stdlib.h>
#include <stdio.h>
#include <sys/mman.h>

int main(void)
{
    void *p = malloc(1<<20);
    if (!p) { perror("malloc"); return 1; }
    if (mlock(p, 1<<20) != 0) { perror("mlock"); return 1; }
    printf("locked 1 MB memory
");
    free(p);
    return 0;
}

Memory Allocation & Release in User Space

Typical allocation methods:

Static allocation – determined at compile time.

Stack allocation – automatic for function locals.

Heap allocation – malloc(), calloc(), realloc(), free().

Memory‑mapped files – mmap() / munmap() for large data.

Shared memory – IPC mechanisms (shmget, mmap with MAP_SHARED).

Heap allocation examples

Simple malloc / free:

#include <stdlib.h>
#include <stdio.h>

int main(void)
{
    int *p = malloc(sizeof(int));
    if (!p) { printf("Allocation failed
"); return -1; }
    *p = 123;
    printf("%d
", *p);
    free(p);
    return 0;
}

Using calloc and realloc:

#include <stdlib.h>
#include <stdio.h>

int main(void)
{
    int *p1 = calloc(5, sizeof(int));
    if (!p1) { perror("calloc"); return -1; }
    for (int i = 0; i < 5; i++) printf("%d ", p1[i]);
    int *p2 = realloc(p1, 10 * sizeof(int));
    if (!p2) { perror("realloc"); return -1; }
    for (int i = 5; i < 10; i++) p2[i] = i * 2;
    for (int i = 0; i < 10; i++) printf("%d ", p2[i]);
    printf("
");
    free(p2);
    return 0;
}

Mapping a file with mmap and releasing it with munmap:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/stat.h>

int main(int argc, char *argv[])
{
    int fd = open(argv[1], O_RDONLY);
    if (fd == -1) { perror("open"); exit(1); }
    struct stat sb;
    if (fstat(fd, &sb) == -1) { perror("fstat"); exit(1); }
    char *addr = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(1); }
    printf("%s", addr);
    munmap(addr, sb.st_size);
    close(fd);
    return 0;
}

Process Switching and Memory Management

During a context switch the kernel saves the current task's registers and page tables, then loads the next task's state. This requires updating the MMU with the new page tables so each process sees its own virtual address space.

Relevant System Calls

fork()

– creates a child with copy‑on‑write address space. execve() – replaces the current image with a new program. mmap() – maps files or anonymous memory. malloc() / free() – heap allocation. sbrk() – legacy heap growth.

Simple fork demonstration:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    pid_t pid = fork();
    if (pid < 0) { perror("fork"); exit(1); }
    else if (pid == 0) {
        printf("Child process
");
        exit(0);
    } else {
        printf("Parent process
");
    }
    return 0;
}

Linux Memory Management Tuning

Performance Optimisation

Adjust /proc/sys/vm/* parameters (e.g., swappiness, min_free_kbytes) to control swap behaviour and free memory thresholds.

Increase mmap cache size via vm.max_map_count.

Use slab allocators or per‑CPU caches to reduce allocation overhead.

Leak Detection & Debugging

valgrind

– runtime memory‑error detector.

AddressSanitizer (compile with -fsanitize=address). gdb – inspect memory state during debugging.

Fragmentation Reduction

Buddy system merges free blocks to minimise external fragmentation.

Memory pools pre‑allocate objects of the same size.

Enable zswap or zram for page compression.

Align allocations to natural boundaries (e.g., posix_memalign).

Practical Applications

Server & Embedded Use‑Cases

Linux memory management scales from high‑load servers to resource‑constrained embedded devices, providing isolation, efficient paging and optional compression.

Driver Development

Character drivers use kmalloc() or vmalloc() for buffers.

Network drivers allocate page‑aligned buffers with alloc_pages().

Block drivers employ mmap() for direct I/O.

Video drivers allocate large contiguous buffers via vmalloc() or dma_alloc_coherent().

System Optimisation Techniques

Memory compression (e.g., zswap) to reduce RAM pressure.

Transparent Huge Pages (THP) to lower page‑table overhead.

Swap configuration – size and priority tuning.

This summary captures the essential concepts, APIs and tuning strategies for Linux memory management without promotional or unrelated content.