Master Linux Memory Management: From Physical to Virtual Memory Explained

Linux Memory Management Overview

Linux memory management refers to the series of operations for allocating, releasing, mapping, swapping, and compressing system memory. In Linux, memory is divided into regions such as kernel space, user space, cache, and swap partitions. The goal is to maximize usable memory while ensuring system stability and reliability.

1.1 What Is Memory Management

Memory management is the mechanism responsible for handling system memory resources, including allocation, release, mapping, and virtual memory management. It improves resource utilization and application performance.

1.2 Importance of Memory Management

Effective memory management prevents system crashes, performance degradation, security issues, and resource waste by ensuring proper allocation, avoiding fragmentation, and protecting process address spaces.

System crash or hang : Prevents failures caused by insufficient memory or leaks.

Performance decline : Optimizes memory usage to avoid fragmentation.

Security issues : Protects against malicious memory modifications.

Resource waste : Avoids unreleased memory blocks.

1.3 Components of Memory Management

Virtual memory management : Maps physical memory to independent process address spaces.

Physical memory management : Handles allocation, reclamation, and mapping of physical pages.

Page replacement algorithms : Choose pages to evict when physical memory is scarce.

Process address space management : Manages code, data, and stack segments.

Memory protection and access control : Sets page attributes and permissions.

Memory statistics and monitoring : Tracks usage for tuning and troubleshooting.

2 Physical Memory Management

Physical memory management tracks and controls the usage of actual RAM, dividing it into fixed‑size pages (typically 4 KB or 8 KB).

2.1 What Is Physical Memory

Physical memory consists of the hardware chips that store programs and data. The kernel reserves pages for its own use at boot.

2.2 Physical Memory Management Methods

Two main approaches are used:

2.2.1 Contiguous Memory Management

Memory is treated as a continuous address range. Linux implements this with the Buddy System, which splits and merges blocks to reduce fragmentation.

2.2.2 Non‑contiguous Memory Management

Memory is managed in pages (paging) or segments. Paging maps virtual pages to physical pages, while segmentation allows variable‑size segments.

2.3 Functions and Examples for Physical Memory

memblock_init()

: Initialize physical memory blocks. memblock_reserve(): Reserve blocks that cannot be allocated. memblock_free(): Release a block. memblock_alloc(): Allocate a block. memblock_find_in_range(): Find a free block within a range.

Example module that allocates a 4 KB block and prints its address:

#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/mm.h>

static int __init test_init(void)
{
    unsigned long size = 4096;
    unsigned long *ptr;
    ptr = memblock_alloc(size, PAGE_SIZE);
    if (!ptr) {
        pr_err("Failed to allocate memory
");
        return -ENOMEM;
    }
    pr_info("Allocated %ld bytes of physical memory at address %p
", size, ptr);
    return 0;
}

static void __exit test_exit(void)
{
    pr_info("Exiting test module
");
}

module_init(test_init);
module_exit(test_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("Test module");

3 Virtual Memory Management

Virtual memory provides each process with an abstract address space that may be larger than physical RAM. The MMU translates virtual addresses to physical addresses, allowing pages to be swapped to disk when needed.

3.1 What Is Virtual Memory

Virtual memory combines RAM with a portion of disk storage, typically using 4 KB pages. It enables processes to use more memory than physically available.

3.2 Principles of Virtual Memory Management

When a process accesses an unmapped virtual address, the kernel allocates a physical page and updates the page tables. The MMU performs the translation.

3.3 Functions and Examples for Virtual Memory

mmap()

: Map a file or anonymous memory into a process address space. munmap(): Remove a mapping. mlock(): Lock a region in RAM to prevent swapping. mprotect(): Change protection flags of a mapped region.

Example using mmap() to map a file and print its contents:

#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

int main()
{
    int fd = open("file.txt", O_RDONLY);
    if (fd < 0) { perror("open file failed"); exit(1); }
    struct stat st;
    fstat(fd, &st);
    char *addr = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(1); }
    printf("%s", addr);
    munmap(addr, st.st_size);
    close(fd);
    return 0;
}

4 Memory Allocation and Release

Allocation and release are core to Linux memory management. They can be performed via C library functions or kernel APIs.

4.1 Overview

Allocation provides memory for applications; release returns it to the system. Proper management avoids leaks and performance degradation.

4.2 Allocation Methods

Static allocation (compile‑time).

Stack allocation (automatic per function call).

Heap allocation (malloc, calloc, realloc, free).

Memory‑mapped files (mmap).

Shared memory (shmget, mmap with MAP_SHARED).

4.3 Functions and Examples

malloc() / free() example:

#include <stdlib.h>
#include <stdio.h>
int main() {
    int *p = (int*)malloc(sizeof(int));
    if (!p) { printf("Failed to allocate memory!
"); return -1; }
    *p = 123;
    printf("%d
", *p);
    free(p);
    return 0;
}

calloc() / realloc() example:

#include <stdlib.h>
#include <stdio.h>
int main() {
    int *p1 = (int*)calloc(5, sizeof(int));
    if (!p1) { printf("Failed to allocate memory!
"); return -1; }
    for (int i = 0; i < 5; i++) printf("%d ", p1[i]);
    printf("
");
    int *p2 = (int*)realloc(p1, 10 * sizeof(int));
    if (!p2) { printf("Failed to allocate memory!
"); return -1; }
    for (int i = 5; i < 10; i++) p2[i] = i * 2;
    for (int i = 0; i < 10; i++) printf("%d ", p2[i]);
    printf("
");
    free(p2);
    return 0;
}

mmap() / munmap() example:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
int main(int argc, char *argv[]) {
    int fd = open(argv[1], O_RDONLY);
    if (fd < 0) { perror("open"); exit(1); }
    struct stat sb;
    fstat(fd, &sb);
    char *addr = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(1); }
    printf("%s", addr);
    munmap(addr, sb.st_size);
    close(fd);
    return 0;
}

5 Process Switching and Memory Management

During a context switch, the kernel saves the current process's page tables and loads the next process's tables, linking process scheduling with memory management.

5.1 Overview of Process Switching

Switching involves saving CPU state, selecting another runnable process, and restoring its state. It incurs overhead but is essential for multitasking.

5.2 Relationship Between Switching and Memory Management

Each process has its own virtual address space; the kernel must manage page table updates and memory mappings during switches.

5.3 Functions and Examples

fork() example:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main() {
    int pid = fork();
    if (pid < 0) { perror("fork error"); exit(1); }
    else if (pid == 0) { printf("Child process
"); exit(0); }
    else { printf("Parent process
"); }
    return 0;
}

exec() example:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main() {
    char *argv[] = {"ls", "-l", NULL};
    execvp("ls", argv);
    perror("exec error");
    exit(1);
}

sbrk() example (heap expansion):

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main() {
    int *p1 = (int*)sbrk(sizeof(int));
    if (p1 == (void*)-1) { perror("sbrk error"); exit(1); }
    *p1 = 123;
    printf("%d
", *p1);
    int *p2 = (int*)sbrk(sizeof(int));
    if (p2 == (void*)-1) { perror("sbrk error"); exit(1); }
    *p2 = 456;
    printf("%d
", *p2);
    return 0;
}

6 Linux Memory Management Tuning

6.1 Performance Tuning

Key areas include memory usage optimization, swap configuration, mmap cache tuning, and allocation algorithm adjustments.

6.2 Detecting and Debugging Memory Leaks

Tools such as Valgrind, AddressSanitizer, GDB, and LeakTracer help identify unreleased memory, double frees, and out‑of‑bounds accesses.

6.3 Reducing Memory Fragmentation

Techniques include the buddy allocator, memory pools, compression, custom allocators, and proper alignment.

7 Applications of Linux Memory Management

7.1 Use Cases

Server environments – high‑load stability.

Embedded systems – efficient use of limited RAM.

Scientific computing – large data sets and high performance.

Kernel and driver development – requires deep understanding of memory APIs.

Virtualization – managing multiple guest address spaces.

7.2 Memory Management in Driver Development

Character drivers use kmalloc() or vmalloc() for buffers.

Network drivers allocate pages with alloc_pages().

Block drivers employ memory‑mapped I/O for disk buffers.

Video drivers often use vmalloc() for frame buffers.

7.3 Memory Management in System Optimization

Memory compression (e.g., zswap) reduces RAM pressure.

Active reclamation frees unused pages.

Transparent Huge Pages (THP) lower page‑table overhead.

Proper swap sizing balances performance and availability.

8 Conclusion

The article introduced the concept and significance of Linux memory management, detailed physical and virtual memory mechanisms, presented allocation/release functions and code samples, explained the link between process switching and memory, and discussed tuning, leak detection, fragmentation mitigation, and practical applications in drivers and system optimization.