Master Linux Memory Management: From Physical Pages to Virtual Addressing

1. Linux Memory Management Overview

Linux memory management handles allocation, release, mapping, swapping, and compression of system memory. The memory is divided into kernel space, user space, caches, swap partitions, etc., aiming to maximize utilization while ensuring stability and reliability.

1.1 What Is Memory Management

Memory management is the mechanism responsible for managing system memory resources, including allocation, release, mapping, and virtual memory management. It improves resource utilization and application performance.

Linux, as an open‑source OS, offers flexible and customizable memory management, essential for system administrators and developers.

1.2 Importance of Memory Management

Effective memory management prevents system crashes, performance degradation, security issues, and resource waste by ensuring proper allocation, avoiding fragmentation, and protecting process address spaces.

System crashes or hangs : Proper management avoids failures caused by insufficient memory or leaks.

Performance decline : Reduces fragmentation and maintains continuous memory availability.

Security issues : Prevents malicious programs from tampering with memory.

Resource waste : Ensures allocated memory is released promptly.

1.3 Components of Memory Management

The main components are:

Virtual memory management : Maps physical memory to process address spaces, providing isolation and protection.

Physical memory management : Handles allocation, reclamation, and mapping of physical pages.

Page replacement algorithms : Choose pages to evict when physical memory is scarce.

Process address space management : Manages code, data, and stack segments.

Memory protection and access control : Sets page attributes and permissions.

Memory statistics and monitoring : Tracks usage for tuning and troubleshooting.

2. Physical Memory Management

2.1 What Is Physical Memory

Physical memory refers to the actual RAM chips in hardware, organized into addressable units. The kernel reserves physical memory at boot and divides it into fixed‑size pages (typically 4 KB or 8 KB).

2.2 Physical Memory Management Methods

Two main methods are used:

2.2.1 Contiguous Memory Management

Linux employs the Buddy System to allocate and free contiguous blocks, splitting and merging buddies to reduce fragmentation.

2.2.2 Non‑Contiguous Memory Management

Non‑contiguous management uses paging or segmentation. Paging divides memory into fixed‑size pages, allowing virtual‑to‑physical mapping. Segmentation allows variable‑size segments with base and length.

2.3 Functions and Examples for Physical Memory Management

Key kernel functions include:

memblock_init()

: Initialize physical memory blocks.

memblock_reserve()

: Reserve blocks from allocation.

memblock_free()

: Release blocks.

memblock_alloc()

: Allocate blocks.

memblock_find_in_range()

: Find free blocks within a range.

Example kernel module:

<code>#include …
static int __init test_init(void)
{
    unsigned long size = 4096;
    unsigned long *ptr;
    ptr = memblock_alloc(size, PAGE_SIZE);
    if (!ptr) {
        pr_err("Failed to allocate memory\n");
        return -ENOMEM;
    }
    pr_info("Allocated %ld bytes of physical memory at address %p\n", size, ptr);
    return 0;
}
static void __exit test_exit(void)
{
    pr_info("Exiting test module\n");
}
module_init(test_init);
module_exit(test_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("Test module");
</code>

3. Virtual Memory Management

3.1 What Is Virtual Memory

Virtual memory abstracts physical memory, mapping process address spaces to physical pages, typically in 4 KB pages, allowing processes to use more memory than physically available.

3.2 Principles of Virtual Memory Management

The MMU (Memory Management Unit) translates virtual addresses to physical addresses on demand, swapping pages to disk when needed.

3.3 Functions and Examples for Virtual Memory Management

Common system calls:

mmap

: Map a file or device into a process’s address space.

munmap

: Unmap a previously mapped region.

mlock

: Lock a region in RAM to prevent swapping.

mprotect

: Change protection flags of a mapped region.

Example of mapping a file:

<code>#include …
int main(int argc, char *argv[])
{
    int fd = open(argv[1], O_RDONLY);
    if (fd == -1) { perror("open"); exit(EXIT_FAILURE); }
    struct stat sb;
    if (fstat(fd, &sb) == -1) { perror("fstat"); exit(EXIT_FAILURE); }
    char *addr = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(EXIT_FAILURE); }
    printf("%s", addr);
    if (munmap(addr, sb.st_size) == -1) { perror("munmap"); exit(EXIT_FAILURE); }
    close(fd);
    return 0;
}
</code>

4. Memory Allocation and Release

4.1 Overview of Allocation and Release

Allocation provides memory to applications; release returns it to the system. Efficient management avoids performance loss and leaks.

4.2 Allocation and Release Methods

Methods include static allocation, stack allocation, heap allocation (malloc/free), memory‑mapped files, and shared memory.

4.3 Functions and Examples for Allocation and Release

4.3.1 malloc() and free()

Typical usage:

<code>#include …
int main()
{
    int *p = (int*)malloc(sizeof(int));
    if (p == NULL) { printf("Failed to allocate memory!\n"); return -1; }
    *p = 123;
    printf("%d\n", *p);
    free(p);
    return 0;
}
</code>

4.3.2 calloc() and realloc()

<code>#include …
int main()
{
    int *p1 = (int*)calloc(5, sizeof(int));
    if (p1 == NULL) { printf("Failed to allocate memory!\n"); return -1; }
    for (int i = 0; i < 5; i++) printf("%d ", p1[i]);
    printf("\n");
    int *p2 = (int*)realloc(p1, 10 * sizeof(int));
    if (p2 == NULL) { printf("Failed to allocate memory!\n"); return -1; }
    for (int i = 5; i < 10; i++) p2[i] = i * 2;
    for (int i = 0; i < 10; i++) printf("%d ", p2[i]);
    printf("\n");
    free(p2);
    return 0;
}
</code>

4.3.3 mmap() and munmap()

<code>#include …
int main()
{
    int fd = open("file.txt", O_RDONLY);
    if (fd < 0) { perror("open"); exit(EXIT_FAILURE); }
    char *ptr = mmap(NULL, 4096, PROT_READ, MAP_PRIVATE, fd, 0);
    if (ptr == MAP_FAILED) { perror("mmap"); exit(EXIT_FAILURE); }
    printf("%s", ptr);
    if (munmap(ptr, 4096) < 0) { perror("munmap"); exit(EXIT_FAILURE); }
    close(fd);
    return 0;
}
</code>

4.3.4 malloc()

<code>#include …
int main()
{
    int *ptr = (int*)malloc(sizeof(int));
    if (ptr == NULL) { perror("malloc error"); exit(1); }
    *ptr = 123;
    printf("%d\n", *ptr);
    free(ptr);
    return 0;
}
</code>

4.3.5 sbrk()

<code>#include …
int main()
{
    int *ptr1 = (int*)sbrk(sizeof(int));
    if (ptr1 == (void*)-1) { perror("sbrk error"); exit(1); }
    *ptr1 = 123;
    printf("%d\n", *ptr1);
    int *ptr2 = (int*)sbrk(sizeof(int));
    if (ptr2 == (void*)-1) { perror("sbrk error"); exit(1); }
    /* ... */
    return 0;
}
</code>

5. Process Switching and Memory Management

5.1 Overview of Process Switching

Process switching saves the current state and loads the next process’s state, involving context saving, scheduling, and memory mapping.

5.2 Relationship Between Process Switching and Memory Management

During a switch, the kernel saves the current page tables and loads the new process’s tables, ensuring each process accesses its own virtual memory.

5.3 Functions and Examples for Process Switching

5.3.1 fork()

<code>#include …
int main()
{
    int pid = fork();
    if (pid < 0) { perror("fork error"); exit(1); }
    else if (pid == 0) { printf("Child process\n"); exit(0); }
    else { printf("Parent process\n"); }
    return 0;
}
</code>

5.3.2 exec()

<code>#include …
int main()
{
    char *argv[] = {"ls", "-l", NULL};
    execvp("ls", argv);
    perror("exec error");
    exit(1);
}
</code>

5.3.3 mmap()

<code>#include …
int main()
{
    int fd = open("file.txt", O_RDONLY);
    if (fd < 0) { perror("open error"); exit(1); }
    char *ptr = mmap(NULL, 4096, PROT_READ, MAP_PRIVATE, fd, 0);
    if (ptr == MAP_FAILED) { perror("mmap error"); exit(1); }
    printf("%s", ptr);
    if (munmap(ptr, 4096) < 0) { perror("munmap error"); exit(1); }
    close(fd);
    return 0;
}
</code>

5.3.4 malloc()

<code>#include …
int main()
{
    int *p = (int*)malloc(sizeof(int));
    if (!p) { perror("malloc error"); exit(1); }
    *p = 123;
    printf("%d\n", *p);
    free(p);
    return 0;
}
</code>

5.3.5 sbrk()

<code>#include …
int main()
{
    int *p = (int*)sbrk(sizeof(int));
    if (p == (void*)-1) { perror("sbrk error"); exit(1); }
    *p = 123;
    printf("%d\n", *p);
    return 0;
}
</code>

6. Linux Memory Management Tuning

6.1 Performance Tuning

Key areas include memory usage optimization, swap configuration, mapping cache sizing, and allocation algorithm tuning.

6.2 Leak Detection and Debugging

Tools such as Valgrind, AddressSanitizer, GDB, and LeakTracer help identify leaks, double frees, and out‑of‑bounds accesses.

6.3 Fragmentation Management and Optimization

Techniques include the buddy system, memory pools, compression (zswap), transparent huge pages, and proper alignment.

7. Applications of Linux Memory Management

7.1 Application Scenarios

Used in servers, embedded devices, scientific computing, kernel development, and virtualization.

7.2 Use in Driver Development

Drivers allocate buffers with kmalloc/vmalloc, manage pages with alloc_pages, and use memory‑mapped I/O for block devices.

7.3 Use in System Optimization

Techniques like memory compression, reclamation, transparent huge pages, and swap tuning improve performance and stability.

8. Conclusion

The article introduced the concepts, importance, and components of Linux memory management, detailed physical and virtual memory mechanisms, allocation and release functions, process‑switch interactions, tuning methods, leak detection, fragmentation handling, and real‑world applications, providing a comprehensive reference.