Mastering Linux Memory Allocation: When to Use brk vs mmap

1. The Dual Engines of Memory Allocation: Core Principles of brk and mmap

brk and mmap are the two fundamental system calls that a Linux process uses to obtain memory, each with its own operating model and ideal use cases. Understanding their internals is essential for performance tuning and debugging memory‑related issues.

1.1 brk: Linear Heap‑Expansion Engine

The brk system call moves the program break pointer to enlarge the heap region linearly. New memory is placed directly after the existing heap, similar to extending a plot of land. Physical pages are allocated lazily on first access, which saves memory but can cause fragmentation because memory must be released in reverse order of allocation.

Virtual‑to‑physical binding is delayed until the page is touched.

Heap memory must be released sequentially from high to low addresses.

glibc’s sbrk wraps brk and provides incremental allocation.

1.2 mmap: Independent Mapping Engine

mmap creates a separate virtual‑memory area that is not contiguous with the heap. It can map anonymous memory or files, supports non‑contiguous large allocations, and allows independent release with munmap. Zero‑copy mapping lets a process access file contents as memory, reducing data copies.

Non‑contiguous allocation suitable for large blocks (typically >128 KB).

Zero‑copy improves I/O efficiency, e.g., Kafka index files.

Each mapping can be unmapped independently, avoiding heap fragmentation.

1.3 Key Differences Between brk and mmap

Memory region: brk adjusts the heap; mmap creates an independent area.

Size suitability: brk excels at small (few KB‑tens KB) allocations; mmap handles large allocations.

Release mechanism: brk requires sequential release; mmap allows arbitrary release.

Address continuity: brk yields contiguous addresses; mmap regions may be scattered.

2. Why malloc Chooses Different Strategies Based on Size

2.1 Layout and Fragmentation

brk’s linear heap can become fragmented when middle blocks are freed, whereas mmap’s independent regions keep fragmentation low because each region can be reclaimed independently.

Comparison Dimension

Heap (brk)

File‑Mapping Area (mmap)

Memory release order

Sequential (high address first)

Independent (any region)

Fragmentation level

High

Low

Allocation overhead

Low (pointer move)

Medium (mapping structures)

2.2 System‑Call Overhead and Use Cases

brk costs only a few hundred nanoseconds, making it ideal for high‑frequency small allocations such as linked‑list nodes or temporary buffers. mmap incurs higher overhead (hundreds of nanoseconds to microseconds) but provides stability for large buffers, shared memory, and file‑backed mappings.

2.3 malloc’s Strategy Selection

glibc’s malloc compares the requested size with a threshold (default 128 KB). Requests below the threshold use brk‑based heap expansion with fast‑bin caching; larger requests are satisfied with mmap, which avoids heap fragmentation despite higher setup cost.

3. Practical Guide: How to Use brk and mmap Correctly

3.1 Basic API Examples

(1) brk / sbrk

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main() {
    // Allocate 1024 bytes with sbrk
    char *p = (char *)sbrk(1024);
    if (p == (void *)-1) {
        perror("sbrk");
        return 1;
    }
    for (int i = 0; i < 1024; i++) p[i] = i;
    for (int i = 0; i < 10; i++) printf("%d ", p[i]);
    printf("
");
    // Release the memory
    if (sbrk(-1024) == (void *)-1) {
        perror("sbrk");
        return 1;
    }
    return 0;
}

(2) mmap / munmap

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>

int main() {
    int fd = open("test.txt", O_RDWR);
    if (fd == -1) { perror("open"); return 1; }
    struct stat st;
    if (fstat(fd, &st) == -1) { perror("fstat"); close(fd); return 1; }
    char *data = (char *)mmap(0, st.st_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
    if (data == MAP_FAILED) { perror("mmap"); close(fd); return 1; }
    printf("File content:
%s
", data);
    strcpy(data, "This is a modified content.");
    printf("Modified content:
%s
", data);
    if (munmap(data, st.st_size) == -1) perror("munmap");
    close(fd);
    return 0;
}

3.2 Performance Optimizations

Reuse small objects with memory pools (e.g., thread‑local caches) to avoid frequent brk calls.

For large allocations, use mmap with MAP_POPULATE or huge pages to reduce page‑fault overhead.

Mitigate fragmentation: call mallopt(M_TRIM_THRESHOLD,0) for brk‑based pools and madvise for mmap regions.

3.3 Common Pitfalls

brk can only shrink the heap to the most recent high address; freeing middle blocks creates holes that may remain unusable.

Forgetting munmap leaks virtual address space, which can eventually prevent new mappings.

Improper permission flags (e.g., PROT_NONE) without appropriate signal handling can cause unexpected crashes.

4. Balancing Efficiency and Flexibility in Modern Allocators

Modern allocators such as glibc’s ptmalloc and Google’s tcmalloc combine brk for small‑to‑medium objects and mmap for large objects, leveraging thread‑local caches, fast‑bin lists, and region merging to achieve both speed and low fragmentation. Developers should generally rely on the standard malloc / free interface, which abstracts these decisions, and only resort to direct brk/mmap usage for specialized scenarios like custom memory pools in game engines.