Mastering Linux CMA: How the Contiguous Memory Allocator Solves Fragmentation

Overview

Linux kernels allocate memory in pages. Large contiguous physical blocks are hard to obtain because repeated allocations and frees cause fragmentation. Devices that lack an IOMMU (e.g., cameras, video codecs, DMA engines) require such contiguous blocks. The Contiguous Memory Allocator (CMA) reserves a region of physical memory at boot time and can reclaim pages from that region on demand, providing a reliable pool for contiguous allocations.

Design of CMA

What is CMA?

CMA is an extension of the kernel’s buddy allocator. It creates a dedicated pool of physically contiguous pages that can be used by drivers that need non‑scatter‑gather memory or that benefit from high cache‑hit rates. The pool is marked as MIGRATE_CMA so that normal allocations can temporarily use it, but the pool can be reclaimed when a contiguous request arrives.

How CMA Works

Page selection and isolation : When a driver requests a contiguous block, CMA identifies the required page‑block range and changes its migration type from MIGRATE_CMA to MIGRATE_ISOLATE. The buddy system will no longer allocate pages from that range.

Page migration : Pages that are currently in use inside the selected range are moved elsewhere using the kernel’s migrate_pages helper. Data is copied and page tables are updated.

Allocation : After migration the selected range becomes a free contiguous area. CMA returns the start address (both virtual and DMA address) to the requester.

Cleanup : CMA restores the migration type to MIGRATE_CMA and updates its bitmap so the region can be reused later.

Key Data Structure

struct cma {
    unsigned long base_pfn;      // first PFN of the CMA region
    unsigned long count;         // number of pages in the region
    unsigned long *bitmap;        // allocation bitmap (one bit per order_per_bit pages)
    unsigned int order_per_bit;   // pages represented by each bitmap bit (2^order)
    struct mutex lock;           // protects concurrent access
#ifdef CONFIG_CMA_DEBUGFS
    struct hlist_head mem_head;   // debugfs list head
    spinlock_t mem_head_lock;    // protects the list
#endif
    const char *name;            // region name
};

Initialization Flow

CMA regions can be defined either in the device‑tree or via a kernel command‑line parameter. During early boot the kernel parses these definitions and calls dma_contiguous_reserve() to create the CMA area.

Device‑tree example (256 MiB region) :

reserved-memory {
    #address-cells = <1>;
    #size-cells = <1>;
    ranges;
    cma_region: cma@0 {
        compatible = "shared-dma-pool";
        reusable;
        size = <0x10000000>;   // 256 MiB
        alignment = <0x100000>; // 1 MiB alignment
    };
};

Kernel command‑line example :

cma=64M@0x38000000   # reserve 64 MiB at physical address 0x38000000

Allocation API

Drivers use the DMA contiguous API to allocate and free CMA memory.

void *dma_alloc_from_contiguous(struct device *dev,
                               size_t size,
                               dma_addr_t *dma_handle,
                               gfp_t gfp_mask);

void dma_free_from_contiguous(struct device *dev,
                              size_t size,
                              void *vaddr,
                              dma_addr_t dma_handle);

Example: allocate 1 MiB

#include <linux/device.h>
#include <linux/dma-mapping.h>

struct device *dev = get_device();
size_t size = 1024 * 1024;          // 1 MiB
dma_addr_t dma_handle;
void *buf = dma_alloc_from_contiguous(dev, size, &dma_handle, GFP_KERNEL);
if (buf) {
    /* use the buffer */
    dma_free_from_contiguous(dev, size, buf, dma_handle);
}

The gfp_mask must match the allocation context (e.g., GFP_ATOMIC in interrupt handlers, GFP_KERNEL in process context).

Configuration

Kernel boot parameter

The generic syntax is cma=size[@start-end]. Example:

cma=256M@3G-4G   # reserve 256 MiB between 3 GiB and 4 GiB

This method is convenient for systems where the memory layout is fixed.

Device‑tree configuration

reserved-memory {
    #address-cells = <2>;
    #size-cells = <2>;
    ranges;
    cma_region: cma@80000000 {
        compatible = "shared-dma-pool";
        reusable;
        linux,cma-default;
        reg = <0x0 0x80000000 0x0 0x40000000>; // 1 GiB at 0x80000000
    };
};

Key properties:

compatible = "shared-dma-pool" – tells the kernel this node is a CMA pool.

reusable – allows the pool to be used by the buddy allocator when idle.

linux,cma-default – marks the region as the default CMA area when multiple pools exist.

Debugging CMA

Enable debug support

Set the following options in the kernel configuration and rebuild:

CONFIG_CMA_DEBUG=y
CONFIG_CMA_DEBUGFS=y

After boot, the debugfs directory /sys/kernel/debug/cma/ contains one sub‑directory per CMA region (e.g., cma-0).

Useful debugfs files

alloc : write a number of pages to allocate (e.g., echo 1024 > alloc allocates 4 MiB).

base_pfn : shows the starting PFN of the region.

size : total number of pages.

used : currently allocated pages (in KiB).

bitmap : raw bitmap of page usage; view with hexdump -C bitmap.

Example workflow :

# List CMA regions
ls /sys/kernel/debug/cma/
# Allocate 1024 pages (4 MiB) on region cma-0
echo 1024 > /sys/kernel/debug/cma/cma-0/alloc
# Verify usage
cat /sys/kernel/debug/cma/cma-0/used
# Release all pages
echo 0 > /sys/kernel/debug/cma/cma-0/alloc

Common problems

Allocation failures : Check size and used. If the pool is exhausted, inspect bitmap for fragmentation. Trigger kernel compaction with echo 1 > /proc/sys/vm/compact_memory to coalesce free pages.

Migration errors : Kernel logs such as cma: migrate_pages failed, page 0x… is locked indicate a page is pinned. Identify the locking process (e.g., ps -eLo pid,cmd | grep mlock) and release the lock or adjust the driver.