Why DMA Is More Than Just Devices Bypassing the CPU

Many newcomers to DMA first remember the simple statement “devices access memory without going through the CPU.” While not wrong, this view quickly becomes insufficient when dealing with real drivers for NICs, NVMe, GPUs, or audio/video capture, because DMA actually reshapes the entire system’s data movement.

Without DMA, data exchange follows Programmed I/O (PIO): Device → CPU → Memory for reads and Memory → CPU → Device for writes. The CPU becomes a “data mover,” wasting cycles on repetitive load/store operations that should be reserved for control, scheduling, protocol handling, and business logic. DMA extracts this task, leaving the CPU to:

Configure the transfer parameters;

Trigger the DMA engine;

Resume normal processing after the transfer completes.

Consequently, DMA is not merely a faster memcpy; it frees the CPU, excels at large, contiguous transfers, and enables parallelism between CPU work and I/O.

Consider a typical NIC receive path: the driver pre‑allocates a ring of buffers, programs their DMA addresses into the NIC, and then the NIC writes incoming packets directly into those buffers. The CPU is notified via interrupt or polling only after the write finishes, at which point it parses packet headers and passes data up the stack. This clear division of labor illustrates why understanding DMA in a concrete scenario clarifies its purpose.

When a system includes a cache, the participants expand to CPU, Cache, Memory, and Device. Visibility problems arise because the device sees Memory while the CPU often reads from Cache. If these views are not synchronized, bugs appear:

CPU writes new data, but the cache line is not flushed; the device reads stale memory.

Device writes new data to memory, but the CPU still reads a cached old value.

These symptoms—old data seen by the device, stale data read by the CPU, CRC errors, dropped packets—trace back to the same root cause: the data visible to CPU is not necessarily the data visible to the device.

Hardware platforms differ in how they handle this coherence. Some provide “DMA coherent” support, automatically keeping CPU and device views consistent. Others are “DMA non‑coherent,” requiring drivers to explicitly flush caches before a DMA read and invalidate caches after a DMA write. Embedded SoCs such as ARM or RISC‑V often fall into the latter category, making driver development more error‑prone.

Linux offers a DMA API (e.g., dma_alloc_coherent(), dma_map_single(), dma_unmap_single(), dma_sync_single_for_cpu(), dma_sync_single_for_device()) that abstracts the ownership and visibility transition of a memory region between CPU and device, rather than being a mere collection of address‑translation or cache‑flush helpers.

Another subtle point is that a DMA address is not the same pointer the CPU uses. Physical address translation, IOMMU mappings, or platform‑specific remapping may be involved, so drivers must obtain a device‑usable address even when they already have a CPU buffer.

In summary, the real difficulty of DMA lies not in moving bytes but in understanding when and to whom the data is visible across CPU, cache, memory, and device. Once this perspective is clear, many seemingly mysterious DMA bugs become straightforward to diagnose.