How DMA and Zero‑Copy Transform Linux I/O Performance

1. Data Copy and Context Switches

Typical client‑server interactions involve a File.read followed by Socket.send, which under a naïve implementation triggers four memory copies and four user‑kernel context switches: disk → page cache, page cache → socket buffer, kernel → user buffer, and user buffer → socket buffer, plus the corresponding mode switches.

Kafka exemplifies this pattern by reading messages from disk and sending them directly to the NIC.

2. DMA‑Assisted Data Transfer

DMA (Direct Memory Access) uses a dedicated controller to move data between memory and I/O devices without CPU involvement, acting as a co‑processor. It excels when transferring large or fast data streams, or when handling very slow transfers, allowing the CPU to issue control commands only.

DMA replaces the CPU in moving data between disk, page cache, socket buffers, and the NIC, reducing the number of copies and context switches.

However, DMA cannot handle intra‑device copies; the CPU must still copy data between kernel and user spaces when required.

3. Zero‑Copy Techniques

3.1 What Is Zero‑Copy?

Zero‑copy means the CPU does not copy data between memory regions; it only orchestrates the transfer. Data may still be moved by DMA or other hardware, but the CPU is not involved in the actual copy.

3.2 sendfile

sendfile is used when an application reads a file and immediately sends it over the network without processing. It combines DMA with file‑descriptor passing to eliminate two copies and two context switches, reducing the total from four to two.

sendfile requires NICs that support SG‑DMA; otherwise a kernel copy remains.

3.3 mmap

mmap maps a file directly into the process address space, allowing the application to read/write without an explicit read/write system call, thus avoiding a copy between kernel and user space.

3.4 Direct I/O

Direct I/O bypasses the page cache, transferring data directly between user buffers and the storage device via DMA. It guarantees that written data reaches the disk (subject to metadata caching) and that reads come straight from the disk.

Direct I/O requires page‑pinning of user buffers and may need an additional fsync to persist metadata.

4. Real‑World Cases

4.1 Kafka

Kafka persists incoming messages using mmap for sequential disk writes.

Kafka serves messages to consumers with sendfile, avoiding kernel‑user copies and allowing multiple consumers to share the same page cache.

4.2 MySQL

MySQL employs zero‑copy techniques (details in a separate article) to reduce I/O overhead for its storage engine.

5. Summary

DMA offloads memory‑to‑device transfers, allowing the CPU to issue only control commands. Linux provides several zero‑copy strategies—page‑cache‑based (sendfile, mmap, splice) and Direct I/O—that reduce or eliminate data copies and context switches, improving throughput for high‑performance workloads such as Kafka and database systems.