Understanding Zero‑Copy I/O: DMA, Sendfile, mmap and Direct I/O
This article explains how zero‑copy techniques such as DMA, sendfile, mmap and Direct I/O reduce the four data copies and context switches normally required for disk‑to‑network transfers, improving performance for systems like Kafka and MySQL while outlining their trade‑offs.
When an application reads data from disk and sends it over the network, the operating system traditionally performs four data copies and four context switches: CPU moves data between disk, page cache, user buffers, and socket buffers, and switches between user and kernel modes.
DMA (Direct Memory Access) offloads the actual data movement to a dedicated controller, allowing the CPU to issue only control signals, thereby cutting the number of copies and context switches.
Zero‑copy is a design principle where the CPU does not fully copy data between memory regions; instead, mechanisms such as sendfile , mmap , splice and Direct I/O achieve this by keeping data in kernel space or bypassing the page cache.
sendfile combines DMA with file‑descriptor passing, reducing two copies and two context switches, and replaces the separate read and write system calls with a single call. It works best when data can be sent directly from page cache to the NIC, but cannot be used if the data must be processed (e.g., encryption).
mmap maps a file directly into the process address space, eliminating the read copy; writes operate on the kernel buffer directly, achieving zero copy for read‑only scenarios.
Direct I/O bypasses the page cache entirely, moving data between user buffers and the device via DMA. It guarantees that successful writes are persisted to disk, but requires page‑pinning and careful buffer management, and may still need fsync for metadata.
These techniques are employed in real‑world systems: Kafka uses mmap for log persistence and sendfile for delivering messages, while databases such as MySQL adopt Direct I/O and other zero‑copy methods for high‑throughput workloads.
Overall, Linux provides multiple zero‑copy strategies—reducing user/kernel copies, bypassing the kernel, or optimizing buffer transfers—each with specific advantages and limitations.
Architects' Tech Alliance
Sharing project experiences, insights into cutting-edge architectures, focusing on cloud computing, microservices, big data, hyper-convergence, storage, data protection, artificial intelligence, industry practices and solutions.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.