How Zero‑Copy, DMA, and RDMA Supercharge Data Transfer Performance

Part 1: Zero‑Copy Technology

In today’s data‑intensive world, system performance hinges on efficient data movement. Zero‑copy (zero‑copy) is a hidden technique that lets data travel between disk, kernel buffers, user buffers, and network buffers with minimal CPU copying, dramatically accelerating throughput.

1.1 Traditional Data Copy Challenges

When a file is downloaded, data traditionally moves from disk to kernel buffer, then to user space, and finally to the network socket, incurring multiple CPU‑driven copies and context switches that waste CPU cycles and throttle speed.

CPU switches from user mode to kernel mode on a read() call.

Disk data is DMA‑copied into the kernel buffer.

Kernel buffer is copied into the user buffer (CPU‑mediated).

CPU returns to user mode after read() completes.

CPU switches to kernel mode on a write() call.

User buffer is copied into the socket buffer (CPU‑mediated).

DMA copies socket buffer data to the NIC.

CPU returns to user mode after write() completes.

Four context switches and four copies (two DMA, two CPU) occur per transfer, limiting throughput in high‑concurrency servers.

1.2 Definition and Core Idea of Zero‑Copy

Zero‑copy does not eliminate all copying; it reduces redundant copies between user and kernel space and often removes CPU‑mediated moves altogether, lowering CPU usage and latency. Its goals are:

Reduce the number of data copies.

Reduce context‑switch overhead by cutting system‑call count.

Avoid redundant memory allocation in user space.

For example, Linux’s sendfile can move data directly from the kernel cache to a socket buffer without touching user space.

Part 2: DMA – The Hardware Backbone of Zero‑Copy

2.1 What Is RDMA?

Remote Direct Memory Access (RDMA) lets a computer read or write memory on a remote machine without involving the CPU or OS stack, dramatically lowering latency and CPU load. It is essential for high‑performance computing, data centers, and cloud environments.

2.2 How DMA Works

Direct Memory Access (DMA) enables hardware devices (e.g., disk controllers, NICs) to transfer data directly to or from memory without CPU intervention, freeing the CPU for other work.

Initialization: CPU programs the DMA controller with source address, destination address, and length.

Request: Device signals the DMA controller that data is ready.

Bus arbitration: DMA controller obtains bus control from the CPU.

Transfer: DMA moves data between device and memory autonomously.

Completion: DMA raises an interrupt; CPU handles post‑transfer tasks.

In Linux, DMA underpins zero‑copy paths such as sendfile, splice, and RDMA.

2.3 Role of DMA in Zero‑Copy

When a NIC receives a packet, DMA can place it directly into the kernel buffer, bypassing user‑space copies. Similarly, disk reads can be DMA‑directed into the kernel’s PageCache, allowing the CPU to perform other work while the transfer proceeds.

Part 3: PageCache – Software Accelerator for Zero‑Copy

3.1 How PageCache Works

PageCache is Linux’s in‑memory cache for disk pages (typically 4 KB each). On a read, the kernel first checks PageCache; a hit returns data from memory instantly, avoiding a slow disk I/O. On a miss, the kernel issues a DMA read to fill PageCache, then serves the request.

Writes use a write‑back policy: data is first written to PageCache and marked dirty. The kernel later flushes dirty pages to disk based on timers, memory pressure, dirty‑page thresholds, explicit sync calls, or file close.

3.2 Synergy Between PageCache and Zero‑Copy

Zero‑copy system calls such as sendfile read data from PageCache and copy it directly to the socket buffer, eliminating a user‑space copy. High‑performance servers like Nginx exploit this path to serve static files with minimal CPU overhead.

Part 4: Implementations of Zero‑Copy

4.1 mmap + write

Mapping a file into the process address space with mmap lets the application access file data as memory. A subsequent write to a socket copies data from the kernel’s PageCache to the socket buffer, avoiding an extra user‑space copy.

Call mmap to map the file.

Kernel reads the file into PageCache (DMA).

Call write to move PageCache data to the socket buffer.

DMA transfers the socket buffer to the NIC.

4.2 sendfile

The sendfile syscall (introduced in Linux 2.1) moves data from a file directly to a socket. In Linux 2.4 it adopted scatter‑gather DMA, achieving true zero‑copy without any CPU copy.

4.3 splice and tee

splice

creates a kernel‑space pipe between two file descriptors, moving data without user‑space copies. It is ideal for forwarding data between sockets or between a file and a socket.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <assert.h>
#include <errno.h>

int main(int argc, char **argv) {
    if (argc <= 2) {
        printf("usage: %s ip port
", basename(argv[0]));
        return 1;
    }
    const char *ip = argv[1];
    int port = atoi(argv[2]);
    int sock = socket(AF_INET, SOCK_STREAM, 0);
    assert(sock >= 0);
    int reuse = 1;
    setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
    struct sockaddr_in address;
    bzero(&address, sizeof(address));
    address.sin_family = AF_INET;
    address.sin_port = htons(port);
    inet_pton(AF_INET, ip, &address.sin_addr);
    int ret = bind(sock, (struct sockaddr*)&address, sizeof(address));
    assert(ret != -1);
    ret = listen(sock, 5);
    assert(ret != -1);
    struct sockaddr_in client;
    socklen_t client_addrlength = sizeof(client);
    int connfd = accept(sock, (struct sockaddr*)&client, &client_addrlength);
    if (connfd < 0) {
        printf("errno is: %s
", strerror(errno));
    } else {
        int pipefd[2];
        ret = pipe(pipefd);
        assert(ret != -1);
        ret = splice(connfd, NULL, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
        assert(ret != -1);
        ret = splice(pipefd[0], NULL, connfd, NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
        assert(ret != -1);
        close(connfd);
    }
    close(sock);
    return 0;
}

tee duplicates data from one pipe to another without consuming the source, useful for logging or fan‑out scenarios.

Part 5: Application Scenarios

5.1 Network Transfer

Web servers such as Nginx use sendfile to serve static files directly from PageCache to the network socket, reducing CPU load and increasing concurrent throughput. CDNs also rely on zero‑copy to stream video and other large assets efficiently.

5.2 File Processing

Zero‑copy accelerates large‑file reads/writes via mmap or sendfile , and speeds up database backups by moving data from disk to backup storage without extra copies.

Part 6: Zero‑Copy in Java

6.1 Java NIO mmap Support

public class MmapTest {
    public static void main(String[] args) {
        try {
            FileChannel readChannel = FileChannel.open(Paths.get("./jay.txt"), StandardOpenOption.READ);
            MappedByteBuffer data = readChannel.map(FileChannel.MapMode.READ_ONLY, 0, 1024 * 1024 * 40);
            FileChannel writeChannel = FileChannel.open(Paths.get("./siting.txt"), StandardOpenOption.WRITE, StandardOpenOption.CREATE);
            writeChannel.write(data);
            readChannel.close();
            writeChannel.close();
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }
}

6.2 Java NIO sendfile Support

public class SendFileTest {
    public static void main(String[] args) {
        try {
            FileChannel readChannel = FileChannel.open(Paths.get("./jay.txt"), StandardOpenOption.READ);
            long len = readChannel.size();
            long position = readChannel.position();
            FileChannel writeChannel = FileChannel.open(Paths.get("./siting.txt"), StandardOpenOption.WRITE, StandardOpenOption.CREATE);
            readChannel.transferTo(position, len, writeChannel);
            readChannel.close();
            writeChannel.close();
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }
}

6.3 Case Study: Zero‑Copy File Server

A Java NIO server can accept a client, open a file, and use FileChannel.transferTo to stream the file directly to the client socket, eliminating user‑space copies.

import java.io.FileInputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.FileChannel;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;

public class ZeroCopyFileServer {
    public static void main(String[] args) throws IOException {
        int port = 8888;
        ServerSocketChannel server = ServerSocketChannel.open();
        server.bind(new InetSocketAddress(port));
        while (true) {
            SocketChannel client = server.accept();
            String filePath = "your_file_path_here"; // replace with actual path
            try (FileInputStream fis = new FileInputStream(filePath);
                 FileChannel fileChannel = fis.getChannel()) {
                long transferred = fileChannel.transferTo(0, fileChannel.size(), client);
                System.out.println("Transferred " + transferred + " bytes.");
            }
            client.close();
        }
    }
}