Zero‑Copy Techniques in Java: Buffers, MappedByteBuffer, Sendfile, and Netty

Zero‑copy means data does not need to be copied back and forth between user space and kernel space, greatly improving system performance. It is a key optimization in frameworks such as Java NIO, Netty, Kafka, and RocketMQ.

I/O Concepts – All I/O revolves around moving data into or out of buffers. A process issues a read or write request to the OS, which either copies data from kernel buffers to user buffers or fills kernel buffers from storage via DMA.

The traditional read/write flow involves copying data from the kernel read buffer to the user buffer and, for writes, from the user buffer to the kernel socket buffer, which incurs extra memory copies.

Zero‑copy addresses this overhead by allowing the kernel and user space to share the same physical memory, eliminating one or more copies.

Zero‑Copy Methods

mmap + write – memory‑maps a file into the process address space and writes directly.

Sendfile – a system call that transfers data from a file descriptor to a socket without copying to user space.

Virtual Memory – Modern OSes use virtual memory, allowing multiple virtual addresses to map to the same physical page. This enables kernel and user space buffers to share the same memory, so DMA can fill a buffer visible to both.

Java Zero‑Copy

MappedByteBuffer – FileChannel.map() creates a memory‑mapped file. Example:

public class MappedByteBufferTest {
    public static void main(String[] args) throws Exception {
        File file = new File("D://db.txt");
        long len = file.length();
        byte[] ds = new byte[(int) len];
        MappedByteBuffer mappedByteBuffer = new FileInputStream(file).getChannel()
            .map(FileChannel.MapMode.READ_ONLY, 0, len);
        for (int offset = 0; offset < len; offset++) {
            byte b = mappedByteBuffer.get();
            ds[offset] = b;
        }
        Scanner scan = new Scanner(new ByteArrayInputStream(ds)).useDelimiter(" ");
        while (scan.hasNext()) {
            System.out.print(scan.next() + " ");
        }
    }
}

The map() method signature:

public abstract MappedByteBuffer map(FileChannel.MapMode mode, long position, long size) throws IOException;

MapMode can be READ_ONLY, READ_WRITE, or PRIVATE (copy‑on‑write). The implementation obtains a native address, handles OutOfMemoryError with GC retries, and creates either a read‑only or read‑write DirectByteBuffer.

DirectByteBuffer – A subclass of MappedByteBuffer that allocates off‑heap memory, e.g.:

ByteBuffer directByteBuffer = ByteBuffer.allocateDirect(100);

Channel‑to‑Channel Transfer – FileChannel.transferTo() moves data directly between channels without an intermediate buffer:

public class ChannelTransfer {
    public static void main(String[] argv) throws Exception {
        String[] files = new String[1];
        files[0] = "D://db.txt";
        catFiles(Channels.newChannel(System.out), files);
    }
    private static void catFiles(WritableByteChannel target, String[] files) throws Exception {
        for (int i = 0; i < files.length; i++) {
            FileInputStream fis = new FileInputStream(files[i]);
            FileChannel channel = fis.getChannel();
            channel.transferTo(0, channel.size(), target);
            channel.close();
            fis.close();
        }
    }
}

transferTo() takes the start position, byte count, and target channel, allowing kernel‑level data movement without user‑space copies.

Netty Zero‑Copy

Netty provides CompositeChannelBuffer and Slice buffers to avoid extra copies when assembling or splitting messages. Example of CompositeChannelBuffer:

public class CompositeChannelBuffer extends AbstractChannelBuffer {
    private final ByteOrder order;
    private ChannelBuffer[] components;
    private int[] indices;
    private int lastAccessedComponentId;
    private final boolean gathering;
    public byte getByte(int index) {
        int componentId = componentId(index);
        return components[componentId].getByte(index - indices[componentId]);
    }
    ...
}

The buffer holds references to component buffers instead of copying their contents, achieving zero‑copy.

Other Zero‑Copy Uses

RocketMQ writes messages sequentially to a commit log and serves consumers via mmap + write. Kafka also employs Sendfile for network transmission of persisted data.

In summary, zero‑copy in Java leverages virtual memory, memory‑mapped files, direct buffers, and kernel‑level transfer APIs to minimize data copying, improve throughput, and reduce CPU overhead.