Netty 4.2 IoUring Zero-Copy Send: Two CQEs Explained

1. What Is Zero-Copy?

Zero-copy in network transmission is often associated with Kafka, which uses Linux sendfile to move data from disk directly to the NIC within kernel space, avoiding copies between kernel and user space. The splice system call provides a more generic mechanism supporting file→socket, socket→file, socket→socket, and file→file transfers.

1.1 The Awkward MSG_ERRQUEUE

Can user-space to kernel-space data transfer be zero-copy? Yes. In 2017, Google engineer Willem de Bruijn proposed an application-layer socket zero-copy send mechanism, merged into Linux 4.14. However, it saw little adoption because of semantic asymmetry : the era relied on synchronous I/O (epoll is non-blocking synchronous I/O), and the abrupt addition of the asynchronous MSG_ZEROCOPY protocol required listening to MSG_ERRQUEUE for callbacks, adding complexity.

1.2 Unified IoUring Async Interface

By 2021, io_uring was widely accepted (at least for disk I/O). Facebook engineer Pavel Begunkov integrated asynchronous notification into the unified CQE mechanism, eliminating MSG_ERRQUEUE complexity, merged into Linux 6.10.

Submit one SQE for zero-copy send, receive two CQEs.

First CQE: I/O completion semantics

Almost identical to a regular send operation.

Generated when the kernel enqueues data to the socket send queue.

The kernel may still be transmitting, so the buffer remains referenced.

Second CQE: Buffer release notification

Generated when the driver or protocol stack confirms the user buffer is fully sent and no longer accessed.

Carries the special flag IORING_CQE_F_NOTIF.

Associated with a tx_ctx_id identifying which buffer group is reusable.

2. Submitting Zero-Copy Send SQE

Netty 4.2 submits zero-copy send events in IoUringStreamUnsafe subclass IoUringSocketUnsafe: scheduleWriteSingle sends a single buffer. scheduleWriteMultiple sends multiple buffers via IovArray.

Core logic submits IORING_OP_SEND_ZC / IORING_OP_SENDMSG_ZC SQE:

@Override
protected int scheduleWriteSingle(Object msg) {
  assert writeId == 0;

  if (IoUring.isSendZcSupported() && msg instanceof ByteBuf) {
    ByteBuf buf = (ByteBuf) msg;
    int length = buf.readableBytes();
    if (((IoUringSocketChannelConfig) config()).shouldWriteZeroCopy(length)) {
      long address = IoUring.memoryAddress(buf) + buf.readerIndex();
      IoUringIoOps ops = IoUringIoOps.newSendZc(fd().intValue(), address, length,
          0, nextOpsId(), 0);
      byte opCode = ops.opcode();
      writeId = registration().submit(ops);
      writeOpCode = opCode;
      if (writeId == 0) {
        return 0;
      }
      return 1;
    }
    // Should not use send_zc, just use normal write.
  }
  return super.scheduleWriteSingle(msg);
}

2.1 Should Zero-Copy Send Be Used?

boolean shouldWriteZeroCopy(int amount) {
  // This can reduce one read operation on a volatile field.
  int threshold = this.getWriteZeroCopyThreshold();
  return threshold != DISABLE_WRITE_ZERO_COPY && amount >= threshold;
}

The decision is based on packet size . Default is DISABLE_WRITE_ZERO_COPY (disabled). For small packets (<4KB), memory copy cost is low, while zero-copy incurs pinning user memory pages, maintaining sendzc context and notification queues. Hence, in "small packet, high frequency" scenarios, zero-copy send can degrade performance .

3. Handling Zero-Copy CQEs

The entry for handling send completion remains writeComplete in AbstractUringUnsafe, with main logic same as regular send. Special handling for zero-copy is in IoUringSocketUnsafe.handleWriteCompleteZeroCopy.

3.1 No Native.IORING_CQE_F_NOTIF Flag

When CQE flags lack F_NOTIF, it indicates an I/O completion event.

writeId = 0;
writeOpCode = 0;

boolean more = (flags & Native.IORING_CQE_F_MORE) != 0;
if (more) {
  // This is the result of send_sz or sendmsg_sc but there will also be another notification
  // which will let us know that we can release the buffer(s). In this case let's retain the
  // buffer(s) once and store it in an internal queue. Once we receive the notification we will
  // call release() on the buffer(s) as it's not used by the kernel anymore.
  if (zcWriteQueue == null) {
    zcWriteQueue = new ArrayDeque<>(8);
  }
}
if (res >= 0) {
  if (more) {
    // Loop through all the buffers that were part of the operation so we can add them to our
    // internal queue to release later.
    do {
      ByteBuf currentBuffer = (ByteBuf) channelOutboundBuffer.current();
      assert currentBuffer != null;
      zcWriteQueue.add(currentBuffer);
      currentBuffer.retain();
      int readable = currentBuffer.readableBytes();
      int skip = Math.min(readable, res);
      currentBuffer.skipBytes(skip);
      channelOutboundBuffer.progress(readable);
      if (readable <= res) {
        boolean removed = channelOutboundBuffer.remove();
        assert removed;
      }
      res -= readable;
    } while (res > 0);
    // Add the marker so we know when we need to stop releasing
    zcWriteQueue.add(ZC_BATCH_MARKER);
  } else {
    // We don't expect any extra notification, just directly let the buffer be released.
    channelOutboundBuffer.removeBytes(res);
  }
  return true;
} else {
  // Ignore error cases
}

Typically IORING_CQE_F_MORE is set ( more == true), so the implementation places I/O-completed buffers into a temporary queue zcWriteQueue to await the subsequent buffer release notification .

When would more be false? The code shows direct buffer release. This occurs when zero-copy send falls back to regular send because the kernel determines conditions don't allow zero-copy (specific conditions not expanded here).

3.2 With Native.IORING_CQE_F_NOTIF Flag

When CQE flags include F_NOTIF, it indicates a buffer release notification event.

if (zcWriteQueue != null) {
  for (;;) {
    Object queued = zcWriteQueue.remove();
    assert queued != null;
    if (queued == ZC_BATCH_MARKER) {
      // Done releasing the buffers of the zero-copy batch.
      break;
    }
    // The buffer can now be released.
    ((ByteBuf) queued).release();
  }
}

The second event's action is straightforward: release the buffers.

References

Kafka efficiency design:

https://docs.confluent.io/platform/6.2/kafka/design.html#efficiency

Linux MSG_ZEROCOPY documentation: https://docs.kernel.org/networking/msg_zerocopy.html io_uring path to zero-copy I/O (kernel-recipes):

https://archives.kernel-recipes.org/document/io_uring-path-to-zerocopy-i-o/