Why Is Kafka So Fast? Uncover the 4 Core Performance Secrets

Kafka's high performance is a frequent interview topic; this article highlights four core reasons.

1. Page Cache

Kafka writes to the operating system's page cache (os cache), allowing data to be written to memory first; the OS later flushes it to disk, effectively turning disk writes into memory writes.

2. Sequential Disk Writes

Kafka appends data to the end of files, avoiding random writes. Sequential writes eliminate seek time and reduce rotational latency, improving I/O efficiency.

Seek time – moving the head to the correct track.

Rotational latency – waiting for the sector to rotate under the head.

Data transfer – moving data between disk, memory, and network.

3. Zero Copy

Traditional data flow copies data from memory to the Kafka process, then to the socket buffer, incurring extra copies.

Kafka uses Linux's sendfile to transfer data directly from the OS cache to the socket, skipping user‑space copies.

4. Partitioning, Segmentation & Indexing

Messages are stored per‑topic and further divided into partitions, each mapped to a directory on the filesystem. Partitions are split into segments, and Kafka creates an index (.index) for each segment.

This design enables efficient reads and parallel processing.