Tag

Sequential I/O

1 views collected around this technical thread.

High Availability Architecture

Dec 17, 2020 · Big Data

How Kafka Achieves Million‑Level TPS: Sequential Disk I/O, MMAP, Zero‑Copy, and Batch Processing

This article explains how Kafka attains million‑level transactions per second by using sequential disk reads/writes, memory‑mapped files, DMA‑based zero‑copy transfers, and batch data transmission, detailing each technique and its impact on throughput and latency.

Big DataKafkaMMAP

0 likes · 11 min read

How Kafka Achieves Million‑Level TPS: Sequential Disk I/O, MMAP, Zero‑Copy, and Batch Processing

macrozheng

Dec 15, 2020 · Big Data

How Kafka Achieves Million‑TPS Through Sequential I/O, MMAP, and Zero‑Copy

Kafka can sustain millions of transactions per second by writing data sequentially to disk, leveraging memory‑mapped files, employing zero‑copy DMA transfers, and batching messages, each technique reducing I/O overhead and CPU involvement, which together enable its high‑throughput performance in big‑data pipelines.

Big DataKafkaMMAP

0 likes · 11 min read

How Kafka Achieves Million‑TPS Through Sequential I/O, MMAP, and Zero‑Copy