BestHub
Sign in
Tagged articles
10 articles
Page 1 of 1
Architect Chen
Architect Chen
Sep 10, 2025 · Big Data

How Kafka Achieves Million‑Message Throughput: Sequential Writes, Page Cache, Batching & Zero‑Copy

The article explains how Kafka attains high‑throughput performance by using sequential disk writes, leveraging the OS page cache, employing producer and consumer batching with configurable parameters, and utilizing zero‑copy sendfile to minimize CPU and memory overhead, enabling stable million‑message per second rates.

BatchingBig DataHigh Throughput
0 likes · 5 min read
How Kafka Achieves Million‑Message Throughput: Sequential Writes, Page Cache, Batching & Zero‑Copy
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Aug 20, 2025 · Backend Development

How Kafka Achieves High‑Performance Storage: Log Segments, Indexes, and Retention

This article explains Kafka's storage architecture, including its partition‑based log files, sequential append writes, log segment management, index files, and configurable time‑ and size‑based retention policies that together enable ultra‑high write throughput while controlling disk usage.

KafkaRetention PolicySequential Write
0 likes · 4 min read
How Kafka Achieves High‑Performance Storage: Log Segments, Indexes, and Retention
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jun 4, 2024 · Big Data

Why Kafka Can Achieve Million‑Message‑Per‑Second Throughput: Disk Sequential Write, Zero‑Copy, Page Cache, and Memory‑Mapped Files

The article explains how Kafka attains ultra‑high write throughput by leveraging disk sequential writes, zero‑copy data transfer, operating‑system page cache, and memory‑mapped files, detailing each technique’s impact on latency, CPU usage, and overall performance.

Sequential WriteZero Copybig-data
0 likes · 5 min read
Why Kafka Can Achieve Million‑Message‑Per‑Second Throughput: Disk Sequential Write, Zero‑Copy, Page Cache, and Memory‑Mapped Files
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Apr 6, 2024 · Big Data

Deep Dive into Kafka’s Underlying Mechanisms: Sequential Writes, Sparse Indexing, Segment Storage, and Replication

This article explores Apache Kafka’s core storage architecture, explaining how sequential append‑only writes, sparse indexing, segmented log files, and a leader‑based replication mechanism together enable high‑throughput, reliable, and scalable event streaming for massive data workloads.

Big DataEvent StreamingKafka
0 likes · 11 min read
Deep Dive into Kafka’s Underlying Mechanisms: Sequential Writes, Sparse Indexing, Segment Storage, and Replication
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Aug 28, 2022 · Backend Development

Why Is Kafka So Fast? Uncover the 4 Core Performance Secrets

This article explains the four key techniques—page‑cache usage, sequential disk writes, zero‑copy transfers, and partitioned segment indexing—that enable Kafka to achieve exceptionally high write performance, detailing how each mechanism reduces latency and maximizes throughput.

Backend DevelopmentKafkaPartitioning
0 likes · 5 min read
Why Is Kafka So Fast? Uncover the 4 Core Performance Secrets
Su San Talks Tech
Su San Talks Tech
Nov 4, 2021 · Backend Development

How RocketMQ’s Storage Architecture Powers High Throughput and Low Latency

This article explains RocketMQ’s disk‑based storage design—including CommitLog, ConsumeQueue, and Index files—detailing sequential writes, memory‑mapped I/O, flexible flush strategies, and memory‑level read/write separation that together enable unlimited backlog, high throughput, and low latency messaging.

Memory Mapped FilesRocketMQSequential Write
0 likes · 13 min read
How RocketMQ’s Storage Architecture Powers High Throughput and Low Latency
Selected Java Interview Questions
Selected Java Interview Questions
Jul 11, 2021 · Big Data

Why Kafka’s I/O Performance Is So High

Kafka’s I/O efficiency stems from sequential data writes, zero‑copy reads using sendfile, batch compression of messages, and a dual‑threaded batch producer that together minimize random disk access and data copying, dramatically speeding up both reading and writing operations.

Batch CompressionIO performanceSequential Write
0 likes · 4 min read
Why Kafka’s I/O Performance Is So High