Tagged articles
Sequential Write
10 articles · Page 1 of 1
The article explains how Kafka attains high‑throughput performance by using sequential disk writes, leveraging the OS page cache, employing producer and consumer batching with configurable parameters, and utilizing zero‑copy sendfile to minimize CPU and memory overhead, enabling stable million‑message per second rates.
KafkaPage CacheSequential Write
0 likes · 5 min read

This article explains Kafka's storage architecture, including its partition‑based log files, sequential append writes, log segment management, index files, and configurable time‑ and size‑based retention policies that together enable ultra‑high write throughput while controlling disk usage.
KafkaRetention PolicySequential Write
0 likes · 4 min read

This article explains how RocketMQ achieves high throughput by using sequential disk writes, OS page caching, asynchronous flushing, and zero‑copy techniques, detailing each mechanism and providing code examples. It also discusses the impact on disk seek time and data durability.
Async FlushMessage QueuePage Cache
0 likes · 5 min read

The article explains how Kafka attains ultra‑high write throughput by leveraging disk sequential writes, zero‑copy data transfer, operating‑system page cache, and memory‑mapped files, detailing each technique’s impact on latency, CPU usage, and overall performance.
Page CacheSequential Writebig-data
0 likes · 5 min read

This article explores Apache Kafka’s core storage architecture, explaining how sequential append‑only writes, sparse indexing, segmented log files, and a leader‑based replication mechanism together enable high‑throughput, reliable, and scalable event streaming for massive data workloads.
KafkaReplicationSegment Storage
0 likes · 11 min read

This article explains the four key techniques—page‑cache usage, sequential disk writes, zero‑copy transfers, and partitioned segment indexing—that enable Kafka to achieve exceptionally high write performance, detailing how each mechanism reduces latency and maximizes throughput.
KafkaPage CachePartitioning
0 likes · 5 min read

This article explains RocketMQ’s disk‑based storage design—including CommitLog, ConsumeQueue, and Index files—detailing sequential writes, memory‑mapped I/O, flexible flush strategies, and memory‑level read/write separation that together enable unlimited backlog, high throughput, and low latency messaging.
Memory Mapped FilesRocketMQSequential Write
0 likes · 13 min read

Kafka’s I/O efficiency stems from sequential data writes, zero‑copy reads using sendfile, batch compression of messages, and a dual‑threaded batch producer that together minimize random disk access and data copying, dramatically speeding up both reading and writing operations.
IO performanceSequential Writebatch compression
0 likes · 4 min read

This article explains how Kafka achieves high throughput by using sequential disk writes, memory‑mapped files, batch compression, and zero‑copy sendfile for reads, while also covering data retention policies and the role of offsets in consumer processing.
Data StreamingMemory Mapped FilesSequential Write
0 likes · 10 min read

This article explains how Kafka attains high throughput by using sequential disk writes, memory‑mapped files, sendfile zero‑copy, and batch compression, detailing both write and read path optimizations and their impact on performance.
KafkaMemory Mapped FilesSequential Write
0 likes · 8 min read
