JDK 26 adds HttpRequest.BodyPublishers.ofFileChannel, letting developers read specific file segments directly from a shared FileChannel without copying data into heap memory, dramatically reducing memory usage and simplifying parallel chunk uploads to object storage such as OSS or S3.
The article explains how Nginx sustains one million simultaneous connections by using asynchronous non‑blocking I/O, a robust multi‑process architecture, zero‑copy file transmission, and optimized caching strategies, with concrete configuration examples and performance reasoning.
This article explains the fundamental differences between user mode and kernel mode in Linux, why the separation matters for security and stability, how mode switches occur via system calls, interrupts, and traps, and provides practical code examples and performance‑optimizing techniques such as zero‑copy and asynchronous I/O.
This article explains how Linux Virtual Server (LVS) uses kernel‑space packet forwarding, zero‑copy techniques, IPVS layer‑4 routing, horizontal scaling, and Linux kernel tuning to support million‑level concurrent connections in large‑scale architectures.
This article breaks down the core techniques Nginx uses to handle millions of simultaneous connections, covering its asynchronous non‑blocking event model, master‑worker process architecture, zero‑copy data transfer, and memory‑management optimizations with practical configuration examples.
This article breaks down Kafka's high‑throughput design by detailing how partitioning enables parallelism, sequential disk writes and zero‑copy reduce I/O overhead, and OS page cache maximizes memory efficiency, together allowing millions of messages per second.
This article explains how Kafka’s sequential disk writes, zero‑copy data path, partition‑based parallelism, and configurable broker and partition settings enable linear‑scale throughput that can reach millions of transactions per second in large‑scale streaming systems.
This article explains the fundamentals of Linux memory management, details how shared memory implements zero‑copy inter‑process communication, and provides step‑by‑step code examples of system calls, mmap, sendfile, splice, and synchronization techniques for high‑performance data transfer.
This article explains how Kafka leverages sequential disk I/O, zero‑copy data transfer, OS page cache, and partition‑based parallelism to deliver massive throughput, detailing the underlying mechanisms, performance numbers, and a practical formula for estimating total system capacity.
This article explains the fundamentals of zero‑copy, DMA, PageCache and RDMA, compares them with traditional I/O, describes Linux implementations such as sendfile, mmap+write, splice and Java NIO APIs, and shows practical use‑cases that dramatically reduce CPU load and latency in high‑throughput networking and file handling.
This article walks through five production‑grade Go unsafe techniques—including zero‑copy string conversion, deep struct copying, lock‑free queues, memory‑mapped files, and custom serialization—explaining core concepts, providing benchmark results, and offering a detailed safety checklist to avoid common pitfalls.
LoongCollector, the open‑source cloud‑native collector behind Alibaba Cloud's Simple Log Service, achieves ten‑fold higher throughput, up to 80% lower CPU and memory usage, near‑linear scaling, zero‑copy processing, lock‑free event pools and adaptive concurrency, while guaranteeing enterprise‑grade reliability for petabyte‑scale log and metric ingestion.
mmap, the classic Linux memory‑mapping technique, often surpasses the modern async io_uring in various I/O scenarios by eliminating redundant data copies, reducing system calls, and enabling zero‑copy access, while the article explains its fundamentals, workflow, performance comparisons, practical usage, pitfalls, and code examples.
This article presents a step‑by‑step guide to designing and implementing a Kafka‑class distributed log‑based message queue kernel, covering architecture, sequential writes, sparse indexing, zero‑copy I/O, partitioning, replication, consumer‑group metadata, batch pipelines, crash recovery, and performance benchmarks.
The article explains how Direct Memory Access (DMA) eliminates CPU‑bound data copies, compares traditional I/O with zero‑copy techniques such as mmap and sendfile, and shows how reducing system calls and context switches can double file‑transfer throughput while highlighting the limits of kernel cache for large files.
This article examines the real impact of zero‑copy in Go, clarifies when the standard library already provides it, identifies three scenarios where it truly matters, and warns against common misconceptions and pitfalls that can waste effort without improving performance.
This article explains how Netty leverages Linux epoll, a reactor thread model, zero‑copy techniques, custom memory pools, and careful OS/JVM tuning to enable a single server to handle up to a million simultaneous connections efficiently.
The article explains how Kafka achieves high throughput by writing messages sequentially to disk, leveraging OS page cache and zero‑copy system calls, using partitioned topics for parallelism, batching and compressing records on both producer and broker sides, and employing asynchronous replication with configurable persistence strategies.
This article explains the key techniques Kafka uses to maximize throughput, including sequential disk writes with zero‑copy, partition‑based parallelism, batch processing with compression, and configurable asynchronous replication for balancing performance and reliability.
This tutorial walks through creating a Go‑based internal file transfer service from scratch, covering custom frame protocol design, Protobuf command handling, zero‑copy sendfile, chunked read/write with resume support, CRC32 integrity checks, and performance‑tuning techniques.
This article walks through every stage a network packet undergoes inside the Linux kernel—from hardware interrupt and driver processing, through the sk_buff structures of the TCP/IP stack, to user‑space delivery—while explaining zero‑copy mechanisms like sendfile, splice, mmap and io_uring, and offering concrete tuning commands for optimal performance.
This article explains how Python's memoryview object eliminates costly data copies when slicing large binary or numeric arrays, offering zero‑copy views, reinterpretation via .cast(), and multi‑dimensional slicing, with practical code examples and guidance on when to use it.
This article explores the evolution of Linux I/O models—from blocking and non‑blocking to epoll—and introduces io_uring as a high‑performance asynchronous framework that reduces system‑call overhead, eliminates data copies, and unifies network and disk I/O for modern high‑concurrency applications.
This article explains how Kafka achieves massive throughput by horizontally scaling partitions across brokers, optimizing sequential writes with Linux page cache, employing zero‑copy system calls, and batching messages to reduce I/O and network overhead.
This article explains the core Kafka techniques—sequential disk writes, Linux page cache usage, producer/broker batching with compression, and zero‑copy data transfer via sendfile—that together enable massive concurrency and near‑memory write performance in large‑scale streaming architectures.
This article examines how to achieve low‑latency network packet processing on NVIDIA GPUs by comparing CPU and GPU implementations, exploring memory optimizations, batch strategies, stream concurrency, persistent kernels, and CUDA graphs, and presenting detailed performance measurements for each technique.
The article explains how Kafka attains high‑throughput performance by using sequential disk writes, leveraging the OS page cache, employing producer and consumer batching with configurable parameters, and utilizing zero‑copy sendfile to minimize CPU and memory overhead, enabling stable million‑message per second rates.
The article explains how storage media speed, kernel and user mode separation, and context‑switch overhead affect I/O performance, then details DMA, zero‑copy methods such as mmap + write and sendfile, the role of PageCache, and best practices like async and direct I/O for large‑file transfers.
mmap maps files directly into a process’s virtual memory, eliminating the double‑copy between kernel and user space and reducing costly read/write system calls, which boosts I/O performance, simplifies code, but requires careful handling of address space limits, page faults, and concurrency.
This article explains the performance bottlenecks of traditional I/O, introduces zero‑copy concepts and their relationship with DMA and PageCache, details RDMA architectures, and demonstrates practical zero‑copy implementations such as mmap+write, sendfile, splice, tee, and Java NIO APIs.
This article explains the principles behind zero‑copy, DMA, and RDMA, compares traditional I/O copying with modern zero‑copy techniques, and shows practical implementations in Linux and Java that dramatically reduce CPU overhead and boost network and file‑transfer throughput.
This article explains why a naïve BufferedReader/Writer approach to splitting large text files is inefficient, demonstrates a zero‑copy solution using FileChannel.transferTo with line‑preserving logic, and shows benchmark results that reveal dramatic performance gains.
This article explains how Kafka sustains massive traffic by writing logs sequentially to disk, leveraging the operating system’s page cache for fast in‑memory writes, employing zero‑copy techniques like sendfile to avoid user‑space copying, and batching messages to reduce network overhead, thereby delivering high‑throughput, low‑latency streaming.
This article explains how RocketMQ’s CommitLog architecture—sequential writes, mmap zero‑copy, PageCache acceleration, fixed‑size log files, flexible flushing strategies, and efficient ConsumeQueue indexing—enables the system to sustain million‑level QPS with high reliability and low latency.
This article explains how Go's concurrent mark‑sweep garbage collector can become a bottleneck in high‑performance scenarios and provides practical techniques—such as using sync.Pool, avoiding unnecessary pointers, pre‑allocating memory, and applying zero‑copy and unsafe tricks—to dramatically lower GC overhead and improve overall program speed.
This article breaks down the core techniques that give Kafka its high‑throughput capability, including producer batch settings (batch.size, linger.ms), broker append‑only writes, consumer poll configuration, partition distribution, zero‑copy data transfer, dual‑thread processing, and configurable compression algorithms.
This article explains how mmap maps files into a process's virtual memory to eliminate double data copies and reduce system‑call overhead, offering performance gains, a simpler programming model, and discusses its limitations such as address‑space constraints and page‑fault latency.
Kafka attains million‑level message throughput by writing logs sequentially to disk, leveraging OS page cache for batch flushing, sending messages in large batches, and employing zero‑copy techniques that move data directly from disk to network sockets, dramatically reducing I/O overhead and CPU usage.
This article provides an in‑depth technical analysis of DeepSeek's open‑source 3FS distributed file system, focusing on the StorageService architecture, space pooling, allocation mechanisms, reference counting, fragmentation handling, and the RDMA‑based read/write data path.
This article explains why traditional I/O interfaces rely on data copying, demonstrates the hidden overhead of read/write in a network server, and introduces zero‑copy methods such as mmap, sendfile, DMA Gather Copy, and splice to dramatically reduce copies and context switches for faster I/O.
This article explains RDMA technology, covering its core principles, programming model with Verbs API, various communication modes, and its impact on data‑center networking, high‑performance computing, and distributed storage, highlighting its low‑latency, zero‑copy advantages over traditional TCP/IP.
Aeron is an open‑source, low‑latency, high‑throughput messaging framework that leverages zero‑copy memory, shared‑memory IPC and UDP transport to deliver microsecond‑level latency for finance, gaming, and distributed systems, offering a simple API and powerful performance features.
The article explains how Kafka’s use of sendfile zero‑copy gives it higher throughput than RocketMQ, which relies on mmap, and discusses the trade‑offs between performance and feature richness when choosing between the two message‑queue systems.
The article compares Kafka and RocketMQ, explaining how Kafka’s use of sendfile zero‑copy reduces system calls and data copies, while RocketMQ relies on mmap, leading to lower throughput but richer features, and offers guidance on choosing between them based on scenario.
This article explains how Kafka achieves ultra‑high throughput and low latency despite being disk‑based, covering its Reactor I/O network model, zero‑copy techniques, partitioning strategies, segment logs with sparse indexes, sequential disk writes, page cache usage, compression, batch processing, and lock‑free offset management.
This article explains the concept of zero‑copy, compares traditional I/O copying with mmap, sendfile, DMA scatter/gather and splice techniques, and shows how Java NIO leverages mmap and sendfile to achieve high‑performance data transfer while minimizing CPU involvement.
This article explains how Kafka attains extremely high throughput by using batch sending, message compression, sequential disk I/O, PageCache, zero‑copy transfers, and memory‑mapped index files, detailing the underlying code paths and trade‑offs that enable millions of messages per second.
The article investigates Go 1.22’s claim that simple type casting []byte(str) can replace unsafe‑based string‑to‑byte conversions, presents four implementation variants, runs detailed benchmarks on macOS M2 and Linux amd64, analyses compiler inlining and escape behavior, and explains the hidden pitfalls of capacity and mutability in the k8s shortcut.
This article examines why Go’s fasthttp library can outperform the standard net/http package by up to tenfold, covering memory allocation strategies, zero‑copy techniques, connection pooling, and additional optimizations, and offers guidance on when to choose each library for high‑performance backend services.
The article explains how RocketMQ’s reliance on mmap‑based zero‑copy leads to more data copies and system‑call overhead compared to Kafka’s sendfile approach, resulting in lower throughput, and discusses when to choose each message‑queue system based on functional needs and performance requirements.
This article explains how RocketMQ achieves high throughput by using sequential disk writes, OS page caching, asynchronous flushing, and zero‑copy techniques, detailing each mechanism and providing code examples. It also discusses the impact on disk seek time and data durability.
The article explains how Kafka attains ultra‑high write throughput by leveraging disk sequential writes, zero‑copy data transfer, operating‑system page cache, and memory‑mapped files, detailing each technique’s impact on latency, CPU usage, and overall performance.
This article explains how zero‑copy technology eliminates redundant memory copies and context switches during data transmission, compares traditional read‑send workflows with zero‑copy approaches such as Linux sendfile/splice and Java's FileChannel.transferTo, and discusses performance benefits and practical considerations.
This article explains the key techniques behind Kafka's high‑throughput performance, including batch sending, message compression, sequential disk reads/writes, efficient use of PageCache, zero‑copy transfers, and memory‑mapped index files, with code examples illustrating each mechanism.
Netty is a high‑performance network framework whose three‑layer architecture—Core, Protocol Support, and Transport Service—combined with a Reactor‑based logical design, diverse I/O models, advanced memory management, zero‑copy techniques, and optimized data structures, enables efficient custom protocol handling and scalable server development.
This article explores the ten key architectural and implementation techniques—such as batch messaging, compression, Netty networking, zero‑copy I/O, sequential writes, efficient storage structures, asynchronous processing, batch handling, lock optimizations, and thread‑pool isolation—that together make RocketMQ a high‑performance message middleware.
This article explains the principles of Direct Memory Access (DMA), compares it with traditional I/O, details the DMA data‑transfer workflow, and explores zero‑copy techniques such as mmap + write and sendfile, highlighting how they reduce context switches, data copies, and improve overall Linux I/O efficiency.
The talk examines the challenges of using standard Linux TCP for high‑performance data‑center workloads, explores how RDMA can provide zero‑copy and asynchronous kernel bypass, and presents experimental results from an FPGA‑based prototype that approaches 800 Gbps packet rates while highlighting congestion‑control and CPU‑utilization trade‑offs.
This article explains how Kafka achieves million‑message‑per‑second throughput by leveraging zero‑copy I/O, an append‑only log, batch processing, compression, consumer pull optimization, unflushed memory buffers, and JVM garbage‑collection tuning, detailing each mechanism and its impact on performance.
The article explains why the naïve approach of reading a file into a small user‑space buffer and sending it piece‑by‑piece is inefficient, and how zero‑copy, PageCache, asynchronous I/O and direct I/O can dramatically reduce context switches, memory copies, and CPU usage for high‑throughput file transfers.
This article explains what Netty is, why it’s essential for Java back‑end development, details its core components and high‑performance mechanisms such as the Reactor model, Zero‑Copy and object pooling, and shows how Netty handles TCP framing issues with practical decoding solutions.
This article examines traditional file transfer methods, highlights their performance drawbacks caused by excessive context switches and memory copies, and explains how zero‑copy, PageCache, asynchronous I/O, and direct I/O can dramatically improve throughput and reduce CPU usage in backend systems.
This article explains the traditional I/O read/write workflow, identifies its performance bottlenecks, introduces the concept of zero‑copy, and demonstrates practical zero‑copy implementations—including DMA, sendfile, shared memory, memory‑mapped files, and Java NIO techniques such as ByteBuffer, Channel, transferTo/transferFrom, and memory‑mapped files.
File transfer can be dramatically accelerated by reducing context switches and memory copies, using techniques such as zero‑copy, leveraging PageCache, and employing asynchronous or direct I/O, which together cut system calls, lower CPU usage, and improve concurrency for large‑scale data delivery.
Fury is a JIT‑compiled, zero‑copy multi‑language serialization framework that delivers up to 170× faster performance than Java’s native serialization, supports automatic cross‑language object graph serialization for Java, Python, C++, Go and JavaScript, and offers specialized protocols for high‑throughput big‑data and AI workloads.
This article breaks down how Kafka achieves up to 20 million messages per second and 600 MB/s throughput by examining producer batching and compression, broker PageCache and zero‑copy techniques, and consumer parallelism with group coordination.
This article explains how storage media performance, kernel‑user mode transitions, DMA, zero‑copy, and PageCache interact to affect I/O latency, why multiple data copies and context switches hurt throughput, and how techniques like mmap, sendfile, and asynchronous I/O can reduce overhead.
This article explains the performance hierarchy of storage media, the distinction between kernel and user modes, how data moves between them, and why excessive context switches hurt I/O, then introduces DMA, zero‑copy, PageCache, mmap + write, sendfile, and tuning tips to dramatically improve Linux I/O efficiency.
The article explains how different storage media affect I/O speed, describes kernel and user mode separation, introduces DMA and zero‑copy techniques such as mmap + write and sendfile, and discusses PageCache behavior, advantages, drawbacks, and tuning for high‑performance file transfers.
This article explains how fastjson2 achieves significant performance gains by replacing reflection with Java 8 Lambda‑generated function mappings, applying zero‑copy String construction, and implementing hand‑crafted parsers for common types such as dates, backed by benchmark data.
This comprehensive guide explores essential backend techniques—including lock‑free programming, zero‑copy I/O, efficient serialization, pooling, concurrency, async processing, caching strategies, sharding, storage optimizations, and queue mechanisms—to build high‑performance, scalable services while highlighting practical code examples and real‑world trade‑offs.
This article walks through every stage of a RocketMQ message—how producers create and route messages, the storage mechanisms and zero‑copy techniques used by brokers, high‑availability modes, consumption models, ordering guarantees, and the automatic cleanup policies that finally retire messages.
This article explains the concepts of virtual memory, kernel and user modes, DMA, and why data must move between kernel and user spaces, then compares standard I/O with zero‑copy techniques, MMAP, and off‑heap buffers, showing how they reduce copy operations and improve Java server performance.
Virtual memory extends limited physical RAM by giving each process a large address space translated via page tables and a TLB, while multi‑level tables manage size, and the OS separates kernel and user space, offering blocking/non‑blocking, synchronous/asynchronous, multiplexed I/O, reactor patterns, and zero‑copy techniques to optimize data transfer.
This article explains the zero‑copy technique in Java, covering I/O fundamentals, kernel read/write flow, mmap+write and Sendfile methods, virtual memory mapping, MappedByteBuffer and DirectByteBuffer usage, channel‑to‑channel transfer, Netty composite buffers, and how frameworks like RocketMQ and Kafka leverage zero‑copy for performance.
Using a step‑by‑step example program, this article shows how to dramatically improve Linux pipe read/write performance—from an initial 3.5 GiB/s to 65 GiB/s—by applying zero‑copy techniques, ring buffers, paging insights, vmsplice/splice system calls, huge pages, and busy‑loop optimizations.
This article explains the concept of zero‑copy, how it eliminates redundant data copying in I/O operations, and demonstrates its implementation in Java through buffers, mmap+write, Sendfile, MappedByteBuffer, DirectByteBuffer, channel‑to‑channel transfers, and Netty’s composite buffers.
This article explains how Kafka achieves extremely high throughput—up to 20 million messages and 600 MB per second per node—by optimizing the producer, broker, and consumer components through batch sending, custom protocols, page‑cache usage, zero‑copy transfers, and efficient compression algorithms.
This article explains how zero‑copy techniques such as DMA, sendfile, mmap and Direct I/O reduce data copies and context switches in Linux, compares their mechanisms, advantages and drawbacks, and shows typical use cases like Kafka for high‑performance I/O.
This article explains the principle of zero‑copy, its role in improving I/O performance, and demonstrates how Java NIO, mmap, sendfile, channel‑to‑channel transfer, Netty composite buffers and other techniques achieve zero‑copy in backend systems.
This article explains why Kafka can handle up to 20 million messages per second and 600 MB/s throughput by detailing producer batching and custom protocols, broker page‑cache, file layout and zero‑copy techniques, as well as consumer group strategies for efficient message consumption.
This article explains the four key techniques—page‑cache usage, sequential disk writes, zero‑copy transfers, and partitioned segment indexing—that enable Kafka to achieve exceptionally high write performance, detailing how each mechanism reduces latency and maximizes throughput.
Fury is a JIT‑based native multi‑language serialization framework that automatically handles shared and cyclic references, offers zero‑copy support, and delivers 20‑200× speed improvements over existing solutions, making it a high‑performance drop‑in replacement for Java, Python, Go, and C++ serialization needs.
This article explains Kafka's core concepts—including topics, partitions and replicas, log segment storage, leader‑follower mechanics, consumer groups, network threading model, zero‑copy I/O, and the essential role of Zookeeper for broker, topic, consumer, and offset management—providing a comprehensive overview for developers and architects.
This article explains the differences between buffers and caches, explores MySQL’s buffer pool architecture, details write‑through/write‑back strategies, and reviews key InnoDB parameters such as innodb_flush_log_at_trx_commit and innodb_flush_method for optimizing data durability and performance.
This article provides a comprehensive guide to Netty, covering its definition, advantages, key features, application scenarios, performance characteristics, architectural components, threading models, code examples, handling of packet framing, zero‑copy techniques, connection management, object pooling, and supported serialization protocols.
This article breaks down Kafka’s high‑availability, high‑performance, and high‑concurrency design, covering controller and leader election, replica and ISR mechanisms, ACK settings, the Reactor NIO model, zero‑copy I/O, compression, producer batching, memory‑pooling, and the multi‑layer network threading architecture.
This article explains Linux memory metrics VSZ, RSS, and PSS, illustrates them with a roommate analogy, details PageCache and its role in I/O, describes swap behavior and the swappiness setting, and introduces zero‑copy techniques to reduce data copying and context switches.
This article explains how Linux zero‑copy mechanisms such as DMA, sendfile, mmap and Direct I/O reduce data copies and context switches, detailing their operation, advantages, limitations, and real‑world usage in systems like Kafka and MySQL.
This article provides a comprehensive overview of Remote Direct Memory Access (RDMA), covering its definition, how it differs from traditional networking, core advantages such as zero‑copy and CPU offload, typical use cases, the three main RDMA protocols, deployment requirements, and essential programming concepts and terminology.
This article explains how DMA offloads memory‑to‑device data transfers, reduces the four data copies and context switches of traditional I/O, and introduces zero‑copy methods such as sendfile, mmap and Direct I/O, illustrating their mechanisms, advantages, limitations, and real‑world use cases like Kafka and MySQL.
Traditional Linux I/O using read() and write() involves multiple CPU and DMA copies and context switches, while modern techniques like zero‑copy, multiplexing, and page‑cache optimization reduce overhead; this article explains the classic I/O flow, read/write operations, network and disk I/O, and high‑performance strategies such as zero‑copy and Direct I/O.
This article presents a collection of interview questions covering backend fundamentals such as linked‑list sorting, encryption differences, TCP reliability, IO models, Hystrix, delayed tasks, HTTPS flow, transaction isolation, index pitfalls, virtual memory, Redis leaderboards, distributed locks, zero‑copy techniques, Java synchronized, and Snowflake ID generation.
The article presents a Guard abstraction that temporarily borrows and returns Protobuf field pointers via set_allocated/release, eliminating costly CopyFrom operations in recommendation pipelines, enabling zero‑copy field sharing across central‑control and recall stages, improving CPU usage and latency while handling safety and rollback.
This article explains Kafka's internal topics, preferred replicas, partition assignment processes, log directory layout, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, legacy consumer design flaws, rebalance workflow, and producer idempotence, providing a comprehensive overview of Kafka's backend architecture.
This article explains the costly four‑copy, four‑context‑switch data path in traditional Linux I/O, introduces DMA as a co‑processor that offloads memory transfers, describes zero‑copy techniques such as sendfile, mmap and Direct I/O, and shows how Kafka and MySQL leverage these methods to reduce CPU overhead and improve throughput.
This article explains why a message system like Kafka is crucial for decoupling services, handling asynchronous workflows such as e‑commerce flash sales, controlling traffic, and achieving high concurrency, high availability, and high performance through sequential disk writes, zero‑copy reads, replication, and careful resource planning.
This article explains how Kafka achieves remarkable speed and massive throughput by using sequential disk I/O, OS page cache, zero‑copy transfers, partitioned log segments with indexes, batch processing, and efficient compression, making it a cornerstone of modern big‑data pipelines.
Zero-copy techniques eliminate unnecessary data copying between user and kernel space, dramatically improving I/O performance; this article explains core I/O concepts, explores Java implementations like MappedByteBuffer, DirectByteBuffer, channel-to-channel transfers, and demonstrates Netty’s composite buffers, plus examples from Kafka and RocketMQ.
Zero‑copy I/O minimizes data copying by using DMA and kernel mechanisms such as mmap + write or the sendfile() system call, with sendfile achieving true zero‑copy through fewer system calls, context switches, and memory copies compared to traditional read/write paths.
