This article explains the fundamentals of zero‑copy, DMA, PageCache and RDMA, compares them with traditional I/O, describes Linux implementations such as sendfile, mmap+write, splice and Java NIO APIs, and shows practical use‑cases that dramatically reduce CPU load and latency in high‑throughput networking and file handling.
This article explains why direct writes to a Java NIO channel fail for large messages, introduces a write‑queue and OP_WRITE listener to handle partial writes, and provides a complete PacketWriter implementation that transforms synchronous writes into efficient asynchronous operations.
This article explains the root causes of TCP sticky and half‑packet issues, designs a length‑field message protocol and a generic decoder, provides complete Java NIO code for a mini Netty framework, and discusses its startup process and current limitations.
This article explains the performance bottlenecks of traditional I/O, introduces zero‑copy concepts and their relationship with DMA and PageCache, details RDMA architectures, and demonstrates practical zero‑copy implementations such as mmap+write, sendfile, splice, tee, and Java NIO APIs.
This article explains why a naïve BufferedReader/Writer approach to splitting large text files is inefficient, demonstrates a zero‑copy solution using FileChannel.transferTo with line‑preserving logic, and shows benchmark results that reveal dramatic performance gains.
Upgrading Spring Boot's RestTemplate to the NIO‑based WebClient improves concurrency and performance, but can trigger a java.lang.OutOfMemoryError: Direct buffer memory; this article explains the cause, diagnostic steps using yCrash, and how adjusting -XX:MaxDirectMemorySize resolves the issue.
This article explains why using raw Java NIO for network communication is problematic, highlights Netty's advantages such as simplified APIs, high performance, and extensive adoption, and provides a complete server‑client demo with detailed code and design insights.
This article explains Netty's internal mechanisms for buffering outbound data, flushing it to the network, handling inbound responses with NIO selectors, outlines the overall Netty workflow, and clarifies the roles of Backlog and KeepAlive parameters in TCP connections.
This article examines the difficulties of building non‑blocking services with Java NIO, compares blocking and non‑blocking pipelines, and presents practical designs for handling partial reads, dynamic buffers, message framing, and efficient write management to scale servers to millions of connections.
This article explains the concept of zero‑copy, compares traditional I/O copying with mmap, sendfile, DMA scatter/gather and splice techniques, and shows how Java NIO leverages mmap and sendfile to achieve high‑performance data transfer while minimizing CPU involvement.
This article provides a detailed overview of Netty, a high‑performance asynchronous NIO framework, covering its key features, architectural design, core components, high‑performance strategies, and typical use cases in internet services, gaming, and big‑data systems, while also highlighting why it outperforms native JDK NIO.
This article walks through Kafka’s network‑layer internals, detailing how the client prepares, pre‑sends, and actually sends messages, then reads and parses responses, with code snippets that illustrate the Selector, KafkaChannel, and related classes in Java NIO.
This article explains the historic C10K problem of handling ten thousand concurrent connections, compares traditional BIO with modern I/O models such as NIO, select, poll, epoll, and AIO, and provides Java and C++ code examples illustrating how each model improves scalability and performance.
This article explains the traditional I/O read/write workflow, identifies its performance bottlenecks, introduces the concept of zero‑copy, and demonstrates practical zero‑copy implementations—including DMA, sendfile, shared memory, memory‑mapped files, and Java NIO techniques such as ByteBuffer, Channel, transferTo/transferFrom, and memory‑mapped files.
This article deeply analyzes Kafka broker’s network architecture and request‑handling pipeline, walking through simple sequential models, multithreaded async designs, the Reactor pattern with Java NIO, key thread roles, core processing flow, and practical tuning parameters for high‑throughput, low‑latency deployments.
The article explains Java NIO’s non‑blocking channels, buffers, and selectors, then shows how the open‑source Tars RPC framework builds a multi‑reactor, multi‑thread network model on top of NIO—detailing server socket setup, event dispatch, session management, and read/write processing.
This article walks through a real‑world case of off‑heap memory leakage in the Dble proxy, detailing symptom observation, log analysis, monitoring, BTrace instrumentation, code review, and the eventual fix that delays buffer allocation until needed.
This article explains the zero‑copy technique in Java, covering I/O fundamentals, kernel read/write flow, mmap+write and Sendfile methods, virtual memory mapping, MappedByteBuffer and DirectByteBuffer usage, channel‑to‑channel transfer, Netty composite buffers, and how frameworks like RocketMQ and Kafka leverage zero‑copy for performance.
This article explains the concept of zero‑copy, how it eliminates redundant data copying in I/O operations, and demonstrates its implementation in Java through buffers, mmap+write, Sendfile, MappedByteBuffer, DirectByteBuffer, channel‑to‑channel transfers, and Netty’s composite buffers.
This article explains the concept of zero‑copy, its benefits for I/O performance, and how Java NIO, Netty, Kafka, RocketMQ and related APIs such as mmap+write and sendfile implement zero‑copy to avoid unnecessary data copying between kernel and user space.
This article summarizes the challenges and optimization techniques for building a high‑performance long‑connection service with Netty, covering non‑blocking I/O, Linux kernel tuning, client‑side testing, VM‑based scaling, data‑structure tweaks, CPU and GC bottlenecks, and the final results of achieving hundreds of thousands of connections and tens of thousands of QPS on a single server.
This article thoroughly explains Kafka's network layer by dissecting the complete send and receive workflow, covering message pre‑sending, selector polling, write and read operations, buffer management, and callback handling, all illustrated with core source code snippets.
This article explores Netty’s low‑level data container ByteBuf by examining Java NIO Buffer design, its core attributes, view mechanisms, heap and direct implementations, byte‑order handling, and how to read and write primitive types, providing a comprehensive foundation for high‑performance network programming.
This article thoroughly examines Kafka's broker‑side network architecture, tracing its evolution from a simple sequential model to a high‑performance, event‑driven Reactor design using Java NIO, and provides practical tuning guidance for achieving optimal throughput and latency.
This article explains how to build a scalable long‑connection server using Netty, discusses the underlying bottlenecks such as Linux kernel limits, CPU and GC issues, and provides practical code examples and tuning techniques to achieve hundreds of thousands of connections and high QPS.
This article walks through the challenges and solutions for building a high‑performance long‑connection service with Netty, covering Netty basics, non‑blocking I/O, sample Java NIO and Netty code, Linux kernel tuning, QPS optimization, data‑structure tweaks, GC adjustments, and the final performance results.
This article explains how to build a scalable long‑connection server using Netty, covering the fundamentals of non‑blocking I/O, practical code examples for achieving millions of concurrent connections, common Linux kernel limits, CPU and GC bottlenecks, and a series of tuning techniques to dramatically improve QPS and stability.
In this interview‑style tutorial, an Alibaba P7 engineer walks through the limitations of BIO, the non‑blocking NIO API, kernel‑level select/poll mechanisms, and the design of epoll, illustrating how each solves the C10K problem and how they are used in Java.
This article walks through the limitations of traditional Java IO, explains how NIO and its selector mechanism solve thread‑resource and efficiency problems, and demonstrates why Netty’s higher‑level abstraction makes network programming far simpler and more performant, complete with runnable code examples.
This article explains the limitations of traditional blocking I/O, introduces Java NIO with selectors, compares the two models using a kindergarten analogy, critiques raw JDK NIO, and demonstrates how Netty simplifies and optimizes network development with concise server and client code examples while promoting a comprehensive Netty learning book.
This article dissects Netty’s core connection‑acceptance mechanism, explains how the OP_ACCEPT event is processed, reveals a subtle bug that limits read loops to a single connection, and shows the fix introduced in version 4.1.69.final, offering developers a complete understanding of Netty’s reactor architecture.
This article explains how Netty constructs its core reactor thread pool using NioEventLoopGroup, detailing the creation of boss and worker groups, the underlying selector optimization, task queue setup, and the round‑robin binding strategy that distributes channels across multiple event loops for high‑performance I/O handling.
This article explains how Netty abstracts Java NIO's event registration by using SelectionKey interestOps, detailing the registration of OP_ACCEPT and OP_READ events, the underlying doRegister and doBeginRead implementations, and the flow of events through the Netty pipeline with illustrative code snippets.
This article explains the core differences between Java NIO and traditional IO, introduces channels and buffers, details NIO’s underlying mechanisms, provides a complete code example, and compares NIO’s workflow with Netty’s model, highlighting performance and architectural distinctions.
This article explains how to build a scalable long‑connection service using Netty, covering the underlying concepts, Linux kernel tuning, code examples for Java NIO and Netty, performance bottlenecks such as QPS, data‑structure and GC issues, and practical optimization techniques to achieve hundreds of thousands of concurrent connections.
This article explains how zero‑copy techniques such as mmap, sendfile, and splice reduce CPU involvement in data transfer by avoiding memory copies between kernel and user spaces, and shows how Java NIO and Netty implement these mechanisms for high‑performance backend I/O.
This article provides a comprehensive overview of Kafka performance optimization, covering network, disk, and complexity challenges, sequential write techniques, zero‑copy data transfer, page‑cache usage, reactor‑based networking, batch processing, compression, partition concurrency, and file‑structure design to help developers build high‑throughput, low‑latency streaming systems.
This article explains Netty’s high‑performance asynchronous NIO architecture, introduces the Reactor pattern and its three thread‑model variants, and details Netty’s core components such as Selector, EventLoopGroup, ChannelPipeline, and ByteBuf, highlighting how they improve scalability and efficiency.
This article summarizes the DBLE network module, describing its key classes such as NIOAcceptor, NIOReactor, and SocketWR, explaining how client connections are accepted and processed, and highlighting the importance of low‑level network I/O knowledge for middleware development.
This article explains the five Linux I/O models—blocking, non‑blocking, I/O multiplexing, signal‑driven, and asynchronous—detailing their system calls, synchronization characteristics, and how they map to Java BIO, NIO, and AIO implementations, with illustrative analogies and usage scenarios.
This article explains Netty’s core architecture and its master‑worker multi‑reactor thread model, detailing the roles of Acceptor, Main Reactor, and Sub Reactor, how connection handling, I/O, encoding/decoding, and business logic are coordinated across threads, and provides practical insights for interview preparation.
This article provides a concise, code‑driven walkthrough of Netty's core components—including event‑loop groups, channel initialization, pipeline handling, and the server‑client handshake process—by simplifying the original source to highlight the essential mechanisms behind asynchronous network communication in Java.
This article explains how Netty's boss and worker NioEventLoop threads collaborate to accept socket connections, register selectors, and transition to read operations, detailing the underlying selector polling, OP_ACCEPT handling, and the code paths that trigger channel activation.
This article provides a detailed source‑code walkthrough of DBLE’s network module, explaining how it processes MySQL packets, implements selector‑based multiplexing, and performs asynchronous task handling to achieve high‑performance database communication for client connections.
This tutorial provides a detailed introduction to Netty, covering its architecture, core concepts such as buffers, channels, selectors, reactor thread models, zero‑copy techniques, and includes practical TCP server/client code examples, helping readers understand and apply Netty for high‑performance Java network programming.
The article explains low‑level I/O fundamentals—kernel versus user space, blocking (BIO) versus non‑blocking (NIO) models, and the evolution to multiplexing with select, poll, and epoll—showing how these mechanisms address the C10K problem and are exposed in Java through NIO and Netty APIs.
This article explains the zero‑copy data transfer technique, compares it with traditional read/write approaches, shows Java NIO code examples, and discusses its use in high‑performance systems such as Kafka and Spark, highlighting the reductions in context switches and memory copies.
This article explains the five Linux I/O models—blocking, non‑blocking, I/O multiplexing, signal‑driven, and asynchronous—detailing their system calls, synchronization concepts, performance characteristics, and how they map to Java BIO, NIO, and AIO implementations.
This article provides an in‑depth tutorial on Java NIO, covering its fundamentals, IO models (BIO, NIO, AIO), core components such as Channel, Buffer and Selector, detailed code examples for a server and client, and a summary of best practices and limitations.
This article explains how zero‑copy techniques such as DMA, mmap, and sendfile work, compares them with traditional file I/O, shows their Java NIO implementations, and presents performance test results demonstrating the advantages of mmap for small files and sendfile for large files.
This article explains the design of event‑driven architectures for high‑performance web services, compares the Reactor (synchronous) and Proactor (asynchronous) models, and provides Java‑style pseudocode for their core components, handlers, and event loops.
This article explains the traditional copy‑based data transmission process, introduces the zero‑copy technique—including basic sendfile(), scatter/gather DMA and mmap support—shows how it reduces context switches and copies, and demonstrates its practical use in Kafka and Spark for high‑throughput workloads.
This article explains the evolution of data transfer in computers—from early CPU‑bound copying to DMA, channel architectures, and I/O processors—then demonstrates how zero‑copy techniques like sendFile and memory‑mapped I/O dramatically reduce CPU usage and improve throughput.
The article explains how Kafka's network layer uses the Reactor pattern with Java NIO, describes the single‑threaded selector model, its limitations, and how Kafka introduces multiple selectors, acceptor, processor, and handler thread pools to achieve high‑concurrency network I/O.
This article explains Java NIO introduced in Java 1.4, covering its core components—Channels, Buffers, and Selectors—detailing channel types, buffer operations, non‑blocking behavior, selector usage, and code examples such as FileChannel and Pipe, and compares NIO with classic IO.
This article explains the internal workflow of Netty when launching a server, covering why Netty is chosen over raw JDK NIO, the key components created during bootstrap, the step‑by‑step source‑code analysis of channel creation, initialization, registration, and the final bind operation that activates the server.
