0 views collected around this technical thread.
This article explains Linux's memory reclamation mechanisms, covering the role of memory as the system's bloodstream, the three reclamation paths (fast, direct, kswapd), zone watermarks, page cache structures, reverse mapping, and how swap and compression are used to keep the system stable under memory pressure.
This article explains how Kafka achieves high concurrency through its distributed broker cluster, partitioned topics, sequential log writes, message compression, asynchronous producer mechanisms, and OS page‑cache techniques, illustrating the combined architectural and performance optimizations that enable massive throughput.
This article explains the Linux Page Cache mechanism, covering its core concepts, read/write workflows, data consistency, optimization strategies, real-world use cases, advanced topics, common misconceptions, and practical tips for improving system performance and resource management.
The article examines a long‑standing hidden bug in the Linux kernel’s page‑cache Xarray that caused occasional data loss with Large Folio support, details its discovery and fix by the TencentOS team, and shows how consolidating multiple tree walks into a single walk in Linux 6.10 reduced latency and improved performance by about ten percent.
This article examines the conditions under which data may be lost when a Linux process crashes while writing a file, explaining page cache behavior, the roles of stdio versus system calls, dirty page handling, write‑back mechanisms, and strategies such as fflush, fsync, and direct I/O to ensure data integrity.
The article explains how Kafka attains ultra‑high write throughput by leveraging disk sequential writes, zero‑copy data transfer, operating‑system page cache, and memory‑mapped files, detailing each technique’s impact on latency, CPU usage, and overall performance.
Linux’s shmem subsystem provides hybrid anonymous/file‑backed pages that enable diverse shared‑memory scenarios—parent‑child communication, IPC, tmpfs, Android ashmem, and memfd—by using APIs such as shmem_file_setup, handling page faults through cache and swap mechanisms, and employing a specialized reclamation process to manage memory efficiently.
This article presents a comprehensive design and implementation of a high‑concurrency memory pool in C++, covering concepts such as fixed‑size block allocation, thread‑local caches, central and page caches, lock‑free techniques, span management, and performance benchmarking against standard malloc.
Linux kernel introduced the struct folio abstraction to replace ad‑hoc compound‑page tricks, giving a clear collection‑oriented representation for power‑of‑two page groups such as THP and HugeTLB, and providing dedicated APIs that eliminate naming confusion, unify reference handling, and make memory‑management code safer and easier to understand.
The article explains how different storage media affect I/O speed, describes kernel and user mode separation, introduces DMA and zero‑copy techniques such as mmap + write and sendfile, and discusses PageCache behavior, advantages, drawbacks, and tuning for high‑performance file transfers.
The article explains Linux I/O optimization through zero‑copy techniques—such as mmap + write, sendfile, and splice—detailing memory hierarchy, the benefits of reducing user‑kernel copies, the suitability of async + direct I/O for large file transfers, real‑world uses like Kafka and Nginx, and inherent platform limitations.
This article provides a comprehensive analysis of how JDK NIO performs file read and write operations by examining the underlying Linux kernel mechanisms, including the differences between Buffered and Direct IO, the structure and management of the page cache, file readahead algorithms, and the kernel parameters governing dirty page writeback.
This article explains the design background, performance benefits, potential drawbacks, synchronous and asynchronous mechanisms, key data structures, operational principles, illustrative examples, and critical code paths of Linux kernel file readahead, providing a comprehensive technical overview for developers and system engineers.
This article explains Linux storage hierarchy, the roles of user‑space and kernel caches, the three‑layer I/O stack, page‑cache synchronization policies, file‑operation atomicity, locking mechanisms, and performance testing techniques for HDD and SSD devices.
The article explains how to redesign a high‑traffic user‑registration flow by introducing an asynchronous queue layer, detailing the broker architecture, commitlog, consumeQueue, page‑cache, mmap, topic/tag routing, high‑availability strategies, and the role of a nameserver in a RocketMQ‑style system.
This article explains the Linux file I/O stack by outlining the path from user space through system calls to the kernel layers—VFS, filesystem, block layer, and SCSI—detailing each layer's role, page cache mechanisms, writeback processes, and direct I/O implementations with code examples.
This article explains the Linux file I/O stack by outlining its clear route from user space to hardware, detailing the roles of VFS, the filesystem, the block layer, and the SCSI driver, and then dives into page cache mechanisms and code paths for both buffered and direct I/O.
This article explains the principles of disk I/O, covering read/write workflows, DMA acceleration, page cache, memory‑mapped files, buffering techniques, Linux kernel parameters and practical Java code examples to illustrate how to reduce CPU involvement and improve overall system performance.
The article explains the fundamentals of disk I/O, covering read/write processes, IO interrupts, DMA, page cache, mmap, buffered versus unbuffered file operations, ByteBuffer usage, Linux dirty‑page parameters, and how these mechanisms affect application performance and reliability.
This article explains how Linux implements traditional system‑call I/O using read() and write(), details the data‑copy and context‑switch overhead of read and write operations, describes network and disk I/O, and introduces high‑performance techniques such as zero‑copy, multiplexing, and page‑cache optimizations.