This guide explains why the default Linux kernel limits hinder Nginx performance, lists eight crucial sysctl parameters to increase port ranges, file descriptors, and connection queues, and shows how to verify and safely apply the changes for high‑concurrency workloads.
This article examines recent efforts to make Linux kernel memory management programmable with eBPF, covering BPF‑MM patches for mTHP order, cache‑ext’s customizable LRU, FetchBPF prefetch policies, and BPF OOM hooks, and discusses their design, implementation details, and performance impacts.
This guide walks through step‑by‑step Nginx high‑concurrency tuning—covering Linux kernel network parameters, system limits, worker process settings, connection reuse, HTTP/2, gzip compression, benchmarking, and monitoring—enabling single‑node throughput of over one million packets per second with sub‑50 ms P99 latency.
This article explores the evolution of Linux I/O models—from blocking and non‑blocking to epoll—and introduces io_uring as a high‑performance asynchronous framework that reduces system‑call overhead, eliminates data copies, and unifies network and disk I/O for modern high‑concurrency applications.
Memory barriers act like traffic signals for concurrent threads, enforcing a strict order of memory operations to prevent data races, cache incoherence, and compiler reordering, thereby ensuring program correctness and stability across multi‑core and multi‑processor systems.
The article analyzes how the newly introduced swap table replaces the kernel's xarray for swapcache, offering O(1) lookups, better locality, reduced lock contention, and lower memory overhead, with benchmark results showing notable performance gains in real workloads.
The article analyzes the new swap table implementation that replaces the kernel's xarray‑based swapcache with a per‑cluster array, offering O(1) lookups, reduced lock contention, lower memory overhead, and measurable performance gains demonstrated by real‑world benchmarks.
This article explains all TCP connection states, the three‑way handshake and four‑way termination, details half‑open and full‑open queues, shows how to monitor and troubleshoot listen and accept queue overflows, and lists key Linux kernel parameters for optimizing TCP performance.
Linux kernel 6.17’s first release candidate arrived without any bcachefs changes, as Linus Torvalds expressed anger over delayed RISC‑V patches, while developers debate the future of the advanced COW file system, its removal from upcoming kernels, and the broader implications for Linux’s storage stack.
This guide outlines a step‑by‑step roadmap for aspiring kernel developers, covering essential knowledge foundations, hands‑on teaching‑OS projects, deep‑dive strategies for reading the real Linux kernel source, and curated book recommendations to build solid OS internals expertise.
The article explains how replacing the global mmap_lock read lock with per‑VMA locks for madvise MADV_DONTNEED/FREE removes severe lock contention and priority inversion in Linux memory management, and it details the patches, reasoning, and performance evidence supporting the change.
This article explains the role of the page‑table cache (TLB) in the Linux kernel, describing how it speeds up virtual‑to‑physical address translation, the underlying mapping process, cache organization, replacement policies, and its impact on system performance across desktops, servers, and high‑performance applications.
This article explains the hardware basis of NUMA, how Linux reads ACPI SRAT/SLIT tables to discover CPU‑memory topology, the kernel functions that initialize NUMA structures, and how tools like numactl can be used to optimize application performance on multi‑node servers.
The article examines the heated dispute between Linus Torvalds and Bcachefs maintainer Kent Overstreet over a controversial patch submitted during the 6.16‑rc phase, explaining how kernel merge‑window rules, community reactions, and the future of Bcachefs intertwine.
Discover how deep understanding of Linux kernel architecture, process, memory, filesystem, and network subsystems combined with practical Bash scripts can boost system performance by up to 300%, offering step‑by‑step tuning, monitoring, and debugging techniques essential for senior operations engineers.
The article examines the escalating dispute between Linus Torvalds and Bcachefs maintainer Kent Overstreet over a controversial patch, explains why the Linux kernel may abandon Bcachefs after version 6.16, and outlines community reactions and possible future paths for the file system.
The article chronicles Xavier Xia’s iterative patches to the Linux kernel’s contpte_ptep_get() function, showing how early‑exit logic and subsequent refinements ultimately yielded consistent performance gains across diverse dirty/young page table scenarios, backed by benchmark data that convinced skeptical reviewers.
This guide explains how to create a custom Android boot animation using frame sequences, package it correctly, compress it, replace it via ADB, and also covers modifying or hiding the Linux kernel and bootloader logos, including required tools, file formats, and safety precautions.
This article provides an in‑depth analysis of BPF maps—explaining their design principles, core features, various map types with code examples, and the macro expansion process that turns high‑level BCC helpers into native kernel map definitions for cloud‑native observability.
This article explains the problem of internal and external memory fragmentation in Linux systems, introduces eBPF as a powerful tracing tool, and provides step‑by‑step guidance for building, loading, and running eBPF programs to analyze and mitigate fragmentation on both Linux and Android platforms.
This article explains the concept, architecture, key data structures, registration process, implementation details, limitations, configuration steps, and practical examples of using Kprobes to dynamically instrument and debug Linux kernel functions without recompiling or rebooting the system.
This article explains the hardware basis of NUMA, how Linux reads ACPI SRAT and SLIT tables to discover CPU‑memory topology, the kernel functions that initialize NUMA structures, and how the memblock allocator incorporates this information to enable performance‑optimizing tools like numactl.
The article introduces KASAN, a Kernel Address Sanitizer for Linux that uses shadow memory and redzones to accurately detect out‑of‑bounds accesses, use‑after‑free, and double‑free errors, explains its underlying mechanisms, shows how to enable it in the kernel, and provides extensive example code and log analysis.
This article explains eBPF from a developer's perspective, covering the eBPF virtual machine, instruction encoding, bytecode generation with LLVM/Clang, loading via the bpf() syscall, verifier rules, execution flow, JIT compilation, and how Android builds and packages eBPF programs.
A recent Intel‑driven Linux kernel test revealed that a single commit fixing THP‑PMD alignment boosted 1‑byte malloc throughput by 3889% (≈40×), highlighting how tiny memory‑management tweaks can dramatically improve performance despite being demonstrated on synthetic benchmarks.
The article explains how the Linux kernel decides the CPU for a newly woken task, detailing the roles of wake_affine and select_idle_sibling, the influence of cache topology, run‑queue load, and various idle‑checking heuristics, with concrete examples and code snippets.
The 3rd eBPF Developer Conference held on April 19, 2025 at Xi'an University of Posts and Telecommunications featured 36 expert talks on eBPF advancements, network and security innovations, observability, performance optimization, a vibrant project marketplace, student projects, and provides video and PPT resources for the community.
This article walks through the registration of the KVM misc device in the Linux 5.9 kernel for arm64, explains how /dev/kvm is created via misc_register, and demonstrates a user‑space program that opens the device and uses an ioctl to retrieve the KVM API version, with detailed code analysis.
This article explains how the x86‑64 paging mechanism maps virtual addresses to physical memory, detailing the required protection‑mode conditions, page‑table structures, control‑register settings, various paging modes (32‑bit, PAE, 4‑level, 5‑level), and provides concrete kernel code examples and address‑translation demonstrations.
This article dissects Linux’s inotify mechanism, detailing the kernel data structures, system‑call flow, and functions that generate and deliver file‑system event notifications, complete with code walkthroughs and diagrams. It explains how read/write operations trigger event creation, how events are queued in inotify_device, and how user‑space processes retrieve them via read and poll interfaces.
The article explains how memory pools reduce allocation overhead and fragmentation by pre‑allocating large memory blocks, describes their implementation in C++ and the Linux kernel, compares various pool designs, and shows practical scenarios where pools improve performance and stability.
This article explains the concept of memory pools, how they reduce allocation overhead and fragmentation compared to traditional malloc/new, describes various pool designs and Linux kernel APIs, provides multiple C and C++ implementations, and discusses performance benefits and typical application scenarios such as servers, real‑time and embedded systems.
This article explains why Linux dominates server workloads, introduces the built‑in ftrace tracing framework, details its diverse tracers, lightweight static instrumentation and stable design, shows how to configure and use it via debugfs, and demonstrates performance tuning and fault diagnosis with real examples while comparing it to systemTap and LTTng.
This article walks through the complete boot sequence—from the UEFI firmware that runs after power‑on, through the boot manager and BootOrder variables, to the systemd‑boot loader loading initrd and the Linux kernel, explaining each component and its role in detail.
Recent testing of an Intel‑powered Linux kernel patch revealed that a single code change dramatically improved memory allocation throughput by nearly 40‑fold, highlighting the massive impact of fine‑tuned memory management optimizations and the importance of addressing THP and PMD alignment issues.
A single commit to the Linux kernel aligned transparent huge pages with PMD boundaries, delivering a near‑40‑fold increase in 1‑byte malloc throughput and illustrating how precise memory‑management tweaks can dramatically improve system performance.
This article walks through the step‑by‑step initialization of KVM's hyp mode memory mapping on ARM64, covering EL2 register setup, identity mapping, page‑table allocation, stack preparation, and code section mappings required for a functional hypervisor environment.
Ubuntu 24.04 LTS (Noble Numbat) introduces a Flutter‑based installer, GNOME 46 desktop, Linux 6.8 kernel, PipeWire as the default audio server, Thunderbird 115 as the default mail client, enhanced AppArmor and frame‑pointer security, broader hardware support and an optional 12‑year Ubuntu Pro support window, marking a substantial upgrade over 22.04.
This article explains the evolution of file system architecture from kernel‑level VFS to userspace via FUSE, reviews the historical role of NFS, details JuiceFS's implementation on top of FUSE, and presents benchmark results that demonstrate its high throughput and practical limitations.
The 2024 Rust State Survey reveals improved developer productivity but persistent challenges like slow compilation, debugging difficulty, and limited industry adoption, highlighting concerns over complexity, low usage in tech, and the uncertain future of Rust in the Linux kernel.
This article explains how the Linux kernel detects physical memory during boot by leveraging the E820 mechanism, where firmware reports memory ranges via interrupt 15H, enabling the kernel to map usable memory addresses for subsequent allocation.
The author, with over two decades of development experience, argues that many learners treat Linux kernel code as a rote exercise instead of mapping real‑world scheduling problems to code, and outlines essential questions about task_struct management, scheduling policies, CPU placement, fairness, and universal scheduler design.
The article provides a detailed technical analysis of Linux 5.9 on arm64 KVM, explaining how Host and Guest OSes are distinguished at EL1, how HCR_EL2 flags control mode switching, and how the EL2 exception vector and related registers are initialized during Linux boot to enable KVM virtualization.
This article explains the purpose, design, and internal data structures of ext4 extents, describes how the B+‑tree indexes extents for efficient mapping of logical to physical blocks, and compares ext4’s extent mechanism with older file‑system addressing methods and other modern file systems.
This article explains the complete boot sequence from powering on a PC, through UEFI firmware and the boot manager, to loading systemd‑boot as the boot loader and finally transferring control to the Linux kernel’s first executable code, detailing each component and its role.
This article introduces Linux's io_uring framework, explains its design goals and advantages over traditional I/O models, details its core components and system calls, provides step‑by‑step implementation examples for file and network operations, and discusses performance comparisons and practical application scenarios.
This article explains the principles of GPU kernel virtualization by analyzing the cgpu project's source code, detailing kernel interception of device operations, the driver’s file_operations, module initialization and cleanup, procfs interfaces, scheduling logic, and compilation steps on Ubuntu 22.04.
This guide walks through creating a 32‑bit Linux‑kernel memory allocator using a free‑list, implementing morecore and add_to_free_list functions, and then adding a basic mark‑and‑sweep garbage collector that scans heap, BSS, data segments and the stack to reclaim unused blocks.
A Microsoft‑submitted performance patch for the x86_64 architecture was merged into Linux kernel 6.13 without maintainer acknowledgment, breaking control‑flow integrity and nearly causing a system‑wide failure, prompting swift emergency fixes from Intel and AMD engineers and sparking a broader discussion on open‑source code‑review practices.
This article details a week‑long investigation of intermittent Nacos service deregistration in a Spring Cloud Alibaba microservice environment, describing the background architecture, multiple hypothesis tests, diagnostic commands, kernel version mismatch, and the final fix by upgrading the Linux kernel.
This article explains the full boot sequence from power‑on through UEFI firmware, the boot manager, and the systemd-boot loader, detailing how PE32+ UEFI applications, BootOrder variables, and EFI_BOOT_SERVICES functions load and start the Linux kernel.
This article explains how Rust's memory safety, concurrency guarantees, explicit error handling, and modern tooling make it a safer and more maintainable alternative to C for developing Linux kernel modules.
The article explains the purpose, implementation, and usage of early I/O memory mapping (early_ioremap) in the Linux kernel, detailing its initialization, fixmap foundations, code flow, and differences from regular ioremap, with examples and kernel source snippets.
This guide explains how to optimize Tomcat by adjusting maxThreads, acceptCount, maxConnections, and connectionTimeout, while also tuning Linux kernel parameters, TCP settings, and JVM options to achieve balanced CPU and I/O performance for high‑concurrency web services.
The article examines a long‑standing hidden bug in the Linux kernel’s page‑cache Xarray that caused occasional data loss with Large Folio support, details its discovery and fix by the TencentOS team, and shows how consolidating multiple tree walks into a single walk in Linux 6.10 reduced latency and improved performance by about ten percent.
The article outlines six distinct kernel‑level bugs affecting the eBPF tailcall feature across multiple Linux versions, explains the underlying causes and the commits that fixed them, and introduces a detection tool to verify whether a running kernel is affected.
This article explains the fundamentals of NUMA architecture, how Linux detects and represents NUMA nodes, the memory zone hierarchy, allocation policies, and practical techniques such as using numactl and taskset to bind processes for optimal performance on multi‑socket servers.
The author, with over two decades of development experience, argues that learning the Linux kernel requires understanding real-world scheduling problems rather than merely copying kernel code, and outlines key questions about task_struct management, priority, preemption, CPU placement, fairness, and universal scheduler design.
This article explains the kfunc mechanism behind Linux's sched_ext, covering instruction modification, verification, address fixup, and detailing common kfunc functions such as scx_bpf_dispatch, scx_bpf_consume, and queue management APIs.
Processes must pause execution to wait for resources, and the Linux kernel provides sleep and wake mechanisms—implemented via functions like schedule_timeout_interruptible and try_to_wake_up—to efficiently relinquish CPU time, avoid busy‑waiting, and resume when resources become ready.
This guide explains why and how to implement MDIO over GPIO on a Linux system, covering MDIO fundamentals, Clause22/Clause45 frame structures, YT8521 PHY pinout, required kernel drivers, device‑tree configuration, log verification, and command‑line usage.
Processes represent execution contexts that include code and resources; to avoid wasteful busy‑waiting, operating systems let them sleep and later wake them, using mechanisms like schedule_timeout_interruptible, timers, and try_to_wake_up, which the Linux kernel implements through specific state changes and scheduling logic.
This article provides a comprehensive overview of Linux memory barriers, explaining why they are needed for correct ordering of memory operations on modern multi‑core CPUs, describing the different barrier types (read, write, full), their implementation in the kernel and Java, and illustrating their use in synchronization primitives and lock‑free data structures with code examples.
This guide explains key GNU C extensions—including zero‑length and variable‑length arrays, case ranges, statement expressions, typeof, variadic macros, designated initializers, built‑in functions, and attribute syntax—showing how they differ from standard ANSI C and how to use them safely in Linux kernel development.
Linus Torvalds merged a small patch into the Linux kernel that streamlines the 64‑bit copy_from_user() routine by skipping the costly barrier_nospec() and applying masked user‑address handling, delivering a 2.6% speedup on Intel’s “will it scale” per‑thread‑ops benchmark and slated for inclusion in the upcoming Linux 6.12 stable release.
The 19th China Linux Kernel Developers Conference in Wuhan on October 26, 2024 attracted over 80,000 online viewers and nearly 400 developers, showcased five technical sub‑forums—from memory and storage I/O to virtualization and scheduling—featured AI‑focused kernel talks, announced a new open‑source innovation institute, and made all materials publicly available on GitHub.
At the 19th China Linux Kernel Developer Conference, vivo unveiled nine open‑source innovations—including DMA‑BUF DIO for AI model loading, ZHP memory compression, EROFS and f2fs filesystem enhancements, and I/O optimizations—contributing significant performance gains and demonstrating its commitment to Linux kernel advancement worldwide.
This article explains how zero‑intrusion eBPF technology enables detailed, non‑disruptive TCP network monitoring, covering data collection interfaces, aggregation methods, implementation steps, usage limitations, and practical installation and visualization guidance for improving network performance and fault analysis.
This article dissects the Linux kernel's TCP connection workflow, explaining how the three‑way handshake prevents stale SYN packets, and walks through the socket(), bind(), listen() and connect() system calls with detailed code analysis of the underlying kernel functions and data structures.
This article walks through removing the per‑CPU map used to pass the TCP‑option offset in an XDP program, shows the required code changes, explains the verifier errors that arise, and presents the final fix using an int offset with a bitwise mask.
This article provides a comprehensive overview of Linux's io_uring, explaining its design goals, shared‑memory mechanism, submission and completion queues, core system calls, performance advantages over traditional I/O models, typical use cases, and includes a complete example of a network server built with io_uring.
This article explains why packets are dropped in Linux, introduces the kfree_skb_reason API added in kernel 5.17, and shows step‑by‑step how to use bpftrace to capture drop reasons, five‑tuple details, and stack traces for precise network debugging.
At RustConf, Miguel Ojeda highlighted the challenges and optimism surrounding the Rust for Linux project, explaining how unstable Rust features, developer hesitancy, and industry interest shape the effort to make Rust a safe, second language for kernel development.
After two decades of development, the PREEMPT_RT real‑time patch finally merged into the mainline Linux kernel, highlighting the technical, political, and funding challenges that delayed its inclusion and explaining the benefits of deterministic, low‑latency performance for critical systems.
This guide explains how to optimize Nginx settings and adjust Linux kernel parameters—such as worker processes, connection limits, caching, HTTP/2, rate limiting, and TCP tweaks—to reliably handle massive traffic spikes while maintaining high availability and performance.
This guide explains how to replicate Linux kernel's initcall mechanism in STM32 projects by using function‑pointer sections, modifying the linker script, and defining ordered initialization macros to replace sequential main‑function calls, enabling modular and loosely‑coupled code execution.
The 19th China Linux Kernel Developers Conference (CLK 2024) in Wuhan on October 26 invites paper submissions from September 10‑25 on topics such as RISC‑V, ARM64, scheduling, memory, storage, networking, containers, AI, automotive and embedded Linux kernel optimization, with abstracts limited to 500 words and results announced September 30.
This article dissects the Linux kernel's page allocation mechanisms, explaining how alloc_pages() follows a fast‑path using low watermarks and falls back to a slow‑path that triggers kswapd, direct reclaim, and compaction, while also detailing the corresponding page‑freeing functions and their internal data structures.
This article provides an extensive overview of embedded software fundamentals, covering heap vs. stack differences, wild pointers, DMA roles, inter‑process communication methods, memory allocation strategies, malloc vs. new, volatile usage, pointer concepts, Linux kernel locks, FreeRTOS mechanisms, stack overflow prevention, compilation stages, quick‑sort algorithm, header inclusion, CAN identifiers, struct memory optimization, STM32 interrupt handling, user‑to‑kernel transitions, and condition‑variable thundering‑herd effects.
This article provides a comprehensive technical walkthrough of Linux's sched_ext scheduler extension, explaining how BPF enables custom scheduling policies, detailing the underlying CFS and EEVDF concepts, the new SCHED_EXT class, dispatch queues, kernel configuration, and practical code examples for building and testing BPF‑based schedulers.
This article explains the concepts of level‑triggered (LT) and edge‑triggered (ET) event notification in Linux epoll, describes epoll's core data structures such as the red‑black tree and ready‑list, details the three main APIs (epoll_create, epoll_ctl, epoll_wait), and examines the locking and internal kernel mechanisms that enable efficient I/O multiplexing.
The article recounts a real‑world incident where a device’s MAC address could not be changed because a previous engineer deliberately disabled the relevant function, discusses the discovery of the backdoor code, and examines the legal and ethical implications of such intentional vulnerabilities.
This article explains why a TCP connection's Send‑Q can exceed the user‑set SO_SNDBUF value, detailing the kernel's double‑buffer trick, sk_wmem_queued accounting, tcp_sendmsg behavior, GSO influence, and possible ways to limit the buffer growth.
A Linux client sets SO_SNDBUF to 4096 bytes, sends 1 KB packets while the server never reads, yet the TCP Send‑Q grows to 14480 bytes because the kernel doubles the buffer size, adds overhead, and expands memory via GSO and sk_wmem scheduling.
This article explains the thundering herd phenomenon in multi‑process/thread servers, details its performance costs, demonstrates the issue with accept() and epoll_wait() through C examples, explores thread‑level herd, and presents mitigation techniques such as accept mutexes, SO_REUSEPORT, and Nginx's handling.
This article explains Linux kernel crash diagnostics, covering Oops messages, BUG and BUG_ON macros, the die() and panic() pathways, preparation steps for reproducing bugs, kernel configuration options for debugging, and useful memory‑debugging utilities such as MEMWATCH, YAMD, Electric Fence and strace.
The Buddy Algorithm is a memory‑management technique used by the Linux kernel that splits physical memory into power‑of‑two blocks, tracks them with a binary‑tree structure, and merges free buddies to reduce external fragmentation while providing efficient allocation and deallocation.
The article examines Linux kernel 6.9 memory-management locks—PG_locked, lru_lock, mmap_lock, anon_vma rwsem, mapping i_mmap_rwsem, and shrinker_rwsem—explaining their roles and presenting eight community-driven optimizations such as per-memcg lru_lock, per-VMA locks, speculative faults, and lock-less shrinker techniques to improve concurrency and performance.
This article explains the concept, benefits, and implementation details of memory pools in the Linux kernel, covering allocation functions, design principles, common strategies, and a thread‑safe C++ memory‑pool class with example code.
Netcap, an open‑source eBPF‑based kernel network capture tool from ByteDance STE, lets developers trace packets across the entire Linux network stack, filter with tcpdump syntax, and extend functionality with custom filters, dramatically improving the efficiency of diagnosing kernel packet loss problems.
The openEuler 24.03 LTS release introduces a dynamic composite page (large folio) that retains 4 KB base‑page compatibility while enabling 64 KB page performance gains, reducing TLB misses and memory overhead, and delivering double‑digit benchmark improvements for big‑data, Kafka, MySQL, I/O and memory‑allocation workloads.
This article explains the background, purpose, and detailed implementation of the Linux kernel cgroup freezer subsystem, including its data structures, command‑line usage, signal‑based freeze workflow, and a practical experiment that demonstrates process suspension and power savings on ARM64.
This article explains how to extend eBPF to make virtio‑net NIC queue metrics observable, walks through the front‑end send/receive flow, defines key queue indices, integrates the probes into the rtrace tool, and demonstrates fault detection with real‑time data.
The article provides a detailed technical overview of Linux's new extensible scheduler class SCHED_EXT, explaining its architecture, eBPF‑based customization interfaces, core data structures, queue management, scheduling points, and a central‑scheduler example, while comparing it with traditional CFS and per‑CPU run‑queues.
The article examines Linux 6.1’s preemption mechanism, explaining latency sources, the three preemption configurations (none, voluntary, full), the TIF_NEED_RESCHED flag and preempt_count tracking, and how preempt_enable/disable affect real‑time responsiveness, illustrated by a case where RT threads cannot preempt CFS due to disabled preemption in critical driver code.
Linus Torvalds, the creator of Linux and Git, has upgraded from an AMD Ryzen Threadripper workstation to a powerful Ampere AArch64 system, boosting his ARM64 kernel testing to match x86‑64 builds and signaling a shift in hardware strategy for upcoming Linux 6.10 merges.
This article systematically explains epoll's core data structures—including a red‑black tree and a doubly linked ready list—its three main APIs, the kernel implementation details, thread‑safety mechanisms, ET vs. LT behavior, and how it improves over select/poll for high‑concurrency network programming.
A Linux client sets SO_SNDBUF to 4096 bytes and sends 1 KB packets while the server never receives them, yet the TCP Send‑Q grows to 14480 bytes, prompting an in‑depth analysis of kernel buffer accounting, double‑sized send buffers, and tcp_sendmsg behavior.
Through detailed code examples and step‑by‑step analysis, this article explains how the Linux 3.12 kernel implements file system readahead, showing how synchronous and asynchronous prefetch windows are initialized, adjusted, and used to improve sequential read performance.
This article explains why the TCP Send‑Q can grow beyond the user‑set SO_SNDBUF value by examining the kernel’s double‑buffer accounting, sk_wmem_queued handling, and the interaction of tcp_sendmsg with GSO, supported by code excerpts and diagrams.
