This article shares 360 Group's systematic Kafka capacity‑planning methodology, covering hardware performance analysis, disk I/O benchmarking, cluster configuration, load‑testing procedures, observed write‑read dynamics, and practical recommendations for reliable Kafka deployments.
This article provides a complete Bash script that gathers Linux system metrics—including IP address, CPU core count, utilization percentages, load averages, memory and swap usage, and disk I/O statistics—while explaining each command (ifconfig, grep, awk, top, vmstat, free, iostat, uptime) and how the script assembles the results.
This article explains how Kafka’s design uses the operating system’s page cache and sequential disk writes to achieve fast, reliable persistence, debunking the myth that disk I/O is inherently slow and outlining practical configuration tips.
This article analyses the security of Hadoop S3A write paths in data lakes, explains fast upload mechanisms, demonstrates disk‑IO and network‑error simulations, compares checksum algorithms, and presents Alibaba Cloud EMR JindoSDK best‑practice results with performance and reliability evaluations.
This article explores Kafka’s performance architecture, covering network and disk bottlenecks, sequential writes, zero‑copy techniques, page cache usage, Reactor‑based networking, batch processing, compression, partition concurrency, and file structures, providing practical optimization methods for high‑throughput streaming applications.
This article explains the physical structure of mechanical disks, partitioning strategies, inode and block allocation, and how filename length and directory layout affect Linux file system performance, providing practical experiments and tips for improving disk I/O.
The article analyzes real‑world network and disk I/O issues encountered in Qunar Message Queue (QMQ), explains root causes such as Netty OOM, file‑handle exhaustion, TCP timeout handling, and large‑traffic bursts, and presents practical mitigation strategies for backend systems.
This guide walks you through essential Linux performance diagnostics—examining CPU details, startup services, load averages, memory usage, and disk I/O—providing commands and practical solutions to identify bottlenecks and improve system responsiveness for your workstation.
