Sharing insights on technical analysis and exchange, making life better through technology.
Kafka’s exceptionally high throughput stems from a combination of design choices—including sequential disk writes, zero‑copy data transfer, batch processing, partition‑level parallelism, OS page‑cache utilization, and producer‑side compression—that together minimize I/O overhead, CPU usage, and latency.
The article walks through estimating peak QPS for a 50‑million‑daily‑request system, applies safety margins, and proposes a concrete deployment stack—including LVS, Nginx, Docker‑managed business servers, Redis cluster, MySQL sharding, and Kafka/RocketMQ—to reliably sustain roughly 4,500 QPS.
This article outlines nine practical Redis scenarios—including hot‑data caching, distributed locks with Redisson, Bloom filters for cache‑penetration protection, delayed queues using ZSet, token‑bucket rate limiting, bitmap boolean statistics, UV deduplication via Set/HyperLogLog/Bitmap, geospatial indexing, and lightweight Stream queues—explaining their motivations, implementation steps, and trade‑offs.
The article compares Seata‑AT, TCC, and Saga distributed‑transaction patterns, explains their two‑phase or three‑phase mechanisms, outlines performance and consistency trade‑offs, and provides concrete guidance on which solution fits backend micro‑service scenarios such as core commerce, third‑party integration, or admin tools.
The article explains how massive simultaneous expiration of Redis keys can overload the single‑threaded server, causing brief request blocking, response‑time spikes, and even memory‑related issues in master‑slave setups.
The article explains ZooKeeper's herd effect, how temporary sequential nodes and chain watching reduce notification storms, how client failures are handled, and why most projects use Curator to simplify fault‑tolerant distributed lock implementations.
The article analyzes why simply adding Redis nodes cannot handle a sudden million‑QPS surge, then presents three practical solutions—key sharding, multi‑level caching with hot‑key detection, and distributed‑lock‑based fallback—to build a resilient high‑concurrency backend.
The article explains ThreadLocal's lifecycle, why userId disappears when spawning asynchronous threads, compares ThreadLocal, InheritableThreadLocal, and Alibaba's TransmittableThreadLocal, and shows how the latter prevents data contamination in thread‑pool reuse.
The article examines common pitfalls of naïve ad insertion in short‑video feeds, explains how cursor‑based pagination prevents duplicate ads, and outlines a client‑side mixing architecture that isolates services, meets strict latency requirements, and ensures accurate ad billing.
This article compares RabbitMQ, RocketMQ, and Kafka on throughput, latency, scalability, and reliability, outlining each system's core features and recommending suitable scenarios such as reliable messaging, high‑performance streaming, and large‑scale real‑time data processing.
