Tagged articles

failure domains

2 articles · Page 1 of 1
Random Bulletin
Random Bulletin
Oct 8, 2026 · Operations

Scaling Service Restarts at 10M QPS: From Serial to Safe Parallel

At 10M QPS, serial restarts of 4,200 instances take 47 hours while naive parallel restarts cause cascading failures; this article details the evolution from graceful single-node shutdown and warm startup to batched parallel orchestration with health gates, restart-storm mitigation, stateful-service handling, and adaptive platform automation.

10M QPSadaptive platformbatched parallel
0 likes · 18 min read
Scaling Service Restarts at 10M QPS: From Serial to Safe Parallel
Random Bulletin
Random Bulletin
Sep 5, 2026 · Operations

Why Static 80% Thresholds Fail at 10M QPS: Dynamic Water Level Management

The article explains why fixed thresholds like 80% CPU are inadequate for large-scale systems, and introduces dynamic water level management that combines load, resource, service, and resilience signals with capacity profiling, trend prediction, and action latency to drive automated scaling, scheduling, rate limiting, and degradation in a closed loop.

10M QPSAuto ScalingSLO
0 likes · 31 min read
Why Static 80% Thresholds Fail at 10M QPS: Dynamic Water Level Management