From Short Connections to Connection Pools: Managing Millions of QPS
A midnight alert revealed latency jumping from 20 ms to 200 ms despite healthy CPU and memory, leading to an investigation that uncovered TCP connection handling—handshake delays, TIME_WAIT buildup, and CPU overhead—as the hidden bottleneck, and explains why short connections must evolve into pooled, globally managed connections at tens of millions of QPS.
