Mastering Long-Connection Load Balancing: Strategies and Pitfalls

Long Connection Overview

Short vs long connections differ in lifecycle; short: connect → transfer → close; long: connect → transfer repeatedly → close. Long connections reduce DNS/TCP handshake overhead and enable server push, but require more memory and CPU and are harder to manage.

Why Load Balance Long Connections

Single‑machine long‑connection count is limited mainly by CPU and memory. Each connection consumes several KB to MB; a 128 GB machine can hold only tens of thousands of 1 MB connections. To achieve high availability and scalability, multiple nodes are deployed, necessitating load‑balancing strategies.

Granularity of Load Balancing

Long connections can be balanced at request granularity (choose a server for each request) or at connection granularity (choose a server when the connection is established and keep using it). The choice depends on expected connection count per server.

Connection‑Level Load Balancing

When connections persist, imbalance can occur, especially after server restarts. Using a “minimum‑connection” algorithm can help, but may need additional measures such as periodic global checks and forcibly closing excess connections.

Server hardware differences also matter; weighting servers by capacity can prevent weaker machines from being overloaded.

Horizontal scaling may fail if each node still processes all service changes; routing consumers to specific nodes can reduce duplicated work.

Conclusion

The article explains short vs long connections, why long‑connection services need load balancing, and common problems with practical solutions.