Rethinking “Add Machines” and “Traffic Shifting” as Effective Scaling Strategies

This article, originally published on the HelloJava WeChat account, examines two common scaling tactics—adding machines and traffic shifting—and argues that they are not low‑effort solutions.

First point: Adding machines – Early systems typically consist of a single application server and one or two database servers; when traffic spikes, the instinct is to simply add more machines. However, expanding an application tier from one to two servers introduces challenges such as maintaining consistent application state (e.g., user login information) across instances and handling clustering concerns like load balancing and health checks. Adding database servers is even more complex, requiring sharding or partitioning to actually relieve pressure; otherwise the added capacity is ineffective. As the number of application servers grows, database connection pools become bottlenecks, meaning that simply buying more hardware does not solve the problem.

When a system scales to hundreds or thousands of services, deciding how many machines each service needs and dealing with physical constraints (e.g., a data center that cannot house all new servers) become major architectural questions. A strong architect must balance manpower, expected growth, cost control, and technical debt to design an architecture that can rely on adding machines while preparing for the next scaling cycle.

Second point: Traffic shifting – Traffic shifting involves moving traffic from one data center to another, often within the same city, to quickly recover from failures. This requires solving technical issues such as seamless database address switching (master‑slave failover without restarting applications) and handling DNS VIP removal, which lacks health checks and may involve propagation delays. Shifting traffic to a remote data center adds latency challenges and typically demands multi‑active write setups to maintain data correctness.

Like adding machines, traffic shifting demands robust technical support but can dramatically improve system availability. With multiple data centers and fast traffic‑shifting mechanisms, any single‑site failure can be mitigated, extending overall uptime.