Evolution and Scaling Strategies for Large Websites: Architecture, Session Management, and Database Optimization

Recently my company invited a senior internet architect for an intensive two‑day training on large‑scale website architecture; the sheer volume of information prompted me to revisit the concepts and reflect on how website technology has evolved based on my own experience.

Defining a "large" website goes beyond raw traffic or concurrent users; it is a blend of technical difficulty and business requirements, where either side being hard enough justifies significant investment.

For example, static portals like hao123 can handle massive visits with very simple web technology—just static pages served from multiple machines—yet they illustrate that high traffic alone does not dictate architectural complexity.

In the early stages of a site, a minimal architecture (two application servers, a single database) usually suffices; redundancy is achieved by deploying the application on at least two machines and using a dedicated database server.

While this simple setup works, the cost of renting physical servers and a data center can be prohibitive for small teams, which is why many now prefer cloud platforms that abstract away hardware concerns, though they introduce a new dependency on the cloud provider’s reliability.

Deploying multiple servers serves two main purposes: ensuring high availability (if one server fails, others keep the service running) and increasing concurrency capacity (more servers can handle more simultaneous requests).

However, multi‑server deployments raise the challenge of maintaining user session state across machines; without a shared session mechanism, a user’s requests could be treated as unrelated.

Web containers store session data in memory and associate it with a session‑id cookie; to share sessions across servers, the session information must be synchronized, which many Tomcat clusters attempt by replicating session data.

Such replication consumes CPU and network resources, and as the number of servers grows the overhead can outweigh the benefits, leading to diminishing or even negative returns on concurrency.

A common solution is to externalize session storage to a dedicated cache server (e.g., Memcached) or a distributed cache, which reduces replication overhead while preserving session availability.

Early Taobao took a different approach by storing session data directly in the client’s cookie, eliminating server‑side synchronization at the cost of security concerns.

Static sites like hao123 avoid session handling altogether, freeing resources for request processing; similarly, Taobao’s cookie‑based sessions were a pragmatic way to achieve high concurrency without heavy server load.

In my company, incoming requests first pass through an F5 hardware load balancer (or a software alternative like LVS), which can also implement session stickiness—binding a session‑id to a specific backend server—to avoid costly session replication.

Ultimately, the core bottleneck for high‑traffic sites is storage; database performance limits become evident once other layers are scaled.

One effective technique is read/write separation: a master database handles writes while one or more replica databases serve reads, reducing contention on the primary node.

Even with replication, read replicas can become a bottleneck under heavy load, so adding a distributed cache for frequently accessed, rarely changing data further offloads the database.

When data volume grows beyond what databases can serve quickly, search technologies (inverted indexes, external search engines) are employed to provide fast, fuzzy queries, complementing traditional indexed database lookups.

In summary, scaling a large website involves a layered strategy: start with simple redundancy, move to external session stores, employ load balancers with stickiness, separate read/write databases, introduce caching, and finally adopt search indexing for massive data retrieval.