Scaling a Growing Website: 8 Architecture Evolution Steps from Physical Separation to Distributed Caching

Step 1: Physical Separation of Web Server and Database

Initially the site ran on a single host, but as traffic grew the database and application began contending for resources, causing slow responses. By moving the web server and database onto two separate machines, the system recovered its original speed and could handle higher load without mutual interference.

Technical knowledge required: none beyond basic server provisioning.

Step 2: Adding Page Cache

When traffic increased again, database queries became the bottleneck. Introducing a reverse‑proxy cache such as Squid (or static page generation) allowed static pages to be served without hitting the web server or database, reducing load and improving response time.

Relevant knowledge: front‑end caching mechanisms, cache invalidation strategies.

Step 3: Adding Page Fragment Cache (ESI)

After page‑level caching, dynamic pages still contained static fragments that caused unnecessary database hits. Implementing Edge Side Includes (ESI) allowed those fragments to be cached separately, further reducing backend load.

Relevant knowledge: ESI syntax, fragment‑level caching.

Step 4: Data Caching in Memory

Repeatedly fetching user information prompted the introduction of in‑process memory caches (e.g., hash maps) to store frequently accessed data, cutting database round‑trips and restoring fast response times.

Relevant knowledge: cache data structures, eviction algorithms.

Step 5: Adding a Second Web Server (Load Balancing)

To improve availability and handle peak traffic, a second web server was added. Challenges addressed included request distribution (Apache mod_proxy, LVS), session state synchronization (database, cookies, shared storage), cache consistency, and shared file handling (NFS or similar).

Relevant knowledge: load‑balancing algorithms, high‑availability techniques, session replication.

Step 6: Database Sharding

When write‑heavy operations caused connection contention, the database was split into multiple shards. Application code was modified to route queries to the appropriate shard, resulting in reduced contention and faster performance.

Relevant knowledge: business‑driven data partitioning, sharding strategy, impact on schema design and maintenance.

Step 7: Table Partitioning, DAL, and Distributed Cache

Even after sharding, some queries remained slow, prompting table partitioning and the creation of a Data Access Layer (DAL) to encapsulate sharding logic. Growing data volumes also made local cache synchronization impractical, leading to the adoption of a distributed caching system.

Relevant knowledge: dynamic and consistent hashing, DAL design patterns, distributed cache technologies.

Step 8: Scaling Out More Web Servers

With database pressure alleviated, traffic spikes again saturated the Apache front‑ends. Adding more web servers required addressing soft‑load‑balancer limits, possibly upgrading to hardware load balancers (F5, Netscaler) or segmenting applications across multiple load‑balancer pools. Additional challenges included scaling session and file‑sharing mechanisms, sometimes necessitating custom distributed file systems.

Relevant knowledge: advanced load‑balancing, high‑availability architectures, custom storage solutions.