Evolution and Core Principles of Large‑Scale Website Architecture

Evolution of a Website

The author repeatedly emphasizes that large websites evolve from business needs rather than being designed up front, and that small companies must iterate quickly to survive.

Initial Stage Architecture

When traffic is low, the application, database, and file resources reside on a single server.

Separation of Application and Data Services

As traffic grows, the application server is split from the data (file and database) servers. Each server type has distinct hardware requirements: fast CPU for application servers, large disk and bandwidth for file servers, and fast disks with ample memory for database servers.

Using Caching to Improve Performance

Adding cache layers (local in‑process cache and remote distributed cache) accelerates data access and reduces database load.

Application Server Clustering for Concurrency

When a single application server becomes a bottleneck, clustering multiple stateless servers behind a load balancer enables horizontal scaling.

Database Read‑Write Splitting

Even with caching, cache misses require database access; read‑write splitting uses a master for writes and replicas for reads, often hidden behind a data‑access module.

Reverse Proxy and CDN Acceleration

Both reverse proxies and CDNs act as caches; CDNs are deployed at ISP data centers, while reverse proxies sit in the site’s own data center, both aiming to reduce latency and backend load.

Distributed File Systems and Distributed Databases

Physical disk capacity and single‑node database throughput are limited, so distributed storage and databases (often behind a proxy layer such as Cobar or Mycat) are required.

NoSQL and Search Engines

NoSQL stores provide flexible, horizontally scalable data storage, while dedicated search engines offload query work from relational databases.

Business Splitting

Large sites can be divided into multiple independent applications, linked via hyperlinks, message queues, or shared data stores.

Distributed Services

Common business functions (e.g., user management) are extracted into independent services, enabling service‑level monitoring, throttling, and graceful degradation.

Core Elements of Large‑Scale Site Architecture

The five key concerns are performance, availability, scalability, extensibility, and security.

Architecture Patterns

Patterns describe recurring problems and reusable solutions; they provide repeatable designs for the core concerns.

Layered Architecture

Divides the system horizontally into presentation, service, and data layers, promoting decoupling, independent evolution, scalability, and extensibility, but requires careful boundary definition.

Segmentation

Vertical splitting creates high‑cohesion, low‑coupling modules.

Distributed Deployment

Enables horizontal scaling but introduces network latency, consistency, and operational complexity.

Clustering

Multiple machines solve problems that a single machine cannot, especially when services are stateless.

Caching

Storing data close to the compute layer speeds up access and reduces backend pressure; used in CDNs, reverse proxies, local caches, and distributed caches.

Asynchrony

Producer‑consumer models decouple components, improve availability, response time, and enable traffic shaping.

Redundancy

Provides high availability and data reliability.

Automation

Automates build, test, deployment, monitoring, alerting, failover, and degradation to increase productivity.

Performance

Performance is pursued from three perspectives:

User view – response time; optimizations include browser tweaks, compression, HTTP keep‑alive, CDN, reverse proxy.

Developer view – system latency, throughput, stability; optimizations include caching, asynchrony, clustering, code tuning.

Operations view – infrastructure performance and resource utilization; optimizations include custom networks, servers, virtualization.

Key metrics are response time, throughput, and concurrency, illustrated with a highway traffic analogy.

Web Front‑End Performance

Reduce HTTP requests, leverage browser cache, compress assets, use CDN and reverse proxy.

Application Server Performance

Four main tactics: caching, clustering, asynchrony, and code optimization.

Cache

Cache placement, consistency (TTL or versioning), and handling cache penetration by caching null results.

Code Optimization

Use multithreading to avoid I/O blocking and fully utilize CPU cores; employ resource pools such as thread pools and connection pools.

High Availability

Availability is measured as (1‑downtime/total time) × 100%; industry uses “nines” to express reliability.

High‑availability architecture relies on redundant data and services with failover mechanisms.

Application Layer

Load balancers detect unhealthy servers and route traffic to healthy ones; stateless services simplify HA, while session state can be handled via sticky sessions, cookies, or dedicated session stores (e.g., Redis).

Service Layer

Uses distributed service frameworks with service registry, heartbeat detection, client‑side load balancing, timeout settings, async calls, degradation, and idempotent design.

Data Layer

Distributed file systems and databases provide redundancy and failover; consistency is addressed via replication protocols, and failover involves detection, traffic redirection, and data recovery.

Scalability

Scalability means the system can grow or shrink by adding or removing servers without redesign.

Application Layer Scalability

Stateless design plus clustering and load balancing enable horizontal scaling.

Cache Scalability

Consistent hashing distributes keys across cache nodes, minimizing impact when nodes are added or removed.

Data Layer Scalability

Relational databases rely on distributed proxies; NoSQL systems prioritize high availability and scalability over strict ACID guarantees.

Extensibility

Modular design reduces coupling and increases reuse; achieved through distributed message queues and services.

Distributed Services

Vertical splitting creates independent applications; horizontal splitting extracts shared functionality into services with stable interfaces.

Service Governance Framework

Provides registration, discovery, load balancing, failover, efficient RPC, version management, and real‑time monitoring.

Other Thoughts

Problems arise when user experience falls short of expectations; they can be addressed by improving the experience or lowering expectations, and by framing issues as shared concerns rather than personal criticisms.