Evolution of Large-Scale Internet Architecture: From Monolith to Service Mesh

The article introduces the evolution of large‑scale internet architecture, briefly explaining the principles and characteristics of each architectural stage.

Internet products often face massive user volumes, billions of daily page views, petabyte‑level data storage, and the need for high availability, elastic scaling, and rapid iteration, which impose stringent requirements on system architecture.

The architectural progression moves from monolithic → cluster → distributed → microservices → service mesh.

01. Simple Architecture

Early internet applications had low traffic and small data volumes, allowing a single application server to meet demand while databases and file services ran on separate external servers. This basic setup is illustrated in the diagram below.

All business logic is handled by a single server; files and databases reside on external single servers. This architecture suffers from lack of fault tolerance, requires downtime for upgrades, and offers low availability, making it unsuitable for modern workloads.

02. Caching and Read/Write Separation

As traffic surges and data grows, I/O demands on application and database servers increase. To alleviate database pressure, read/write separation and sharding are employed. Read/write separation uses a primary‑replica setup where writes go to the master and reads to replicas. Sharding distributes data across multiple databases and tables.

Introducing a cache layer further reduces database load and improves performance. Caching evolves from local caches to clustered, highly available caches, providing fast access while supporting sharing and redundancy.

03. Static/Dynamic Separation

To further reduce backend pressure, static resources are separated from dynamic services. Static assets are hosted on CDNs, allowing users to fetch them from geographically close edge nodes, improving load times and decreasing backend load.

While this improves performance, it introduces challenges such as cache invalidation and coordination between front‑end and back‑end deployments.

04. Clustered High‑Availability Architecture

With growing user bases, single servers become bottlenecks. Clustering distributes load across multiple application servers, while load balancers route traffic. NoSQL databases are introduced to handle petabyte‑scale data and high concurrency.

Clients now access services through load balancers and CDNs, and GSLB (Global Server Load Balancing) directs requests to optimal nodes.

05. Business Splitting and Distributed Systems

Complex business scenarios lead to splitting large applications into smaller, domain‑specific services (e.g., product, order, user, cart). Each module is owned by a dedicated team, enabling independent development, deployment, and operation.

As the number of services grows, inter‑service communication relies on RPC or messaging, giving rise to distributed architectures capable of handling high concurrency and traffic.

06. Service‑Oriented Architecture (SOA)

SOA extracts common functionalities into independently deployed services, promoting reuse and loose coupling. Typical implementations use an Enterprise Service Bus (ESB) to handle protocol conversion, message routing, and integration of heterogeneous systems.

07. Microservices

Microservices refine SOA by decomposing applications into fine‑grained, independently deployable services built around business capabilities. Each service communicates via lightweight APIs (often HTTP/REST) and can be implemented with diverse languages and data stores.

Key supporting components include service discovery, API gateways, load balancers, centralized configuration, circuit breakers, and observability tools. Popular ecosystems comprise Netflix OSS, Spring Cloud, Dubbo, gRPC, Consul, Eureka, Kong, Zipkin, and others.

Containerization (Docker, etc.) simplifies deployment, scaling, and isolation of microservices.

Service mesh further abstracts service‑to‑service communication, providing traffic management, security, and observability without requiring changes to application code.

In summary, the microservice era delivers modularity, scalability, and rapid iteration, while still confronting classic distributed system challenges such as transaction management, monitoring, and fault tolerance.