Mastering Distributed Architecture: Principles, Applications, and Evolution

1. What Is Distributed Architecture

A distributed system is a software system built on a network, where each database node is highly autonomous with its own DBMS (cohesion) and appears transparent to users, who cannot tell whether data resides locally or remotely.

In a distributed data system, users perceive the data as a single unified entity, unaware of partitioning, replication, or the location of transactions.

Simply put, a group of independent computers presents itself as a single cohesive system to the user.

Distributed systems provide services as a whole, with internal collaboration hidden from users, similar to using a single MySQL instance.

Example: the distributed MySQL middleware MyCat handles high concurrency and large data volumes.

2. Applications of Distributed Architecture

Distributed file systems – e.g., Hadoop HDFS, Google GFS, Taobao TFS.

Distributed caching systems – e.g., Memcached, HBase, MongoDB.

Distributed databases – e.g., MySQL, MariaDB, PostgreSQL.

Distributed web services.

Distributed computing.

Case study: MyCat middleware is used in large e‑commerce platforms to process billions of records daily, supporting real‑time queries for projects such as China Mobile billing and IoT data processing.

Studying MyCat deepens understanding of distributed architecture, consistency services like ZooKeeper, and high‑availability solutions such as HAProxy/keepalived.

3. Recommended Resources

Large‑Scale Distributed Website Architecture Design and Practice

Core Principles and Case Studies of Large‑Scale Website Architecture

Large‑Scale Website Systems and Java Middleware Practice

Distributed Java Applications: Fundamentals and Practice

4. Evolution of Distributed Architecture

1) Initial Stage

All resources (applications, databases, files) reside on a single server.

2) Separation of Application, Data, and File Services

Applications, databases, and files are deployed on independent resources.

3) Introducing Caching for Performance

Local and remote distributed caches store the hot 20% of data, reducing database load.

4) Application Server Clustering

Multiple servers behind a load balancer provide services, alleviating single‑server bottlenecks.

5) Database Read‑Write Splitting

Separating read and write traffic improves scalability.

6) Reverse Proxy and CDN Acceleration

CDN and reverse proxies cache content to speed up access and reduce backend load.

7) Distributed Files and Databases

Adopting distributed file systems and databases addresses massive data growth.

8) NoSQL and Search Engines

Introducing NoSQL databases and search engines handles complex data storage and retrieval needs.

9) Business Splitting

Vertical splitting isolates independent applications; horizontal splitting extracts reusable services into distributed components.

10) Distributed Services

Common modules are extracted and deployed on distributed servers for shared use.

5. Challenges of Distributed Services

1) Managing an increasing number of service URLs and hardware load balancers becomes difficult. 2) Service dependencies grow complex, making startup order unclear. 3) Scaling services raises questions about required capacity and when to add resources. 4) Communication overhead rises, complicating failure diagnosis and parameter agreements. 5) Ensuring quality of service for multiple consumers is challenging. 6) Upgrades can cause unexpected failures (e.g., cache errors leading to memory overflow), requiring strategies like degradation or resource throttling.