An Introduction to Distributed Architecture and Its Evolution

1. What is Distributed Architecture

Distributed systems are software systems built on top of a network, where each database node is highly autonomous with its own DBMS (cohesion) and appears transparent to users, who cannot tell whether data resides locally or remotely.

In a distributed data system, users perceive the data as a single unified store without needing to know about partitions, replicas, or the site where transactions execute.

Simply put, a distributed system presents a group of independent computers as a single unified system to the user, similar to using a single MySQL instance.

The whole system provides services transparently; users interact with it as if it were a single MySQL server. An example of a distributed MySQL middleware is MyCat, which handles high concurrency and large data volumes.

2. Applications of Distributed Architecture

1) Distributed File Systems

Examples include Hadoop HDFS, Google GFS, and Taobao TFS.

2) Distributed Caching Systems

Examples include Memcached, HBase, MongoDB, etc.

3) Distributed Databases

Examples include MySQL, MariaDB, PostgreSQL, and others.

4) Distributed Web Services

...

5) Distributed Computing

...

Case Study: MyCat Middleware

MySQL is widely used in e‑commerce and internet companies because it is free, open‑source, and supports horizontal scalability. Companies like Taobao, Tmall, and Vipshop use distributed systems to handle massive concurrent traffic and data. MyCat has been used in projects such as China Mobile’s billing settlement (processing 200 million records daily) and IoT platforms (processing up to 2.6 billion records with real‑time query interfaces).

Studying MyCat deepens understanding of distributed architecture, including related technologies such as ZooKeeper for consistency, HAProxy/keepalived for high availability, clustering, load balancing, and disaster recovery.

3. Recommended Resources

1. Large‑Scale Distributed Website Architecture Design and Practice (link)

2. Large‑Scale Website Technical Architecture: Core Principles and Case Studies (link)

3. Large‑Scale Website Systems and Java Middleware Practice (link)

4. Distributed Java Applications: Fundamentals and Practice (link)

4. Evolution of Distributed Architecture

1) Initial Stage

All resources (applications, databases, files) reside on a single server.

2) Separation of Application, Data, and File Services

As traffic grows, additional web servers are added, and application, database, and file services are deployed on separate machines.

3) Introducing Caching to Improve Performance

80 % of traffic typically accesses 20 % of data; caching (local and remote distributed) stores hot data in cache servers, reducing database load.

4) Application Server Clustering

After sharding databases, additional application servers are added; load balancers distribute requests across multiple servers to avoid single‑node bottlenecks.

5) Database Read‑Write Splitting

When write operations become a contention point, separating read and write traffic across different servers alleviates pressure.

6) Reverse Proxy and CDN Acceleration

CDN and reverse proxies cache content to speed up access and reduce backend load.

7) Distributed File Systems and Distributed Databases

When data volume grows, both file storage and databases are split and distributed across multiple nodes.

8) Adoption of NoSQL and Search Engines

Complex business needs lead to the use of NoSQL databases and search engines for flexible storage and retrieval.

9) Business‑Level Splitting

Systems are refactored by dividing functionality into independent services (vertical splitting) or extracting reusable components into distributed services (horizontal splitting).

10) Distributed Services

Common modules are extracted and deployed as shared services that multiple applications call, reducing duplicated connections and improving scalability.

5. Challenges Faced by Distributed Services

1) Managing an ever‑growing number of service URLs becomes difficult, and hardware load balancers become single points of pressure.

2) Service dependencies become tangled, making it hard to determine startup order.

3) Increasing call volume raises capacity questions: how many machines are needed and when to add them?

4) Communication overhead grows; identifying owners of failing services and understanding parameter contracts becomes harder.

5) Multiple consumers of a service raise concerns about maintaining quality of service.

6) Continuous upgrades can cause unexpected failures (e.g., cache bugs causing memory overflow); strategies such as graceful degradation or resource throttling are needed to limit impact.

Source: https://www.jianshu.com/p/585cba922b74

Copyright notice: Content is sourced from the internet; rights belong to the original author. We will credit the author and source unless proven otherwise.