Mastering Distributed Architecture: From Single‑Node to Consensus Algorithms

Distributed Architecture Overview

1. Evolution of Distributed Systems

1.1 Single‑Node Centralized

All components (App, DB, FileServer) run on one machine with low request volume.

1.2 Separation of Application and Data Services

App, DB, and FileServer are deployed on independent servers, still with low request volume.

1.3 Caching for Performance

Frequently accessed data is stored in a cache server, reducing database load.

1.4 Application Server Cluster

Multiple application servers provide services via load balancing, overcoming single‑machine capacity limits.

1.5 Database Read/Write Splitting

Master‑slave design separates reads and writes to alleviate database pressure.

1.6 Reverse Proxy and CDN Acceleration

Reverse proxy and CDN are used to speed up system access.

1.7 Distributed File System and Distributed Database

Adopt distributed databases and file systems when business growth outpaces read/write splitting.

Distributed Technology Details

1. Concurrency

2. Distribution

Large tasks are split across multiple machines for service delivery.

3. Lack of Global Clock

Time must be synchronized.

4. Peer Equality

Identical services run on multiple machines without differences.

5. Faults Are Inevitable

Hardware failures such as disk crashes or CPU burnout can occur.

Distributed Transactions

1. ACID

Atomicity: All operations in a transaction either complete fully or not at all; failures trigger a rollback.

Consistency: Database integrity remains intact before and after a transaction.

Isolation: Concurrency control prevents inconsistent data from interleaved transactions; levels include read‑uncommitted, read‑committed, repeatable‑read, and serializable.

Durability: Once a transaction commits, its changes persist despite system failures.

2. Two‑Phase (2P) / Three‑Phase (3P) Commit

2P ensures strong consistency via a prepare phase (vote) and a commit phase.

3P adds a pre‑commit phase to further separate the steps.

CAP Theory

Consistency: Data remains consistent across distributed nodes.

Availability: The system continues to serve requests even if some nodes fail.

Partition Tolerance: The system tolerates network partitions; failed nodes can be recovered and synchronized.

Only two of the three properties can be fully satisfied at once.

BASE Theory

Basically Available, Soft state, Eventual consistency.

Soft state allows intermediate states without affecting overall availability.

Paxos Algorithm

Roles: Proposer (submitter), Acceptor (receiver), Learner (observer).

Key rules: Acceptors must accept the first proposal if none received; proposals have monotonically increasing IDs; higher IDs override lower ones.

Protocol phases: Prepare (proposer asks acceptors), Accept (proposer sends proposal), and learning.

Zookeeper ZAB Protocol

Node states: looking (leader election), following (follower), leading (leader), observing (read‑only).

Two modes: crash recovery and message broadcast.

Leader assigns a unique 64‑bit zxid to each request, broadcasts it, collects acknowledgments, and commits once a quorum is reached.

Data synchronization ensures followers catch up to the leader’s log before becoming a true leader.

Source: Advanced Internet Architecture