Master System Architecture: CAP Theory, Shared‑Nothing, Load Balancing & HA

1. System Architecture Design Theory and Principles

Here we introduce several common architecture design theories and principles used in medium to large distributed systems.

(1) CAP Theory

1. What is CAP

The famous CAP theorem, proposed by Brewer, stands for Consistency, Availability, and Partition Tolerance.

(1) Consistency: after a successful update, all nodes see the same data at the same time.

This consistency differs from the transaction consistency in traditional RDBMS.

In traditional RDBMS, transactions have ACID properties: Atomicity, Consistency, Isolation, Durability.

ACID is the fundamental principle of relational databases; adhering to ACID emphasizes consistency but incurs high cost and performance impact.

a. Atomicity: a transaction is an atomic unit; its modifications either all succeed or all fail.

b. Consistency: data must remain consistent at the start and end of a transaction, enforcing all rules and internal structures.

c. Isolation: the system provides isolation so intermediate states are invisible to other transactions.

d. Durability: once a transaction commits, its changes persist even after failures.

MIT's Gilbert and Lynch reinterpreted Consistency as Atomic in their proof of CAP.

(2) Availability: reads and writes always succeed.

Availability means the service can always respond to requests within a reasonable time.

(3) Partition Tolerance: the system continues to operate despite network partitions.

In practice, node crashes or network splits should not cause the distributed system to stop.

2. How to prove CAP?

The proof is simple: assume two node sets G1 and G2 are partitioned.

If a write occurs in G1, a read from G2 cannot return the just‑written value.

Partition tolerance is a basic requirement for distributed data systems.

Because of the availability requirement, G2 must return a response, making consistency impossible to satisfy simultaneously.

CAP theorem tells us a distributed system cannot satisfy consistency, availability, and partition tolerance all at once; at most two can be achieved.

Consequently, any horizontal scaling strategy must trade off between consistency and availability.

3. CAP extension: BASE

BASE stands for Basically Available, Soft state, Eventually consistent, extending the C and A of CAP.

(1) Basically Available: data consistency meets the 80/20 rule, guaranteeing at least 80% consistency.

(2) Soft state: the state may be out‑of‑sync for a period.

(3) Eventual consistency: data may be inconsistent for a short time but will become fully consistent eventually.

BASE originated from e‑commerce practice and is an evolution of CAP, sacrificing strong consistency for basic availability and soft reliability while aiming for eventual consistency.

CAP and BASE are foundational theories for NoSQL in the Internet era.

(2) Shared‑Nothing Architecture

1. What is Shared‑Nothing Architecture

Shared‑Nothing Architecture (SNA) is a distributed computing architecture where each node is independent, self‑sufficient, and does not share memory or disk storage.

In summary, SNA eliminates centralized state, making each node autonomous and removing resource contention, which provides strong scalability and is widely used in web applications.

A key practice is to avoid using Session state in distributed systems, as synchronizing session data across nodes degrades performance.

2. Comparison

Shared‑nothing, shared‑memory, and shared‑disk are common parallel system models.

Shared‑memory : multiple CPUs share the same memory and communicate via internal mechanisms.

Shared‑disk : each CPU has private memory but can directly access all disks.

Compared with the other two, shared‑nothing scales better for many‑user parallel access, reducing response time and increasing throughput.

3. Sharding

Shared‑nothing requires a sharding strategy to reduce resource contention. Three basic sharding structures:

(1) Functional sharding – based on non‑overlapping features; used successfully by eBay.

(2) Key‑value sharding – find a key that distributes evenly across shards.

(3) Lookup table – a node acts as a directory to locate data; can become a bottleneck.

Open‑source DALs such as CobarClient, Fastser‑DAL, Uncode‑DAL implement routing with a lookup table.

4. Current status

SNA is widely used in systems like MapReduce, BigTable, Cassandra, MongoDB, etc.

(3) ED‑SOA Architecture

ED‑SOA (Event‑Driven Service‑Oriented Architecture) has become the standard for large enterprises with complex business and relational interactions.

ED‑SOA combines SOA (expose and handle) with EDA (event‑centric) to enable asynchronous communication and improve scalability.

Building loosely‑coupled systems with ED‑SOA significantly enhances website scalability.

2. Load Balancing

1. What is Load Balancing?

Load balancing distributes concurrent service requests across multiple backend servers with similar capacity, increasing throughput, flexibility, and availability.

Typical scenarios include web clusters and MapReduce workloads.

2. Load Balancing Algorithms

Algorithms dictate how a load balancer selects a backend service.

Common algorithms include:

(1) Round Robin / Weighted Round Robin – cycles through servers; simple and stateless.

Formula: i = (i + 1) mod n

(2) Hash – computes a hash of a request attribute (e.g., source IP) and mods by server count.

Formula: idx = Hash(key) % n

Key selection examples:

a. Timestamp or random number – simple but unstable.

b. Source IP – simple; works well if clients are distributed.

(3) Consistent Hash – used for distributed caches like memcached or Redis.

(4) Least Connection / Request – routes to the server with the fewest active connections.

(5) Most Idle – based on CPU, memory, bandwidth metrics.

(6) Fastest Average Response – selects server with lowest average latency.

(7) Least Traffic – selects server handling the least traffic.

Session‑based load balancing records a SessionID on the first request and forwards subsequent requests with the same SessionID to the same backend.

3. Load Balancing Modes

Modes refer to where in the stack the load balancing is implemented.

(1) External (RR‑DNS) – DNS round‑robin distributes traffic across multiple IPs; fast but cannot detect backend health.

(2) Application layer – uses forward or reverse proxy (e.g., Apache, nginx) to balance traffic; cost‑effective.

(3) Network layer – IP translation (NAT, IP tunneling, direct routing) or dedicated appliances (e.g., F5); high performance but higher cost.

LVS (Linux Virtual Server) is a popular open‑source network‑layer solution offering NAT, TUN, and DR modes.

3. High‑Availability System Design

1. System Availability

Availability = MTTF / (MTTF + MTTR) × 100%.

MTTF = mean time to failure; MTTR = mean time to restoration.

High availability aims to minimize planned and unplanned downtime.

2. HA Patterns

Common designs include:

(1) Active‑Standby – primary handles traffic; standby takes over on failure.

(2) Active‑Active – multiple primaries monitor each other and take over instantly.

(3) Cluster – many identical nodes provide transparent service, coordinated by a controller (e.g., Zookeeper).

3. HA Design Principles

There is no perfect solution, but a key metric is to eliminate single points of failure.

In practice, focus on the most failure‑prone components (network, storage) and mitigate them with caching, queues, sharding, load balancing, and disaster recovery.

Even with careful design, physical damage, permission errors, or DDoS attacks can still cause outages.