Why You Can’t Have It All: Unpacking the CAP Theorem and Its Real‑World Implications

What is CAP?

CAP theory, also called the “CAP theorem”, was proposed by Eric Brewer and later formalized as a theorem.

Three properties of distributed systems

Consistency (C) : All replicas see the same data at the same time.

Availability (A) : The system continues to respond to reads/writes despite some node failures.

Partition tolerance (P) : The system tolerates network partitions, which forces a trade‑off between C and A.

Because partitions are inevitable, real systems must choose two of the three properties.

CA without P : If partitions are ignored, both consistency and availability can be guaranteed, but in practice partitions always occur.

CP without A : Guarantees consistency at the cost of availability; typical of traditional distributed transactions.

AP without C : Prioritises availability and partition tolerance, sacrificing consistency; many NoSQL databases follow this model.

Proof of the CAP theorem

Brewer’s conjecture was proved by Lynch and others in 2002, showing that a system cannot simultaneously provide C, A, and P. The proof uses a simple contradiction: if all three held, a partition would cause lost messages, breaking consistency.

Clarifying CAP

Later analyses argue that the “choose two” phrasing is a simplification. In most cases, when partitions are rare, systems can appear to provide both C and A, but when a partition occurs a strategy is needed:

Detect the partition. Enter an explicit partition mode that limits certain operations. Run a recovery process to restore consistency and compensate for errors.

The scope of consistency is limited to a “primary partition”; outside that scope the service may be unavailable. Systems such as Paxos and atomic multicast follow this pattern. Large‑scale services like Google’s Chubby and Megastore combine global consensus with high‑availability storage.

ACID, BASE, and CAP

ACID and BASE are related but distinct. BASE (“Basically Available, Soft state, Eventually consistent”) was introduced to handle partitions by relaxing consistency. The relationship between CAP and ACID is often misunderstood because the letters C and A refer to different concepts in each model.

What happens after a partition?

After a partition, the system must either:

Swap data types that support eventual consistency (e.g., CRDTs such as riak_dt).

Replay and merge operations (e.g., Bayou).

Apply constrained handling, like limiting ATM withdrawals during a partition.

These approaches illustrate the trade‑off between availability and consistency in real services.

Beyond CAP

Nathan Marz’s 2011 article “How to beat the CAP theorem” proposes redesigning storage systems so that updates become appends and only create/read operations exist, making the trade‑off more manageable.

OceanBase’s alternative path

OceanBase aims to support strong consistency (CA) while still tolerating partitions, using a hybrid architecture.

UpdateServer performance considerations

Typical databases have limited daily write volume; modern servers with large memory can handle the load.

TiDB

TiDB is a distributed SQL database inspired by Google’s F1, supporting both traditional RDBMS and NoSQL features.

Conclusion

CAP remains a cornerstone of distributed system design, but its interpretation can be extended. The “choose two” rule applies only under certain conditions; practical systems must balance consistency, availability, and partition tolerance according to their specific workloads.