Where Does Database Innovation Come From? Exploring the Future of Distributed Databases

Sources of Database Innovation

Database innovation originates from two main drivers:

Intrinsic shortcomings: Fundamental issues such as the lack of true serializable isolation in commercial systems (e.g., Oracle) motivate research to achieve perfect correctness and performance.

Integration with emerging technologies: The rapid development of AI, especially large‑language models (LLMs) like ChatGPT, creates the AI4DB and DB4AI movements. LLMs can generate SQL, manage query results, provide observability, and automate scheduling, thereby improving database usability.

Third‑Generation Distributed Database

Under the “three‑high‑one‑easy” requirement (high performance, high availability, high correctness, easy to use), a new generation of distributed databases should embody three core technical characteristics.

1. Minimal Usability

Understandability: All theoretical foundations must be fully explainable, reducing learning cost for developers and operators.

Maintainability: The architecture should allow components to be added, removed, or evolved without breaking the system, ensuring long‑term viability.

Usability (Automation & Intelligence): The system should automatically analyze inputs/outputs, perform intelligent scheduling, and minimize manual DBA effort.

2. Controllable‑Latency High Availability

High availability must be designed together with latency constraints, which vary from nanoseconds to seconds depending on the workload.

Geographic HA: Cross‑region disaster recovery using consensus protocols such as Paxos or Raft.

Node‑level HA: Primary‑standby replication or consensus‑based replication to tolerate node failures.

Transactional HA: Apply the PACELC model (proposed by Daniel J. Abadi) to balance consistency, latency, and partition tolerance for distributed transactions.

Different latency tiers (nanosecond‑level, millisecond‑level, second‑level) require distinct architectural choices; for example, traditional single‑node databases like MySQL or PostgreSQL typically achieve ~20 ms transaction latency, which is insufficient for ultra‑low‑latency applications when scaled out.

3. 100% Data Correctness

The system must guarantee absolute data correctness for all operations while preserving performance. This eliminates the primary source of concern for application developers and aligns with the CAP theorem’s consistency requirement.

These three pillars—minimal usability, controllable‑latency high availability, and 100% data correctness—are intended to guide the design of third‑generation distributed databases and are consistent with CAP and PACELC principles.