Why SQLite Became the World’s Most Popular Embedded Database

SQLite’s ubiquity

SQLite is the most widely deployed database engine. It is embedded in virtually every smartphone, desktop operating system (macOS, Windows), major web browsers, set‑top boxes, and countless desktop applications. A simple file‑system search for "*.db" on a typical computer reveals thousands of SQLite database files.

Origin of SQLite

In 2000 Richard Hipp was asked to write control software for a U.S. Navy destroyer that managed ship valves. The existing solution relied on an external IBM Informix server, which could become unavailable and halt the system. To eliminate the network dependency, Hipp’s team decided to store all data locally on disk and load it into memory, creating an embedded database that lives in the same process as the application. When a colleague suggested writing the database himself, Hipp did so, and SQLite was born.

First version: a GDBM shell

The initial release wrapped the GNU Database Manager (GDBM) – a key‑value store derived from the original Unix DBM – with an SQL layer. The README of SQLite v1 declared: SQLite: An SQL Database Built Upon GDBM GDBM provided CRUD functions but no SQL support; SQLite added a tokenizer, parser, and code generator on top of the GDBM API.

Layered architecture

User Interface (UI) layer

Tokenizer layer

Parser layer

Code Generator layer

Virtual Database Engine (VDBE) layer

Database Back‑End (DBBE) layer

An SQL statement passes through each layer before reaching the underlying storage.

Example: processing an INSERT

The UI passes the text to the Tokenizer, which splits it into tokens via sqliteGetToken() (e.g., [INSERT, INTO, examp, VALUES, 'Hello, World!', 99]).

The Parser matches the token sequence against grammar rules and invokes the handler sqliteInsert().

The Code Generator translates the statement into bytecode instructions such as Open examp, Integer 99, and Put.

The VDBE executes each opcode; for Put it calls sqliteDbbePut().

The DBBE layer finally calls gdbm_store() to write the record into the underlying GDBM file.

This flow turns a high‑level SQL command into a series of low‑level operations that store data in a hash‑based file.

Transition to B‑tree storage

In 2001 Hipp replaced GDBM with his own B‑tree implementation, removing the external dependency and providing a true relational storage engine. The commit history in the SQLite source tree shows the diff and the timeline of this replacement.

References

Building SQLite v1: https://corecursive.com/066-sqlite-with-richard-hipp/#building-sqlite-v1

First‑version commit timeline: https://sqlite.org/src/timeline?a=2000-05-01

Source tree for v1: https://www.sqlite.org/src/tree?ci=e8521fc10dcfa02f

B‑tree replacement timeline: https://sqlite.org/src/timeline?a=2001-04-14