Why NoSQL Matters: From Basics to LevelDB, RocksDB, and Beyond

Background of NoSQL

NoSQL first appeared in 1998 and gained significant attention in 2009 when it was promoted as a new direction for data storage. The term is commonly interpreted as “Not Only SQL”, meaning NoSQL databases complement relational databases rather than replace them.

select fun, profit from real_world where relational=false;

Typical scenarios that motivate the use of NoSQL include:

Schema evolution bottleneck: Adding a column to a massive MySQL table requires DBA intervention and long downtime.

Unstructured payloads: Storing JSON with an uncertain schema forces temporary extra columns in MySQL.

High‑concurrency access: Services that must handle billions of keys (e.g., user IDs) cannot achieve required performance with MySQL.

Comparison of MySQL and NoSQL

MySQL stores highly structured data; NoSQL stores unstructured or semi‑structured data.

MySQL uses a standardized SQL query language; many NoSQL systems lack a universal query language.

MySQL requires a predefined schema; NoSQL allows flexible schema evolution.

MySQL performance degrades with massive data volumes; NoSQL maintains high read/write throughput.

MySQL scalability is limited compared with the horizontal scalability of most NoSQL solutions.

Key Open‑Source Key‑Value NoSQL Projects

LevelDB

LevelDB is a single‑process, on‑disk key‑value store created by Google engineers in 2011. It implements only the basic Get, Put, and Delete operations.

LevelDB achieves high random‑write performance by using a Log‑Structured Merge (LSM) tree. Writes are first buffered in memory and later flushed to disk in sequential batches, converting random writes into sequential writes at the cost of slightly slower reads.

Core components of LevelDB:

MemTable: In‑memory skip‑list that holds recent writes.

Immutable MemTable: Frozen memtable awaiting flush to disk.

SST Files: Sorted String Table files organized into multiple levels (Level0 … LevelN).

Manifest Files: Metadata index describing the layout of SST files across levels.

Current File: Tracks the active manifest.

RocksDB

RocksDB is a Facebook‑maintained fork of LevelDB that adds many performance‑oriented enhancements while remaining API‑compatible.

Major improvements over LevelDB:

Batch retrieval of multiple keys and range queries.

Multi‑threaded compaction for better CPU utilization.

Compaction filters that discard irrelevant keys during merge.

Support for multiple compression algorithms (snappy, zlib, bzip2).

Incremental and full backup capabilities.

Pipeline‑style multiple MemTables for higher concurrency.

RocksDB introduces the concept of Column Families , each with its own MemTable and SST files while sharing WAL, Current, and Manifest files.

SSDB

SSDB is an SSD‑backed, Redis‑compatible key‑value store. It combines Redis‑like data structures with persistent disk storage, providing a low‑cost alternative to in‑memory Redis for large datasets.

Architecture highlights:

Uses LevelDB as the underlying storage engine.

Implements a translation layer that maps Redis commands to LevelDB operations.

Supports basic string, hash, list, and sorted‑set data types.

Pika

Pika is a C++‑based, multi‑threaded, Redis‑compatible storage system built on RocksDB. It offers persistence, large capacity, and migration tools for moving data from Redis.

Key characteristics:

Fully compatible with the Redis protocol.

Multi‑threaded design leverages multi‑core CPUs for higher throughput.

Provides full and partial synchronization mechanisms.

Includes migration utilities for seamless data transfer from Redis to Pika.

Conclusion

The article provides a concise technical overview of NoSQL concepts, compares them with traditional relational databases, and reviews several open‑source key‑value stores—LevelDB, RocksDB, SSDB, and Pika. Building a high‑performance, highly available industrial‑grade NoSQL system remains a challenging engineering task.