Inside Kvrocks: How This RocksDB‑Based Redis‑Compatible Store Handles Massive Data

Interview Overview

Lin Tianyi, PMC of Apache Kvrocks (Incubating) and technical manager at AfterShip, discusses Kvrocks – a NoSQL storage service built on RocksDB that implements the Redis protocol, targeting big‑data scenarios where stable, low‑cost storage is essential.

Why Large‑Scale Storage Matters

Massive data volumes create pressure on storage costs, require support for diverse data types, demand fast analysis of rapidly generated data, and hold high commercial value that must be preserved; lost data means lost opportunities.

Community Structure and Roles

Kvrocks follows an open‑source community model without formal employment relationships. Project direction is set annually by the Core Team, and tasks are taken by volunteers. The three main roles are:

Project Mentor : Bridges Apache and the incubating project, promotes Apache Way, and often contributes code (e.g., mentor tison).

Committer : Active contributors granted code‑merge rights, recognized for past contributions.

Contributor : Anyone who uses, suggests, files issues, or submits pull requests.

Data Replication and Cluster Design

Kvrocks uses an asynchronous replication model similar to Redis. The initial full sync transfers a snapshot of the DB to the replica; thereafter, the latest sequence number starts partial sync for incremental updates. This approach can cause inconsistency during active failover, which was addressed by implementing a global replication ID (see https://github.com/apache/incubator-kvrocks/issues/462 and PR https://github.com/apache/incubator-kvrocks/pull/538).

For clustering, Kvrocks is compatible with the Redis Cluster protocol, allowing existing Redis Cluster SDKs to connect. Unlike Redis’s decentralized topology, Kvrocks delegates node discovery to external services such as Zookeeper or ETCD, simplifying topology management.

Redis Benefits and Use Cases

Redis is widely used for storing user relationships, comment counters, and other high‑frequency data due to its rich data structures (hash, list, zset, set, string, HyperLogLog, bitmap) and low learning curve.

Open‑Source Ecosystem Insights

Lin notes that China’s open‑source landscape is maturing, citing projects like Doris, InLong, and ShenYu. He shares a personal story of a Ukrainian contributor ( https://github.com/torwig) helping during the Russia‑Ukraine conflict, highlighting the community’s pure passion.

Kvrocks Advantages

All data resides on disk, giving Kvrocks a cost advantage (about one‑tenth of Redis) and larger capacity (one to two orders of magnitude). It also offers strong Redis command compatibility, Lua scripting, and clustering support.

Elastic Scaling and Migration

Kvrocks migrates data at the slot level, unlike Redis’s key‑level migration, enabling more predictable scaling. Administrators can move slots using the CLUSTERX MIGRATE command (see https://mp.weixin.qq.com/s/Ev81UGz08aPu50s218ej_w).

Future Roadmap

The 2022 community plan (see https://github.com/apache/incubator-kvrocks/projects/2) includes:

Release of a cluster management service (project https://github.com/KvrocksLabs/kvrocks_controller) around July 2022.

Enrich data structures, starting with JSON support.

Performance and usability improvements, especially for public‑cloud deployments.

Security enhancements, notably TLS support.

Participation is encouraged, but no formal commitments are made.

Key Takeaways for Database Tools

From a user perspective, essential qualities are stability, ease of use, performance suited to specific scenarios, and good scalability for online expansion.