How KeeWiDB Solves Redis’s Memory Cost with Tiered Storage & Async I/O

Architecture Overview

KeeWiDB is a distributed KV‑store database designed to overcome the memory‑cost and scalability limits of Redis. Its architecture consists of a stateless proxy layer and a service layer composed of multiple server nodes that provide high availability, horizontal scalability, and fault tolerance.

Proxy Layer

Stateless Proxy nodes handle client connections, maintain routing information, and forward requests to the appropriate server nodes, allowing clients to interact with the cluster without awareness of topology changes.

Proxy maintains its own connection pool to backend servers, reducing the number of connections on the server side.

Supports read‑write separation, command interception and audit for high‑risk commands, hot upgrades, and horizontal scaling by adding more Proxy instances.

Service Layer

The service layer forms a cluster of Server nodes. Each shard contains multiple nodes arranged in a primary‑replica configuration. Slots are dynamically reassigned to support elastic scaling.

Thread Model and Concurrency

KeeWiDB adopts a single‑process multi‑threaded design. Each worker thread owns its own connection manager, transaction module, and storage engine, eliminating shared‑resource contention (Shared‑Nothing architecture).

To avoid lock contention during WAL and Binlog writes, the slot range is further split so each thread handles a distinct sub‑slot range.

Threads use asynchronous I/O to prevent blocking on disk operations, and a coroutine is created per client connection to enable synchronous‑style code while performing non‑blocking I/O.

Coroutines

Coroutines suspend execution during disk I/O and resume when the operation completes, allowing a single thread to keep busy processing other requests. Coroutine switches involve only user‑space context changes, which are cheaper than thread switches.

Storage Engine and Persistent Memory

Data is stored on a hierarchy: hot data resides in Persistent Memory (PMem), warm data on SSD, and cold data on lower‑cost SSD tiers.

PMem provides DRAM‑like latency with larger capacity, making it suitable for caching hot pages and WAL logs.

A high‑efficiency LRU algorithm evicts pages from PMem, and dirty pages are asynchronously flushed back to SSD.

Direct I/O bypasses the OS page cache to achieve predictable latency for file operations.

Replication Improvements

Replication uses a RelayLog on the replica to decouple command reception from replay.

Each Binlog record includes a seqnum (global commit sequence) and parent (max committed seqnum at transaction start) to form a (P, S) interval.

Records with non‑overlapping intervals are replayed concurrently, increasing replica replay QPS and ensuring the replica can keep up with the primary under heavy write loads.

Performance Highlights

Through tiered storage, multi‑threaded processing, coroutine‑driven async I/O, and enhanced replication, KeeWiDB achieves >180k QPS per node with millisecond‑level latency, supports TB‑scale data per node and PB‑scale clusters, and reduces overall memory cost compared to pure in‑memory Redis deployments.

Overall, KeeWiDB demonstrates how combining modern hardware (PMem), software architecture (Shared‑Nothing, coroutines), and storage strategies (tiered hot/cold data, direct I/O) can address the cost, capacity, and latency challenges of traditional in‑memory KV stores.