Understanding KeyDB: Architecture, Multithreaded Design, and Active Replication

What is KeyDB?

KeyDB is a high‑performance branch of Redis that focuses on multithreading, memory efficiency, and high throughput.

In addition to multithreading, KeyDB includes features that are only available in Redis Enterprise, such as Active Replication, FLASH storage support, and the ability to back up directly to AWS S3.

KeyDB maintains full compatibility with the Redis protocol, modules, and scripting, preserving atomicity guarantees for scripts and transactions. Because KeyDB evolves alongside Redis, it is a superset of Redis functionality and can replace existing Redis deployments.

On the same hardware, KeyDB can execute roughly twice the number of queries per second as Redis while reducing latency by about 60 %. Active Replication simplifies hot‑standby failover, allowing writes to be distributed to replicas and enabling simple TCP‑based load balancing and failover. The high performance lets you do more with less hardware, lowering operational costs and complexity.

Exploring KeyDB

The KeyDB project is a fork of Redis. While Redis is a single‑threaded in‑memory key‑value store, KeyDB transforms it into a multithreaded system while remaining 100 % compatible with the Redis API.

Project Git repository: https://github.com/JohnSully/KeyDB

Public technical details are scarce; this article summarizes findings from reading the source code.

Multithreaded Architecture

Thread Model

KeyDB splits Redis’s original main thread into a main thread and multiple worker threads. Each worker thread is an I/O thread that listens on the port, accepts connections, reads data, and parses the protocol.

KeyDB uses the SO_REUSEPORT feature, allowing multiple threads to bind to the same listening port.

Each worker thread is bound to a CPU core and uses the SO_INCOMING_CPU feature to designate which CPU receives incoming data.

After protocol parsing, each thread operates on in‑memory data protected by a single global lock.

The main thread is also a worker thread (index 0 in the worker array) and performs tasks that only the main thread can handle.

The main thread’s primary work is performed in serverCron , which includes:

Processing statistics

Client connection management

Resizing and resharding of DB data

Handling AOF

Replication master‑slave synchronization

Cluster‑mode tasks

Connection Management

In Redis all connection management is done in a single thread. In KeyDB each worker thread maintains its own set of connections, and a connection must be created, used, and destroyed within the same thread.

Each connection gains an additional field to indicate the owning thread:

int iel; /* the event loop index we're registered with */

KeyDB maintains three key data structures for connection management:

clients_pending_write : per‑thread list of connections awaiting synchronous writes.

clients_pending_asyncwrite : per‑thread list of connections awaiting asynchronous writes.

clients_to_close : global list of connections that need to be closed asynchronously.

Separate synchronous and asynchronous queues are needed because some Redis APIs (e.g., pub/sub ) involve publishing from one thread to a client handled by another thread.

Synchronous sending is performed directly in the owning thread, as illustrated by the following diagram:

When asynchronous sending is required, KeyDB uses a pipe to pass messages between threads:

int fdCmdWrite; // write pipe
int fdCmdRead;  // read pipe

The local thread checks whether the client belongs to it; if not, it forwards the write request to the owning thread via the pipe, which then adds the write event to its event loop.

Some client‑close requests are not executed in the thread that owns the connection, so a global asynchronous close list is also maintained.

Lock Mechanism

KeyDB implements a spin‑lock‑like mechanism called fastlock .

The main data structures are:

struct ticket {
    uint16_t m_active;  // unlock count +1
    uint16_t m_avail;   // lock count +1
};

struct fastlock {
    volatile struct ticket m_ticket;
    volatile int m_pidOwner; // thread ID that currently holds the lock
    volatile int m_depth;    // recursion depth for the owning thread
};

Atomic operations such as __atomic_load_2 , __atomic_fetch_add , and __atomic_compare_exchange are used to compare m_active and m_avail to decide if the lock can be acquired.

fastlock provides two acquisition methods:

try_lock : attempts once and returns immediately on failure.

lock : busy‑waits; after 1 024 × 1 024 attempts it yields the CPU with sched_yield to avoid excessive spinning.

In KeyDB, try_lock is combined with the event loop: each client has a dedicated lock; before reading client data the thread tries to acquire the lock, and if it fails the operation is deferred to the next epoll_wait cycle.

Active‑Replica

KeyDB implements an active‑replica mechanism where each replica can be writable (non‑read‑only) and synchronizes data with other replicas. Key features include:

Each replica has a UUID to prevent circular replication.

A new rreplay API packages incremental commands with the local UUID.

Keys and values carry a timestamp‑based version number for conflict detection; if a local key has a newer timestamp than an incoming one, the write is rejected. The version is generated by left‑shifting the current timestamp by 20 bits and appending a 44‑bit auto‑increment counter.

Reference documentation: https://docs.keydb.dev/docs/commands