FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges

FeatureKV is a high-performance key-value storage system developed by WeChat to address two billion-scale challenges: supporting 1 billion queries per second for online recommendation feature retrieval and enabling the ingestion of 1 billion keys worth of data within one hour for offline batch updates.

The system separates concerns between DataSvr (responsible for writing data) and KVSvr (providing read-only service). DataSvr ingests data from a user-facing distributed file system (USER_FS), performs data formatting, routing, and indexing, then writes the resulting data files to FKV_WFS while updating metadata in Chubby. KVSvr polls Chubby for version changes, loads data from FKV_WFS into local memory or SSD, and serves read requests.

FeatureKV supports three table structures: MemTable (fully in-memory), IdxTable (in-memory ordered index), and BlkTable (block-indexed, enabling TB-scale storage). Write performance scales horizontally by adding more DataSvr instances, while read performance scales by increasing the number of Sects (read replicas) or Roles (storage shards). The system guarantees eventual consistency and provides versioned reads through a copy-on-write mechanism that ensures BatchGet returns a consistent snapshot.

Additional features include task-based batch write interfaces with failure retry, incremental/full updates, TTL support, historical version rollback, and robust fault tolerance across data center and storage failures. Deployment statistics show over 270 machines serving more than ten business units, achieving peak read rates of 11 billion features per second with sub‑15 ms latency for 96.3% of requests and a transactional BatchGet success rate of 99.999999%.

By decoupling offline batch writes from online read‑only service, FeatureKV solves the timeliness and consistency problems of scenarios such as user profiles, machine‑learning models, rule dictionaries, and indexing data, and has been widely adopted across WeChat’s ecosystem.