How ZestKV Redefines Cloud‑Native Serverless KV Storage: Design, Goals, and Use Cases

Traditional key‑value stores face capacity bottlenecks, slow scaling, complex operations, and high costs, especially in massive data, mixed workloads, and cloud‑native environments. ZestKV, launched by 360 Zhihui Cloud, addresses these issues as a serverless KV service derived from the classic high‑capacity database Pika.

Core Design Goals

ZestKV adopts a seven‑layer compute‑storage separation architecture aimed at breaking capacity limits, delivering extreme elastic scaling, ensuring stable high performance, providing high reliability, reducing cost, simplifying operations, and achieving strong multi‑tenant isolation.

Seven‑Layer Architecture

ZestManager : global cluster manager, persists metadata to ETCD for cluster‑wide consistency.

ZestPika : stateless read/write units that can scale elastically.

ZestProxy : traffic entry and routing gateway, horizontally scalable without state.

ZestSstserver : consumes logs, generates SST files, and uploads them to S3.

Log Service : independent high‑availability component that handles data change logs.

Object Storage (S3) : global persistence layer where RocksDB‑Cloud flushes full data.

Compact‑Runner : dedicated compaction node that offloads background pressure from compute nodes.

This layered design fully decouples compute, storage, logging, and background tasks, laying the foundation for elasticity, performance, and reliability.

Unlimited Capacity

By integrating RocksDB‑Cloud with S3, ZestKV offers theoretically unlimited storage, supporting petabyte‑scale scenarios. Pay‑as‑you‑go pricing shines in low‑QPS, large‑volume use cases, as cold data resides in S3 without consuming expensive compute resources.

Second‑Level Elastic Scaling

Metadata‑driven scaling : range‑sharding modifies only metadata, achieving scaling in seconds without physical data migration.

Automated elastic scheduling : a serverless manager combined with Kubernetes provides fully unattended scaling.

Seamless traffic switching : routing updates are applied uniformly, making scaling transparent to clients.

Stable P99 Latency

Compared with a Pika 3.5 cluster, ZestKV improves write throughput by 40 % and, thanks to architectural decoupling and asynchronous design, keeps P99 latency consistently stable, which is friendly to latency‑sensitive online services.

Zero Data‑Loss Guarantee

Abandoning asynchronous Binlog replication, ZestKV writes data instantly to a high‑availability log service and persists the full dataset to S3, completely avoiding data loss and meeting the reliability requirements of finance, government, and other critical domains.

Dual Isolation for Multi‑Tenant Environments

Data and resource isolation are achieved by assigning each tenant an independent database and dedicated shard, together with quota and traffic controls, ensuring both data security and resource fairness.

Full Redis Compatibility

ZestKV fully supports the Redis protocol, enabling zero‑code migration and dramatically reducing learning costs and migration risks.

Target Users and Typical Scenarios

1) Large‑scale, low‑QPS archival or cold‑data teams – benefit from S3 pay‑per‑use cost reduction of over 50 %.

2) High‑throughput, latency‑sensitive workloads – background tasks are offloaded, keeping P99 latency stable for content platforms, user centers, and real‑time recommendation.

3) Online services with traffic spikes – second‑level automatic scaling handles e‑commerce promotions, live streaming, and holiday traffic surges.

4) Mid‑to‑large enterprises with mixed workloads and tenant isolation – data + resource double isolation prevents resource contention and fault propagation.

5) Teams migrating from Redis or Pika – seamless Redis‑protocol compatibility allows migration without code changes, addressing the four major shortcomings of the legacy stacks.

Non‑suitable scenario: ultra‑low‑latency (microsecond‑level) pure‑in‑memory trading, where a pure in‑memory Redis deployment remains preferable.

Comparison with Traditional Pika

Storage architecture : local disk (capacity‑limited) vs. compute‑storage separation with S3 (unlimited).

Scaling : time‑consuming manual ops vs. seconds‑level automatic scaling without physical migration.

Reliability : asynchronous Binlog with loss risk vs. log + S3 dual persistence achieving zero loss.

Multi‑tenant support : data‑only isolation vs. data + resource double isolation.

Latency : noticeable P99 jitter vs. extremely stable P99.

Cost model : hardware‑based high cost vs. usage‑based pay‑as‑you‑go with lower storage cost.

Operations : manual management vs. serverless automation.

Ecosystem compatibility : Redis protocol compatibility only vs. full compatibility enabling seamless migration.

Conclusion and Outlook

ZestKV’s core logic builds on a compute‑storage separation foundation to fill Pika’s capacity, elasticity, reliability, and operations gaps while preserving Redis compatibility. It is purpose‑built for cloud‑native massive, elastic, multi‑tenant workloads. Future plans include evolving into a multi‑model database that supports tabular and time‑series data models.