Architecture Evolution and Performance Optimization of Tencent's CKV Distributed In-Memory Storage System

This article presents a comprehensive technical overview of the architectural evolution and large-scale refactoring of Tencent's CKV distributed in-memory key-value storage system. Driven by the need to adapt to evolving business scenarios, hardware advancements, and cost optimization, the system underwent significant structural changes to enhance scalability, maintainability, and performance.

The architecture transitioned from a traditional three-tier model to a streamlined two-tier design, merging access and storage layers while introducing an optional Proxy component for flexible protocol compatibility. A major shift involved replacing the multi-process model with a single-process multi-threaded architecture. By aggregating network I/O, memory engines, and core logic into a unified process and implementing asynchronous programming, the system achieves lock-free data processing on dedicated CPU cores, effectively eliminating performance spikes and improving resource utilization in multi-tenant environments.

Deep compatibility with the Redis protocol was prioritized to leverage its simplicity and ecosystem, while addressing native limitations through self-developed innovations. The system introduces a robust multi-tenant architecture that consolidates multiple business datasets into single clusters, significantly reducing operational overhead. To mitigate concerns regarding resource contention and maintenance windows, the team implemented transparent hot-upgrade mechanisms and fine-grained internal resource isolation for connections, traffic, memory, and CPU. Furthermore, data consistency was strengthened by integrating Raft-based strong synchronization alongside eventual consistency, and the memory engine was optimized with shared memory management, dynamic allocation, and millisecond-level data loading to eliminate snapshot overhead.

Performance benchmarks demonstrate substantial improvements over native Redis, particularly in concurrent scenarios where CKV+ leverages multi-core parallelism for network and disk I/O, achieving significantly higher throughput. The optimized Proxy handles ASN and Memcached protocols at up to 1.2 million QPS per instance. By combining hybrid storage, CAS operations, and extended Redis cluster capabilities, CKV+ delivers enhanced usability, reliability, and cost efficiency. The article concludes that self-developed systems and open-source technologies are complementary, with CKV+ aiming to continuously refine its architecture and contribute valuable innovations back to the broader database community.