This article explores Olric, a pure‑Go distributed key‑value engine, detailing its dual embedded/stand‑alone mode, clean three‑layer architecture, core data structures, and engineering choices that illustrate best practices for building high‑performance, maintainable backend systems.
This article walks through the core concepts of the Raft consensus algorithm, explains why Elixir is a good fit, details the leader election and log‑replication mechanisms, and shows how to integrate a user state machine to create a fully functional distributed key‑value database with snapshots and recovery.
Bilibili transformed its search indexing pipeline by replacing a manual, low‑throughput process with a distributed KV store (Taishan) and Spark‑based construction, achieving unified data ingestion, reduced resource consumption, faster full‑ and incremental builds, and a shift from daily to hourly indexing cycles.
This article details Bilibili's transformation of its search offline indexing pipeline, moving from manual MySQL‑based processes to a high‑capacity, distributed KV store and Spark‑driven builds, addressing performance, maintenance, and scalability challenges while improving resource efficiency and iteration speed.
This article examines Alibaba's internal Doris KV storage system, detailing why large companies build proprietary data products, the project's kickoff criteria, the two‑layer architecture, virtual‑node routing, failover mechanisms, and cluster scaling strategies for massive KV workloads.
This article details how Bilibili’s relationship‑chain service evolved from a MySQL‑based design to a KV store with multi‑layer caching, introducing bloom‑filter‑enhanced Redis caches and hotspot mitigation techniques to sustain near‑million QPS traffic while maintaining data accuracy and high availability.
Bilibili’s relationship‑chain service, which handles follows, blacklists, whispers and mutual follows, migrated from a single sharded MySQL instance to an internal distributed KV store and introduced a three‑tier cache (memcached, Redis and a Bloom filter) plus automated hotspot routing, achieving near‑million QPS, lower latency, and preparing for multi‑tenant reuse.
This article explains how Bilibili’s KV storage system evolved from early Redis/Memcache solutions to a custom distributed KV architecture, detailing design goals, architecture components, shard management, Raft replication, multi‑active disaster recovery, and the operational challenges solved to support massive traffic growth.
The article details a Redis‑compatible, disk‑based KV service built atop TiKV using a compute‑storage split (Tula), describes custom key encoding and expiration mechanisms, and explains four optimization stages that introduce slot‑based hashing and adaptive concurrency to dramatically cut garbage‑collection latency while preserving write performance.
The study benchmarks Tencent Cloud KeeWiDB with YCSB workloads, showing that its Redis‑compatible KV service delivers up to 149 k QPS reads and 127 k QPS writes for 128‑byte strings, out‑performing self‑built Pika and Redis clusters, though larger values suffer bandwidth throttling, confirming KeeWiDB as a low‑cost, high‑performance option for small‑value, read‑heavy use cases.
KeeWiDB, a next‑generation distributed KV store from Tencent Cloud, combines a proxy layer, multi‑node service layer, tiered hot‑cold data placement, multi‑threaded and coroutine‑based processing, persistent memory caching, direct I/O, and enhanced replication to address Redis’s memory cost, scalability, and latency challenges.
RedKV is Xiaohongshu's self‑developed NVMe‑SSD based distributed NoSQL KV store that combines Gossip and Shard control, delivering petabyte‑scale storage, near‑100 million QPS, multi‑cloud elasticity, low‑cost Redis compatibility, and advanced features such as token‑bucket rate limiting, online compression, backup‑read tail‑latency mitigation, and robust backup‑recovery mechanisms.
ByteDance’s NoSQL ecosystem, spanning KV stores like ABase, document databases, columnar systems, and a custom distributed graph database, underpins over 90% of its online services, handling tens of thousands of instances and billions of daily requests, while embracing BASE principles and cloud‑native scalability.
Peng Liangyou describes how Bilibili’s large‑scale distributed KV storage adopts Netflix‑style chaos engineering—defining steady‑state hypotheses, replicating production environments, injecting CPU, memory, network and replica faults via automated “monkey” experiments, monitoring latency and durability with Prometheus/Grafana, and over 1.5 years preventing critical incidents while cutting testing costs and enabling incremental, standards‑based reliability improvements.
This article explains Redis’s overall architecture, covering its data model, supported value types, operation interfaces, memory versus disk storage choices, access pattern selection, I/O threading models, indexing mechanisms, specific CRUD logic, and persistence strategies, helping readers grasp both high‑level design and detailed implementation.
This article explains how Ctrip’s technical team identified the limitations of Redis on private cloud, evaluated numerous NoSQL/NewSQL options, selected Kvrocks, and engineered TRocks with functional, availability, and operational enhancements that deliver higher performance, lower cost, and robust data consistency for large‑scale KV workloads.
This article describes Ctrip's journey from using Redis to developing TRocks, a persistent key‑value database based on Kvrocks, covering background, challenges, research, architectural enhancements, performance and cost evaluations, and future development plans.
The MIT 6.824 course offers rich video resources, low entry difficulty, and well‑structured labs covering MapReduce, Raft, a simple KV store, and sharding, while the author shares personal challenges and tips for tackling the coursework.
Tendis, an open‑source distributed KV store developed by Tencent, combines full Redis protocol compatibility with RocksDB persistence, a decentralized gossip‑based cluster architecture, horizontal scalability to thousands of nodes, and cost‑effective hot‑cold data mixing, making it suitable for large‑scale, high‑throughput workloads.
Tendis is a Tencent‑developed distributed key‑value database that fully supports Redis protocols, leverages RocksDB for persistent storage, and uses a decentralized gossip‑based cluster architecture to deliver large‑capacity, low‑cost, and high‑availability solutions for warm‑cold data workloads.
The article details JD Retail's JIMKV distributed database, explaining its unified cache‑storage architecture, fault‑detection and elastic‑scaling mechanisms, hot‑cold data tiering, read/write amplification mitigation, real‑world product‑detail use case, and future plans for intelligent operations and OLAP support.
This article explains why Redis achieves exceptional performance by using pure in‑memory operations, a single‑threaded model, efficient data structures like SDS, skip‑list, ziplist, and various encoding and rehash optimizations that reduce memory allocation and CPU overhead.
QQ Album, storing over 2 trillion images across 300 PB, uses a distributed KV store, high‑performance compression (WebP, sharpP), regional “parks” with synchronized indexes, multi‑region disaster recovery, and a nationwide CDN to deliver low‑latency uploads and downloads for billions of daily user interactions.
This article examines multiple fault‑tolerant transaction designs—RSM‑based KV, RSM‑based transactions, shared‑storage approaches, high‑availability KV layers, and single‑node engine extensions—comparing their replication strategies, lock handling, and performance trade‑offs while raising open questions about ordering and consistency.
This article details the design, implementation, and performance tuning of a Java‑based key‑value storage engine built for a competition, covering the competition requirements, interface definitions, data layout, recovery logic, multi‑threaded read/write strategies, Direct I/O techniques, JVM tuning, and practical lessons learned.
This article presents Lest, a persistent KV‑store cache designed to overcome the limitations of traditional in‑memory caches by combining disk‑based persistence, lock‑free multithreading, custom protocols, and advanced load‑balancing, and it evaluates its performance on petabyte‑scale workloads.
The article summarizes the 58 Group technical salon where experts compared Meituan‑Dianping's Cellar and 58's WTable distributed KV storage systems, detailing their architectures, improvements, scalability, high‑availability mechanisms, and operational considerations, and concludes with a comparative analysis and preview of the next session.
This article details the design, architecture, and implementation of ZanKV—a high‑performance, distributed key‑value store that combines RocksDB storage, etcd‑Raft consensus, and a Redis‑compatible protocol, covering data partitioning, namespace isolation, expiration strategies, cross‑datacenter deployment, and performance tuning.
The 2018 Youzan PaaS meetup in Hangzhou featured deep dives into TiDB's distributed SQL architecture and performance gains in version 2.1, introduced the open‑source Zankv KV store built on Raft, and shared practical Elasticsearch search‑engine optimizations used at Youzan.
This article introduces Tendermint’s modular blockchain framework, explains its Core and ABCI components, provides step‑by‑step installation and execution of the built‑in key‑value store example, demonstrates transaction creation and querying, and walks through the Go source code for CheckTx, DeliverTx, and Commit functions.
The paper presents a generic feature‑production framework for Meituan’s food‑delivery ranking system that abstracts statistical feature generation, storage, retrieval, and online loading into configurable dimensions, metrics and operators, enabling developers to add new features with minimal code and dramatically speeding up machine‑learning model iteration.
This article presents a comprehensive technical overview of TiDB, a NewSQL distributed database, covering its architecture, SQL layer, KV engine, distributed transaction mechanisms, code implementation in Go, open‑source practices, and future roadmap.
This article examines common NoSQL key‑value stores such as Memcached and Redis, compares their strengths and limitations, and proposes a distributed architecture with routing, storage, management, and migration nodes that achieves high availability, automatic fault‑tolerance, load balancing, and elastic scaling.
