Sep 29, 2022 · Databases

Design and Challenges of TafDB: A Scalable Metadata Storage Engine for Cloud Data Lakes

TafDB, Baidu’s Spanner‑like distributed transaction database built on RocksDB and Multi‑Raft, provides a virtually unlimited metadata layer for cloud data lakes by unifying hierarchical and flat namespaces, minimizing cross‑shard transaction overhead, handling garbage collection, and employing a distributed clock, thus delivering trillion‑scale metadata capacity and tens of millions of QPS with low latency.

Cloud StorageMetadata StorageNamespace

0 likes · 21 min read

Design and Challenges of TafDB: A Scalable Metadata Storage Engine for Cloud Data Lakes

Hacker Afternoon Tea

Jan 27, 2022 · Databases

Undermoon: Self‑Managing Redis Cluster Built on the Redis Cluster Protocol

Undermoon is an open‑source system that implements a self‑managing Redis cluster using the Redis Cluster protocol, offering horizontal scalability, high availability, HTTP‑API management, automatic master‑replica failover, and rapid expansion, with its architecture detailed across metadata storage, coordinator, storage cluster, and chunk components.

ChunkCoordinatorMetadata Storage

0 likes · 4 min read

Undermoon: Self‑Managing Redis Cluster Built on the Redis Cluster Protocol