BestHub
Sign in
Tagged articles

distributed storage

275 articles · Page 2 of 3
DataFunTalk
DataFunTalk
Jun 5, 2022 · Big Data

JD Big Data Platform: Cross‑Region and Tiered Storage Architecture and Practices

This article presents JD's large‑scale big‑data platform, detailing its overall architecture, the challenges of cross‑region storage, the design of a unified cross‑domain data synchronization mechanism, and the implementation of tiered storage to improve performance, cost efficiency, and data reliability across multi‑datacenter clusters.

Big DataHDFSTiered Storage
0 likes · 15 min read
JD Big Data Platform: Cross‑Region and Tiered Storage Architecture and Practices
IT Architects Alliance
IT Architects Alliance
May 23, 2022 · Industry Insights

Why RDMA Is Replacing TCP/IP for AI and High‑Performance Storage

The article analyzes how the AI boom and high‑performance SSD storage demand sub‑microsecond latency, exposing TCP/IP’s inherent context‑switch and CPU overhead, and explains why RDMA’s kernel‑bypass, zero‑copy design and 1 µs latency make it the preferred network stack for modern data‑center workloads despite challenges in Ethernet deployment.

AI computingData Center NetworkNetwork Architecture
0 likes · 11 min read
Why RDMA Is Replacing TCP/IP for AI and High‑Performance Storage
Ctrip Technology
Ctrip Technology
May 19, 2022 · Backend Development

Design and Implementation of a Scalable Delay Message Service Using Apache BookKeeper

This article describes how the QMQ delay‑message service was refactored by separating business logic from storage, adopting Apache BookKeeper for high‑availability, zone‑aware disaster recovery, a configurable DNS resolver, a ZooKeeper‑based task coordinator, and a multi‑level sliding‑time‑bucket scheduler to achieve a stateless, elastic architecture.

BookKeeperElastic ArchitectureZookeeper
0 likes · 13 min read
Design and Implementation of a Scalable Delay Message Service Using Apache BookKeeper
Bilibili Tech
Bilibili Tech
May 17, 2022 · Cloud Computing

Bilibili Object Storage Service (BOSS) Design: Building a Large-Scale Distributed Storage System in 13 Days

In just 13 days Bilibili transformed a simple MySQL‑based S3 prototype into the BOSS distributed object storage system by separating metadata and data, adding an RPC abstraction layer, implementing two‑level sharding, switching to RocksDB, and deploying a three‑replica, multi‑zone high‑availability architecture.

RocksDBS3 protocolSharding
0 likes · 15 min read
Bilibili Object Storage Service (BOSS) Design: Building a Large-Scale Distributed Storage System in 13 Days
Architecture Digest
Architecture Digest
Mar 31, 2022 · Databases

Seven Key Aspects of Distributed Storage Systems

This article outlines the motivation and seven fundamental aspects of distributed storage—replication, storage engine, transactions, analytics, multi‑core processing, computation, and compilation—detailing their roles, challenges, and design considerations for building scalable, reliable, and high‑performance data systems.

Database ArchitectureTransactionsdistributed storage
0 likes · 14 min read
Seven Key Aspects of Distributed Storage Systems
Top Architect
Top Architect
Mar 28, 2022 · Databases

Key Aspects of Distributed Storage Systems: Replication, Engines, Transactions, Analytics, Multi‑Core, Computation, and Compilation

This article provides a comprehensive overview of distributed storage, covering seven core aspects such as replication, storage engines, transaction processing, analytical query execution, multi‑core scalability, computation models, and compilation techniques, while also highlighting practical challenges and design considerations for modern database systems.

CompilationStorage EngineTransaction
0 likes · 13 min read
Key Aspects of Distributed Storage Systems: Replication, Engines, Transactions, Analytics, Multi‑Core, Computation, and Compilation
IT Architects Alliance
IT Architects Alliance
Mar 21, 2022 · Databases

Seven Key Aspects of Distributed Storage Systems

This article outlines seven fundamental aspects of distributed storage—replication, storage engine, transactions, analysis, multi‑core processing, computation, and compilation—explaining their roles, challenges, and design considerations for building scalable, reliable, and high‑performance storage systems.

Database Architecturedistributed storageperformance
0 likes · 13 min read
Seven Key Aspects of Distributed Storage Systems
Bilibili Tech
Bilibili Tech
Mar 11, 2022 · Databases

Design and Architecture of Bilibili's High‑Performance Distributed KV Store

Bilibili’s high‑performance distributed KV store combines hash and range partitioning, Raft‑based multi‑replica consistency, and a Metaserver‑managed topology of pools, zones, nodes, tables, shards and replicas, offering features such as partition splitting, binlog streaming, multi‑active replication, bulk loading, KV‑storage separation, and automated load, leader and health balancing for reliable, scalable data services.

Failure RecoveryPartitioningRaft consensus
0 likes · 22 min read
Design and Architecture of Bilibili's High‑Performance Distributed KV Store
DataFunTalk
DataFunTalk
Jan 16, 2022 · Big Data

Time Series Database Capabilities and Application Scenarios in IoT, Smart Cities, and Edge Computing

This article explains the fundamentals of time‑series data, outlines the architecture and core technical advantages of Baidu Cloud's TSDB, and demonstrates how the database powers IoT, smart‑city, industrial, power‑grid, and autonomous‑driving use cases through multi‑level storage, distributed query optimization, and edge‑cloud integration.

Big DataCloud ComputingData Analytics
0 likes · 11 min read
Time Series Database Capabilities and Application Scenarios in IoT, Smart Cities, and Edge Computing
Baidu Geek Talk
Baidu Geek Talk
Sep 27, 2021 · Databases

Baidu's High-Performance KV Storage System Practice: Cloud-Native Optimization and Software-Hardware Co-Design

Baidu’s UNDB KV storage system, built with cloud‑native containerization and software‑hardware co‑design, separates key‑value processing to cut write amplification below 1.5×, leverages an OpenChannel SSD engine for over 30% performance gains, scales to 10,000+ servers and 100 PB, handles more than a trillion daily requests across search and feed, and cuts operating costs by nearly half.

KV storageNoSQLOpenChannel SSD
0 likes · 19 min read
Baidu's High-Performance KV Storage System Practice: Cloud-Native Optimization and Software-Hardware Co-Design
DataFunTalk
DataFunTalk
Sep 22, 2021 · Big Data

Distributed Storage and Application Solutions for Massive Spatiotemporal Data

This article explains the rapid growth of global spatiotemporal data, the limitations of traditional GIS, and presents SuperMap's distributed storage architecture, unified data access APIs, dynamic rendering techniques, and geographic processing modeling with real‑world case studies to address performance and scalability challenges.

GISbig data analyticsdistributed storage
0 likes · 16 min read
Distributed Storage and Application Solutions for Massive Spatiotemporal Data
Xianyu Technology
Xianyu Technology
Sep 9, 2021 · Backend Development

Design and Scaling of Xianyu Messaging System (1.0 → 4.0)

The Xianyu instant‑messaging system evolved from a 2014 MVP to a robust, multi‑version architecture—adding distributed storage, hybrid sync, ACK‑based delivery, and feature‑driven monitoring—to handle billions of messages, improve stability, and boost user satisfaction, cutting technical complaints by half.

Real‑time communicationdistributed storagemessaging
0 likes · 10 min read
Design and Scaling of Xianyu Messaging System (1.0 → 4.0)
Architects' Tech Alliance
Architects' Tech Alliance
Aug 6, 2021 · Big Data

Performance Optimization Techniques for the Ceph Distributed Storage System

This article reviews Ceph's architecture, enumerates common benchmarking tools, analyzes its advantages and challenges, and presents a comprehensive set of performance‑optimization methods covering storage‑engine tuning, network communication, data placement, configuration parameters, hardware‑specific adaptations, and future research directions.

CephNVMeRDMA
0 likes · 20 min read
Performance Optimization Techniques for the Ceph Distributed Storage System
JD Tech
JD Tech
Jul 30, 2021 · Databases

Practical Use of HBase in a Logistics HR Data Preprocessing Platform

This article details how the logistics HR data preprocessing platform processes around 20 million daily records by adopting HBase for high‑performance, scalable, column‑oriented storage, covering its architecture, read/write mechanisms, best practices, and performance considerations.

Big DataHBaseNoSQL
0 likes · 10 min read
Practical Use of HBase in a Logistics HR Data Preprocessing Platform
vivo Internet Technology
vivo Internet Technology
Jul 28, 2021 · Industry Insights

How to Quantify Data Reliability in Distributed Storage Systems

This article analyzes the quantitative model for data reliability in distributed storage, covering factors such as disk count, replication factor, recovery time, annualized failure rate, and copyset configuration, and derives formulas to estimate yearly data loss probability for both replica and erasure‑coding schemes.

AFRData Reliabilitycopyset
0 likes · 16 min read
How to Quantify Data Reliability in Distributed Storage Systems
Cloud Native Technology Community
Cloud Native Technology Community
Jul 2, 2021 · Cloud Native

Typical Cloud‑Native Storage Solutions for Stateful Applications and Their Performance Comparison

This article reviews typical cloud‑native storage options for stateful workloads, analyzes their advantages and drawbacks, discusses the specific needs of OLTP and OLAP databases, outlines challenges in multi‑cloud environments, and presents performance test results of four popular cloud‑native storage systems.

Performance Testingcloud-nativedistributed storage
0 likes · 13 min read
Typical Cloud‑Native Storage Solutions for Stateful Applications and Their Performance Comparison
Dada Group Technology
Dada Group Technology
Jul 2, 2021 · Backend Development

Design and Implementation of a High‑Availability Coupon Platform with Distributed Storage (JimDB)

This article describes the architecture and optimization of JD.com’s coupon platform, covering the JimDB distributed in‑memory database for core storage, a massive distributed task system for product coupons, high‑availability strategies for store coupons, and the overall middle‑platform design that ensures scalability, low latency, and data consistency across millions of daily transactions.

High Availabilitybackendcaching
0 likes · 8 min read
Design and Implementation of a High‑Availability Coupon Platform with Distributed Storage (JimDB)
Architects' Tech Alliance
Architects' Tech Alliance
Jun 30, 2021 · Cloud Computing

Ceph Distributed Storage System: Architecture, CRUSH Algorithm, and Backend Evolution

This article provides a comprehensive overview of Ceph, covering its origins, cluster architecture, CRUSH data placement algorithm, unified access interfaces, the transition from FileStore to BlueStore, and emerging storage back‑ends such as SeaStore and PFStore, highlighting performance characteristics and design trade‑offs.

BlueStoreCRUSHCeph
0 likes · 12 min read
Ceph Distributed Storage System: Architecture, CRUSH Algorithm, and Backend Evolution
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 24, 2021 · Big Data

Comprehensive Overview of HBase Architecture, Design, and Operations

This article provides an in‑depth technical overview of HBase, covering its Bigtable origins, distributed column‑store design, core components such as ZooKeeper, HMaster and RegionServer, data flow, storage formats, row‑key design, bulk loading, SQL integration, indexing, coprocessors, and performance tuning for big‑data environments.

Columnar DatabaseHBaseHDFS
0 likes · 30 min read
Comprehensive Overview of HBase Architecture, Design, and Operations
Tencent Cloud Developer
Tencent Cloud Developer
May 14, 2021 · Cloud Native

Exploration and Practice of Tencent Cloud Native Database TDSQL-C Architecture

Tencent’s cloud‑native database TDSQL‑C separates compute and storage, offering petabyte‑scale network‑attached storage, automatic scaling, million‑level QPS, MySQL/PostgreSQL compatibility, serverless per‑use billing, two‑level caching, instant snapshot‑based backup and restore, and a roadmap toward ultra‑minimal operations, multi‑region DR and column‑store analytics.

Cloud DatabaseDatabase ArchitectureTDSQL-C
0 likes · 16 min read
Exploration and Practice of Tencent Cloud Native Database TDSQL-C Architecture
Volcano Engine Developer Services
Volcano Engine Developer Services
May 13, 2021 · Databases

Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database

The article offers a comprehensive technical deep‑dive into ByteDance’s home‑grown distributed graph database and graph‑processing engine, ByteGraph, covering its directed‑property graph model, Gremlin query support, multi‑layer architecture, storage strategies for massive data, and real‑world graph‑computing practices.

Big DataByteGraphGraph Database
0 likes · 28 min read
Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database
DataFunTalk
DataFunTalk
Apr 20, 2021 · Databases

Meituan's Graph Database Selection and Platform Construction

This article presents Meituan's comprehensive evaluation of open‑source graph databases, the rationale for selecting NebulaGraph, and the design of a high‑availability, high‑throughput graph database platform that supports multi‑hop queries, massive data ingestion, real‑time synchronization, and visualization for various business scenarios.

Graph DatabaseHigh AvailabilityMeituan
0 likes · 21 min read
Meituan's Graph Database Selection and Platform Construction
Volcano Engine Developer Services
Volcano Engine Developer Services
Mar 23, 2021 · Cloud Native

How Douyin Handled 70B Red Packet Interactions in 27 Days with Cloud‑Native Magic

In just 27 days, Douyin and Volcano Engine's cloud‑native team built a Kubernetes‑based, elastically scalable infrastructure that supported 703 billion red‑packet interactions and over a trillion live‑stream views during the 2021 Spring Festival Gala, ensuring zero downtime and seamless user experience.

Cloud NativeEdge computingElastic Scaling
0 likes · 12 min read
How Douyin Handled 70B Red Packet Interactions in 27 Days with Cloud‑Native Magic
Liangxu Linux
Liangxu Linux
Mar 18, 2021 · Operations

Deploying GlusterFS Distributed Replicated Volumes with NFS‑Ganesha

This guide explains the architecture of GlusterFS, compares its volume types, walks through setting up a multi‑node cluster, configuring NFS‑Ganesha for NFSv4 access, and shows how to mount the storage from clients using both GlusterFS and NFS protocols.

GlusterFSLinuxNFS-Ganesha
0 likes · 12 min read
Deploying GlusterFS Distributed Replicated Volumes with NFS‑Ganesha
Tencent Architect
Tencent Architect
Feb 23, 2021 · Artificial Intelligence

Analysis and Optimization of CephFS I/O Performance for AI Training on the Xingchen Compute Platform

This article investigates why AI training tasks on Tencent's Xingchen compute platform experience severe I/O slowdown when using CephFS, analyzes the underlying Ceph‑FUSE and MDS mechanisms, and proposes metadata‑caching and file‑caching optimizations that can accelerate training speed by three to four times.

AI trainingCeph-FUSECephFS
0 likes · 21 min read
Analysis and Optimization of CephFS I/O Performance for AI Training on the Xingchen Compute Platform
Architects' Tech Alliance
Architects' Tech Alliance
Jan 25, 2021 · Fundamentals

Ceph Storage Architecture Overview and Detailed Technical Features

This article provides a comprehensive technical overview of Red Hat Ceph, covering its distributed object storage design, cluster architecture, storage pools, authentication, placement groups, CRUSH algorithm, I/O operations, replication, erasure coding, internal management tasks, high availability, client interfaces, data striping, and encryption mechanisms.

CRUSHCephData Striping
0 likes · 42 min read
Ceph Storage Architecture Overview and Detailed Technical Features
IT Xianyu
IT Xianyu
Jan 18, 2021 · Backend Development

FastDFS Overview, Docker Installation, and Spring Boot Integration Guide

This article introduces FastDFS as a lightweight distributed file system, explains its tracker and storage components, provides step‑by‑step Docker installation commands, demonstrates how to integrate FastDFS with Spring Boot using Maven dependencies and configuration, and includes sample code for uploading, downloading, and deleting files.

FastDFSbackend-developmentdistributed storage
0 likes · 7 min read
FastDFS Overview, Docker Installation, and Spring Boot Integration Guide
Java Captain
Java Captain
Jan 2, 2021 · Backend Development

Design and Implementation of a Lightweight Open‑Source Netdisk System

This article introduces a lightweight, open‑source netdisk system built with a front‑end Vue.js interface and a back‑end micro‑service architecture using SpringBoot, Dubbo, Zookeeper, Redis, Solr, and FastDFS, detailing its deployment, technology stack, and core features for file management and integration.

SpringBootVue.jsdistributed storage
0 likes · 5 min read
Design and Implementation of a Lightweight Open‑Source Netdisk System
dbaplus Community
dbaplus Community
Nov 29, 2020 · Databases

Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database

ByteGraph is ByteDance's home‑grown distributed graph storage and computation platform that supports massive social‑graph workloads with directed‑property models, Gremlin queries, multi‑layer architecture, adaptive KV storage, hot‑spot handling, indexing, and a roadmap toward cloud‑native, HTAP‑capable graph processing.

ByteGraphGraph ComputationGraph Database
0 likes · 35 min read
Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database
Architects' Tech Alliance
Architects' Tech Alliance
Nov 9, 2020 · Cloud Computing

Ceph Storage Architecture: Overview, Cluster Design, Client Interfaces, and Encryption

This article provides a comprehensive technical overview of Red Hat Ceph, covering its distributed storage architecture, cluster components, storage pool types, authentication, placement algorithms, I/O paths, replication and erasure‑coding strategies, internal management operations, high‑availability mechanisms, client libraries, data striping, and encryption details.

CRUSHCephCloud Storage
0 likes · 39 min read
Ceph Storage Architecture: Overview, Cluster Design, Client Interfaces, and Encryption
JD Tech Talk
JD Tech Talk
Sep 21, 2020 · Operations

Investigation and Optimization of Ceph Slow‑Request Issues Related to Scrub in the Luminous Release

This article analyzes a frequent slow‑request problem in a Ceph cluster caused by heavy write‑induced RocksDB compaction combined with deep‑scrub activity, explains the underlying scrub mechanisms, and presents parameter‑tuning and custom scheduling strategies to mitigate performance impact while preserving data consistency.

CephScrubdistributed storage
0 likes · 13 min read
Investigation and Optimization of Ceph Slow‑Request Issues Related to Scrub in the Luminous Release
JD Cloud Developers
JD Cloud Developers
Sep 15, 2020 · Databases

How JD’s HoraeDB Tackles Massive Time‑Series Data at Scale

This article introduces JD Cloud’s self‑built time‑series database HoraeDB, explaining its core concepts, typical use cases, architectural layers, high‑performance features, down‑sampling strategies, compression techniques, and stability measures for handling massive, 24‑hour monitoring data at scale.

DownsamplingMonitoringcompression
0 likes · 18 min read
How JD’s HoraeDB Tackles Massive Time‑Series Data at Scale
Java Backend Technology
Java Backend Technology
Sep 10, 2020 · Backend Development

Designing a Billion-Scale Feed Stream System: Architecture & Best Practices

This article explains how to design a high‑performance, billion‑user feed stream system, covering product definition, data modeling, storage choices, synchronization modes, metadata handling, commenting, likes, search, sorting, deletion, updates, and practical architecture examples for different feed‑type applications.

backenddistributed storagefeed stream
0 likes · 22 min read
Designing a Billion-Scale Feed Stream System: Architecture & Best Practices
Didi Tech
Didi Tech
Aug 28, 2020 · Operations

Ceph Performance Optimization: Lock-Related Issues and Solutions

The article details how Didi’s large‑scale Ceph deployment suffered from high tail latency due to long‑held and coarse‑grained locks, and describes a series of fixes—including asynchronous read threads, fine‑grained object caches, per‑thread lock‑free logging, and lock‑free filestore apply—that cut latency by up to 90 % and more than doubled read throughput.

BlueStoreCephFileStore
0 likes · 12 min read
Ceph Performance Optimization: Lock-Related Issues and Solutions
Full-Stack DevOps & Kubernetes
Full-Stack DevOps & Kubernetes
Aug 3, 2020 · Operations

How to Deploy a Ceph Distributed Storage Cluster from Scratch

This guide walks through installing and configuring a Ceph distributed storage cluster, covering node preparation, hostname and hosts setup, password‑less SSH, required packages, firewall handling, time sync, ceph‑deploy installation, monitor creation, OSD configuration, pool and RBD creation, verification, and common troubleshooting steps.

CephDevOpsLinux
0 likes · 16 min read
How to Deploy a Ceph Distributed Storage Cluster from Scratch
Senior Brother's Insights
Senior Brother's Insights
Jul 30, 2020 · Backend Development

Integrate FastDFS with Spring Boot for File Upload, Download, and Management

This tutorial shows how to integrate the FastDFS distributed file system into a Spring Boot application, covering Maven dependencies, configuration files, utility classes, core client initialization, and controller code for uploading, downloading, retrieving, and deleting files, with complete code examples and a GitHub source link.

FastDFSSpring Bootbackend
0 likes · 9 min read
Integrate FastDFS with Spring Boot for File Upload, Download, and Management
MaGe Linux Operations
MaGe Linux Operations
Jul 22, 2020 · Cloud Native

Why Ceph Is the Backbone of Modern Cloud‑Native Storage Solutions

This article traces Ceph’s evolution from its academic origins to widespread adoption in cloud‑native environments, explains the four stages of storage development, details Ceph’s unified object, block, and file interfaces, and describes its architecture—including monitors, OSDs, CRUSH algorithm, placement groups, and high‑availability features.

CephCloud Nativeblock storage
0 likes · 17 min read
Why Ceph Is the Backbone of Modern Cloud‑Native Storage Solutions
Cloud Native Technology Community
Cloud Native Technology Community
Jul 7, 2020 · Cloud Native

Taming etcd Instability: Lessons from Managing Million‑Node Kubernetes Clusters

This article details how Tencent Cloud’s TKE team identified, analyzed, reproduced, and resolved multiple etcd stability and performance issues—including data inconsistency, memory leaks, mvcc deadlocks, and WAL crashes—while sharing the lessons learned and the optimizations applied to support million‑node Kubernetes deployments.

Kubernetescloud-nativedistributed storage
0 likes · 29 min read
Taming etcd Instability: Lessons from Managing Million‑Node Kubernetes Clusters
dbaplus Community
dbaplus Community
Apr 15, 2020 · Big Data

How Ctrip Scaled Hadoop Across Data Centers: Architecture and Lessons

This article details Ctrip's Hadoop evolution, the challenges of expanding across multiple data centers, the evaluation of multi‑cluster versus single‑cluster designs, and the concrete architectural changes, migration tools, bandwidth monitoring, and future plans that enabled a stable cross‑datacenter big‑data platform.

Big DataCross-DataCenterHDFS
0 likes · 19 min read
How Ctrip Scaled Hadoop Across Data Centers: Architecture and Lessons
vivo Internet Technology
vivo Internet Technology
Mar 18, 2020 · Databases

Vivo Feature Storage Practice: Architecture, Design, and Future Directions Using Nebula Graph

Vivo’s feature‑storage platform, built on Nebula Graph’s Raft‑based, storage‑compute‑separated architecture and exposed via Redis‑compatible proxies, meets massive, low‑latency AI data demands while offering strong consistency, horizontal scalability, backup, active‑active replication, and a roadmap toward general‑purpose KV, cloud‑native integration, and advanced storage engines.

Feature StoreKVNebula Graph
0 likes · 21 min read
Vivo Feature Storage Practice: Architecture, Design, and Future Directions Using Nebula Graph
DataFunTalk
DataFunTalk
Feb 26, 2020 · Databases

ByteGraph: ByteDance’s Distributed Graph Database and Graph Computing System – Architecture, Data Model, and Practices

This article presents an in‑depth technical overview of ByteGraph, ByteDance’s self‑built distributed graph database and its accompanying graph‑computing engine, covering graph data characteristics, the directed‑property graph model, API design, three‑tier system architecture, storage strategies using KV stores and B‑Trees, hotspot handling, indexing, and future research directions.

B+TreeByteGraphGraph Database
0 likes · 33 min read
ByteGraph: ByteDance’s Distributed Graph Database and Graph Computing System – Architecture, Data Model, and Practices
Big Data Technology Architecture
Big Data Technology Architecture
Feb 15, 2020 · Databases

An Introduction to HBase: Architecture, Data Model, Storage Engine, Indexing, Features, and Use Cases

This article provides a comprehensive overview of HBase, covering its LSM‑Tree based storage engine, key‑value data model, column‑family storage design, indexing mechanisms, major advantages and drawbacks, and typical scenarios where HBase excels for massive, high‑throughput data workloads.

Data ModelingHBaseIndexing
0 likes · 8 min read
An Introduction to HBase: Architecture, Data Model, Storage Engine, Indexing, Features, and Use Cases
Big Data Technology Architecture
Big Data Technology Architecture
Feb 11, 2020 · Databases

JDHBase Multi‑Active Architecture and Asynchronous Replication Practices

This article describes JDHBase’s large‑scale KV storage architecture, its HBase‑based asynchronous replication mechanism, multi‑active cluster design, client‑side routing via Fox Manager, automatic failover strategies, dynamic replication tuning, and serial replication techniques to ensure data consistency across geographically distributed data centers.

Dynamic TuningFailoverHBase Replication
0 likes · 12 min read
JDHBase Multi‑Active Architecture and Asynchronous Replication Practices
dbaplus Community
dbaplus Community
Jan 15, 2020 · Databases

How Didi Built Fusion-NewSQL: A High‑Throughput, Low‑Latency NewSQL on Distributed KV

Fusion-NewSQL is Didi’s internally‑developed NewSQL system built atop the Fusion distributed KV store, offering MySQL compatibility, high throughput, low latency, flexible schema changes, secondary indexes, and integration with ElasticSearch and Hive, with detailed architecture, data flow, and future roadmap.

MySQL CompatibilityNewSQLSchema Evolution
0 likes · 16 min read
How Didi Built Fusion-NewSQL: A High‑Throughput, Low‑Latency NewSQL on Distributed KV
JD Retail Technology
JD Retail Technology
Jan 6, 2020 · Backend Development

JDHBase Multi‑Active Architecture and Replication Practices

This article describes JDHBase’s large‑scale KV storage deployment, its HBase‑based asynchronous replication mechanism, the multi‑active architecture with active‑standby clusters, client interaction via Fox Manager, automatic failover strategies, dynamic replication tuning, and serial replication techniques to ensure data consistency across data centers.

Cluster ManagementHBaseHigh Availability
0 likes · 13 min read
JDHBase Multi‑Active Architecture and Replication Practices
Meitu Technology
Meitu Technology
Dec 12, 2019 · Databases

Migrating Meitu Push Service Storage to Titan: Architecture, Challenges, and Solutions

Meitu migrated its high‑traffic push‑service storage from fragmented Redis nodes to the TiKV‑based Titan system, using a dual‑write rollout, batch‑operation tuning, and TiKV configuration tweaks, achieving required QPS, cutting costs by about 60%, and delivering stable, maintainable service after six months.

Database MigrationPush ServiceTiKV
0 likes · 13 min read
Migrating Meitu Push Service Storage to Titan: Architecture, Challenges, and Solutions
UCloud Tech
UCloud Tech
Nov 19, 2019 · Cloud Computing

Why Object Storage Is the Future of Scalable Cloud Data Management

This lesson introduces object storage, explaining its concept, technical principles, market scale, advantages over traditional DAS/NAS/SAN solutions, and key architectural components, while highlighting scalability, cost efficiency, high availability, and design considerations for modern cloud storage systems.

data availabilitydistributed storageobject storage
0 likes · 7 min read
Why Object Storage Is the Future of Scalable Cloud Data Management
58 Tech
58 Tech
Nov 8, 2019 · Databases

Design and Optimization Practices of 58.com Distributed KV Storage System (WTable)

The article details the three‑stage evolution of 58.com’s distributed key‑value storage system WTable, covering its initial architecture, performance and operational optimizations, and the introduction of NewSQL‑style strong consistency and Raft‑based replication to improve scalability and reliability.

NewSQLdistributed storagekey-value store
0 likes · 9 min read
Design and Optimization Practices of 58.com Distributed KV Storage System (WTable)
JD Retail Technology
JD Retail Technology
Oct 14, 2019 · Databases

Overview of JDNoSQL Platform and Its Real-Time Advertising Use Cases

The article introduces JDNoSQL, a distributed column‑oriented key‑value store built on HDFS, outlines its core features, describes various business scenarios including real‑time ad computation, details the system architecture with Kafka and Flink, and presents table designs for ad impression and click statistics.

Big DataFlinkKafka
0 likes · 13 min read
Overview of JDNoSQL Platform and Its Real-Time Advertising Use Cases
Didi Tech
Didi Tech
Sep 20, 2019 · Big Data

FastLoad: A One-Click DTS Platform for Online Data Migration

FastLoad, Didi’s one‑click DTS platform, accelerates migration of terabyte‑scale offline data into its Fusion storage by using RocksDB’s IngestFile to import SST files directly, cutting a 1 TB load from twelve hours to one, while supporting thousands of daily tasks with 99.99% stability.

DTS platformData MigrationRocksDB
0 likes · 8 min read
FastLoad: A One-Click DTS Platform for Online Data Migration
Alibaba Cloud Native
Alibaba Cloud Native
Sep 19, 2019 · Databases

How etcd’s Fully Concurrent Read Boosts Kubernetes Performance

This article reviews the evolution of etcd’s read‑write mechanisms, explains the “Fully Concurrent Read” feature introduced in etcd 3.4, and presents experimental results showing how it dramatically reduces expensive read latency and improves overall throughput in Kubernetes clusters.

BoltDBFully Concurrent ReadKubernetes
0 likes · 14 min read
How etcd’s Fully Concurrent Read Boosts Kubernetes Performance
Programmer DD
Programmer DD
Sep 10, 2019 · Databases

Understanding Nebula Graph: Data Model and Architecture Explained

This article introduces Nebula Graph, an open‑source distributed graph database, detailing its directed property graph model, vertex and edge schemas, graph partitioning, storage, metadata, query engine, and client APIs, highlighting its strong schema design, high‑availability architecture, and scalability for trillion‑scale graphs.

Graph DatabaseNebula Graphdata-model
0 likes · 11 min read
Understanding Nebula Graph: Data Model and Architecture Explained
DevOps Cloud Academy
DevOps Cloud Academy
Aug 11, 2019 · Operations

Recovering MFS Master Node Using Metalogger After Node Failure

When the mfsmaster node in a six‑node MFS distributed storage cluster becomes unresponsive, the issue can be resolved by using metalogger on another node to restore the metadata file, copy it to the master directory, adjust permissions, and restart the mfsmaster service.

LinuxMFSNode Recovery
0 likes · 1 min read
Recovering MFS Master Node Using Metalogger After Node Failure
Tencent Cloud Developer
Tencent Cloud Developer
Aug 1, 2019 · Databases

FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges

FeatureKV, WeChat’s high‑performance key‑value store, handles one‑billion queries per second and ingests a billion keys per hour by separating write‑only DataSvr from read‑only KVSvr, supporting in‑memory, indexed, and block‑indexed tables, scaling horizontally, guaranteeing eventual consistency with versioned reads, and delivering up to 11 billion reads per second with sub‑15 ms latency.

Batch ProcessingFeatureKVKey-Value
0 likes · 22 min read
FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges
58 Tech
58 Tech
Jul 25, 2019 · Databases

Design and Evolution of WTable’s Scaling Process Using RocksDB

This article explains how the WTable distributed key‑value store leverages RocksDB’s LSM‑tree architecture and slot‑based data distribution to redesign its scaling workflow, separating full and incremental data migration to reduce compaction overhead and achieve high‑speed, low‑impact cluster expansion.

Data MigrationRocksDBScaling
0 likes · 8 min read
Design and Evolution of WTable’s Scaling Process Using RocksDB
System Architect Go
System Architect Go
Jul 19, 2019 · Big Data

Introduction to HBase: Architecture, Data Model, and Operations

This article provides a comprehensive overview of HBase, covering its distributed column‑oriented architecture, data model components, storage mechanisms, read/write processes, WAL lifecycle, MemStore flushing, region splitting and merging, and failure recovery within the Hadoop ecosystem.

Big DataHBaseHadoop
0 likes · 20 min read
Introduction to HBase: Architecture, Data Model, and Operations
dbaplus Community
dbaplus Community
Jul 18, 2019 · Databases

How JD.com Scales HBase to 90PB: Architecture, Optimizations, and Lessons

This article examines JD.com's massive HBase deployment, detailing its evolution from early adoption to a 90PB, 7,000‑node cluster, the platform's architecture, multi‑active disaster recovery, multi‑tenant isolation, and the integration of Phoenix for SQL‑based access, offering practical insights for large‑scale distributed storage.

Big DataDatabase ArchitectureHBase
0 likes · 15 min read
How JD.com Scales HBase to 90PB: Architecture, Optimizations, and Lessons
DataFunTalk
DataFunTalk
Jul 15, 2019 · Big Data

Key Infrastructure Considerations for Autonomous Driving: Storage, Computing, and Services

The article reviews the essential infrastructure for autonomous driving, covering massive sensor data storage strategies, the role of metadata, offline and real‑time computing platforms, basic micro‑service components, and various business scenarios, highlighting why robust big‑data handling is critical.

Big DataReal-Time Computingautonomous driving
0 likes · 14 min read
Key Infrastructure Considerations for Autonomous Driving: Storage, Computing, and Services
Architects' Tech Alliance
Architects' Tech Alliance
Jul 14, 2019 · Fundamentals

Comparison of Ceph and GlusterFS Storage Architectures

This article provides an in‑depth comparison of Red Hat’s Ceph and GlusterFS open‑source storage solutions, detailing their underlying architectures—including RADOS, CephFS, RBD, and Gluster’s brick‑based design—while discussing performance, scalability, high‑availability, hardware requirements, and suitability for various cloud environments.

CephCloud StorageComparison
0 likes · 10 min read
Comparison of Ceph and GlusterFS Storage Architectures
360 Quality & Efficiency
360 Quality & Efficiency
Jun 11, 2019 · Backend Development

NebulasFs: A Distributed High‑Availability Small‑File Storage System Developed by 360 Infrastructure Team

NebulasFs is a self‑developed distributed, highly available, and persistent storage system designed to efficiently store billions of small files, offering simple RESTful APIs, automatic request routing, multi‑tenant isolation, customizable replication, automated scaling, rebalancing, and fault‑tolerant replica recovery for large‑scale unstructured data workloads.

High AvailabilityNebulasFsSmall Files
0 likes · 8 min read
NebulasFs: A Distributed High‑Availability Small‑File Storage System Developed by 360 Infrastructure Team
360 Tech Engineering
360 Tech Engineering
Jun 11, 2019 · Databases

NebulasFs: A Distributed High‑Availability Small‑File Storage System

NebulasFs is a self‑developed distributed, highly available and durable storage system designed to efficiently store billions of small files by using a master‑datanode architecture, multi‑tenant isolation, customizable replication, automatic scaling, and automated replica repair, addressing the challenges of massive unstructured data generated by modern applications.

Cloud NativeHigh AvailabilityNebulasFs
0 likes · 7 min read
NebulasFs: A Distributed High‑Availability Small‑File Storage System
21CTO
21CTO
Jun 8, 2019 · Cloud Computing

Unlocking Ceph: Deep Dive into Architecture, IO Flow, Heartbeat, CRUSH & QOS

This article provides a comprehensive overview of Ceph, covering its architecture and use cases, detailed IO processes, heartbeat mechanisms, communication framework, the CRUSH data placement algorithm, and customizable RBD QoS policies, illustrated with diagrams and code snippets to help readers understand and implement Ceph in cloud environments.

CRUSHCephCloud Computing
0 likes · 27 min read
Unlocking Ceph: Deep Dive into Architecture, IO Flow, Heartbeat, CRUSH & QOS
58 Tech
58 Tech
May 21, 2019 · Backend Development

Design and Architecture of WOS: 58 Group's Self‑Developed Object Storage System

This article presents the architecture and key design features of WOS, the 58 Group’s self‑developed object storage system, covering its overall framework, proxy, store, directory, detector modules, fast‑upload “秒传” mechanism, and erasure‑coding solution for efficient, scalable, and reliable unstructured data storage.

Backend ArchitectureWOSdistributed storage
0 likes · 12 min read
Design and Architecture of WOS: 58 Group's Self‑Developed Object Storage System
Architects' Tech Alliance
Architects' Tech Alliance
May 15, 2019 · Fundamentals

Red Hat Ceph Storage Architecture Guide – Overview and Core Concepts

This article provides a comprehensive overview of Red Hat Ceph's distributed object storage architecture, covering storage pools, CRUSH placement, authentication, I/O workflows, internal operations, client interfaces, data striping, erasure coding, high availability, and encryption mechanisms for secure, scalable deployments.

CRUSHCephdistributed storage
0 likes · 40 min read
Red Hat Ceph Storage Architecture Guide – Overview and Core Concepts
Big Data Technology Architecture
Big Data Technology Architecture
May 11, 2019 · Cloud Computing

Open Source Distributed Object Storage Solutions Overview

This article introduces the concepts of block, file, and object storage and reviews several open‑source distributed object storage solutions—including Swift, Ceph, MinIO, HBase MOB, and Hadoop Ozone—highlighting their architectures, features, and typical use cases for large‑scale data handling.

CephHBase MOBMinIO
0 likes · 8 min read
Open Source Distributed Object Storage Solutions Overview
Alibaba Cloud Developer
Alibaba Cloud Developer
May 6, 2019 · Backend Development

How Modern IM Systems Achieve Multi-Device Sync, Roaming, and Real-Time Search

Modern instant messaging platforms have evolved from simple CS/P2P designs to complex distributed architectures that store messages before synchronizing, enabling features like multi-device sync, message roaming, and online search, with Alibaba Cloud Tablestore’s Timeline model providing scalable storage, indexing, and flexible synchronization strategies.

Cloud Computingdistributed storagemessage synchronization
0 likes · 16 min read
How Modern IM Systems Achieve Multi-Device Sync, Roaming, and Real-Time Search
JD Retail Technology
JD Retail Technology
Apr 10, 2019 · Databases

HBase at JD.com: Architecture, Use Cases, and Evolution

This article explains how JD.com leverages the open‑source HBase database for massive, low‑latency data storage across various business lines, detailing its architecture, multi‑tenant isolation, disaster‑recovery mechanisms, and integration with Phoenix SQL for OLTP workloads.

Big DataDatabase ArchitectureHBase
0 likes · 13 min read
HBase at JD.com: Architecture, Use Cases, and Evolution
Tencent Cloud Developer
Tencent Cloud Developer
Mar 29, 2019 · Databases

Design of High‑Availability System and Fast Recovery in Tencent CynosDB

Tencent CynosDB achieves high availability and rapid recovery through an external HA service that combines a co‑located monitoring agent, a ZooKeeper‑backed scheduler for fault detection, decision making, and automated switch/rejoin/rebuild actions, and a VDL‑driven distributed storage recovery mechanism that prevents split‑brain scenarios.

Agent‑SchedulerCynosDBDatabase Architecture
0 likes · 19 min read
Design of High‑Availability System and Fast Recovery in Tencent CynosDB
Tencent Cloud Developer
Tencent Cloud Developer
Mar 26, 2019 · Databases

Technical Overview of Tencent Cloud CynosDB Distributed Storage and Compute‑Storage Separation Architecture

The Beijing technical exchange detailed Tencent Cloud CynosDB’s compute-storage separation architecture, explaining how CynosStore’s block and file-system layers asynchronously persist write-ahead logs via Raft, provide multi-transaction atomicity, continuous snapshots, and rapid recovery, while benchmarks demonstrate high log-throughput and the Q&A covered tenancy, failover, and scaling.

CynosDBDatabase ArchitectureNewSQL
0 likes · 14 min read
Technical Overview of Tencent Cloud CynosDB Distributed Storage and Compute‑Storage Separation Architecture
58 Tech
58 Tech
Feb 26, 2019 · Backend Development

Architecture Evolution of the WeChat‑like Instant Messaging Service at 58 Group

The article details the step‑by‑step evolution of the WeChat‑like instant messaging platform at 58 Group, describing how the system transitioned from a simple PHP short‑connection architecture to a layered micro‑service architecture using Java‑based RPC, Go‑based short‑connection services, distributed storage, message queues, and clear separation of storage, logic, and interface layers.

Backend ArchitectureGoInstant Messaging
0 likes · 12 min read
Architecture Evolution of the WeChat‑like Instant Messaging Service at 58 Group
JD Tech
JD Tech
Feb 18, 2019 · Big Data

Understanding HBase: Advantages, Use Cases, Data Model, and Architecture

This article explains HBase as a high‑performance, column‑oriented distributed storage system, outlines its advantages and limitations, presents real‑world scenarios such as seller operation logs and message logs, and details its data structures, architecture components, and design considerations for big‑data applications.

Data ModelingHBaseNoSQL
0 likes · 9 min read
Understanding HBase: Advantages, Use Cases, Data Model, and Architecture