Tagged articles
265 articles
Page 2 of 3
Baidu Geek Talk
Baidu Geek Talk
Sep 27, 2021 · Databases

Baidu's High-Performance KV Storage System Practice: Cloud-Native Optimization and Software-Hardware Co-Design

Baidu’s UNDB KV storage system, built with cloud‑native containerization and software‑hardware co‑design, separates key‑value processing to cut write amplification below 1.5×, leverages an OpenChannel SSD engine for over 30% performance gains, scales to 10,000+ servers and 100 PB, handles more than a trillion daily requests across search and feed, and cuts operating costs by nearly half.

KV storageNoSQLOpenChannel SSD
0 likes · 19 min read
Baidu's High-Performance KV Storage System Practice: Cloud-Native Optimization and Software-Hardware Co-Design
DataFunTalk
DataFunTalk
Sep 22, 2021 · Big Data

Distributed Storage and Application Solutions for Massive Spatiotemporal Data

This article explains the rapid growth of global spatiotemporal data, the limitations of traditional GIS, and presents SuperMap's distributed storage architecture, unified data access APIs, dynamic rendering techniques, and geographic processing modeling with real‑world case studies to address performance and scalability challenges.

GISbig data analyticsdistributed storage
0 likes · 16 min read
Distributed Storage and Application Solutions for Massive Spatiotemporal Data
Xianyu Technology
Xianyu Technology
Sep 9, 2021 · Backend Development

Design and Scaling of Xianyu Messaging System (1.0 → 4.0)

The Xianyu instant‑messaging system evolved from a 2014 MVP to a robust, multi‑version architecture—adding distributed storage, hybrid sync, ACK‑based delivery, and feature‑driven monitoring—to handle billions of messages, improve stability, and boost user satisfaction, cutting technical complaints by half.

MessagingScalabilitySystem Architecture
0 likes · 10 min read
Design and Scaling of Xianyu Messaging System (1.0 → 4.0)
Architects' Tech Alliance
Architects' Tech Alliance
Aug 6, 2021 · Big Data

Performance Optimization Techniques for the Ceph Distributed Storage System

This article reviews Ceph's architecture, enumerates common benchmarking tools, analyzes its advantages and challenges, and presents a comprehensive set of performance‑optimization methods covering storage‑engine tuning, network communication, data placement, configuration parameters, hardware‑specific adaptations, and future research directions.

CephNVMePerformance Optimization
0 likes · 20 min read
Performance Optimization Techniques for the Ceph Distributed Storage System
JD Tech
JD Tech
Jul 30, 2021 · Databases

Practical Use of HBase in a Logistics HR Data Preprocessing Platform

This article details how the logistics HR data preprocessing platform processes around 20 million daily records by adopting HBase for high‑performance, scalable, column‑oriented storage, covering its architecture, read/write mechanisms, best practices, and performance considerations.

Big DataHBaseNoSQL
0 likes · 10 min read
Practical Use of HBase in a Logistics HR Data Preprocessing Platform
vivo Internet Technology
vivo Internet Technology
Jul 28, 2021 · Industry Insights

How to Quantify Data Reliability in Distributed Storage Systems

This article analyzes the quantitative model for data reliability in distributed storage, covering factors such as disk count, replication factor, recovery time, annualized failure rate, and copyset configuration, and derives formulas to estimate yearly data loss probability for both replica and erasure‑coding schemes.

AFRData Reliabilitycopyset
0 likes · 16 min read
How to Quantify Data Reliability in Distributed Storage Systems
Cloud Native Technology Community
Cloud Native Technology Community
Jul 2, 2021 · Cloud Native

Typical Cloud‑Native Storage Solutions for Stateful Applications and Their Performance Comparison

This article reviews typical cloud‑native storage options for stateful workloads, analyzes their advantages and drawbacks, discusses the specific needs of OLTP and OLAP databases, outlines challenges in multi‑cloud environments, and presents performance test results of four popular cloud‑native storage systems.

Performance Testingcloud-nativedistributed storage
0 likes · 13 min read
Typical Cloud‑Native Storage Solutions for Stateful Applications and Their Performance Comparison
Dada Group Technology
Dada Group Technology
Jul 2, 2021 · Backend Development

Design and Implementation of a High‑Availability Coupon Platform with Distributed Storage (JimDB)

This article describes the architecture and optimization of JD.com’s coupon platform, covering the JimDB distributed in‑memory database for core storage, a massive distributed task system for product coupons, high‑availability strategies for store coupons, and the overall middle‑platform design that ensures scalability, low latency, and data consistency across millions of daily transactions.

BackendCouponSystem Architecture
0 likes · 8 min read
Design and Implementation of a High‑Availability Coupon Platform with Distributed Storage (JimDB)
Architects' Tech Alliance
Architects' Tech Alliance
Jun 30, 2021 · Cloud Computing

Ceph Distributed Storage System: Architecture, CRUSH Algorithm, and Backend Evolution

This article provides a comprehensive overview of Ceph, covering its origins, cluster architecture, CRUSH data placement algorithm, unified access interfaces, the transition from FileStore to BlueStore, and emerging storage back‑ends such as SeaStore and PFStore, highlighting performance characteristics and design trade‑offs.

BlueStoreCRUSHCeph
0 likes · 12 min read
Ceph Distributed Storage System: Architecture, CRUSH Algorithm, and Backend Evolution
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 24, 2021 · Big Data

Comprehensive Overview of HBase Architecture, Design, and Operations

This article provides an in‑depth technical overview of HBase, covering its Bigtable origins, distributed column‑store design, core components such as ZooKeeper, HMaster and RegionServer, data flow, storage formats, row‑key design, bulk loading, SQL integration, indexing, coprocessors, and performance tuning for big‑data environments.

Columnar DatabaseHBaseHDFS
0 likes · 30 min read
Comprehensive Overview of HBase Architecture, Design, and Operations
Tencent Cloud Developer
Tencent Cloud Developer
May 14, 2021 · Cloud Native

Exploration and Practice of Tencent Cloud Native Database TDSQL-C Architecture

Tencent’s cloud‑native database TDSQL‑C separates compute and storage, offering petabyte‑scale network‑attached storage, automatic scaling, million‑level QPS, MySQL/PostgreSQL compatibility, serverless per‑use billing, two‑level caching, instant snapshot‑based backup and restore, and a roadmap toward ultra‑minimal operations, multi‑region DR and column‑store analytics.

Database ArchitectureTDSQL-Ccloud database
0 likes · 16 min read
Exploration and Practice of Tencent Cloud Native Database TDSQL-C Architecture
Volcano Engine Developer Services
Volcano Engine Developer Services
May 13, 2021 · Databases

Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database

The article offers a comprehensive technical deep‑dive into ByteDance’s home‑grown distributed graph database and graph‑processing engine, ByteGraph, covering its directed‑property graph model, Gremlin query support, multi‑layer architecture, storage strategies for massive data, and real‑world graph‑computing practices.

Big DataByteGraphGremlin
0 likes · 28 min read
Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database
DataFunTalk
DataFunTalk
Apr 20, 2021 · Databases

Meituan's Graph Database Selection and Platform Construction

This article presents Meituan's comprehensive evaluation of open‑source graph databases, the rationale for selecting NebulaGraph, and the design of a high‑availability, high‑throughput graph database platform that supports multi‑hop queries, massive data ingestion, real‑time synchronization, and visualization for various business scenarios.

MeituanNebulaGraphdistributed storage
0 likes · 21 min read
Meituan's Graph Database Selection and Platform Construction
Volcano Engine Developer Services
Volcano Engine Developer Services
Mar 23, 2021 · Cloud Native

How Douyin Handled 70B Red Packet Interactions in 27 Days with Cloud‑Native Magic

In just 27 days, Douyin and Volcano Engine's cloud‑native team built a Kubernetes‑based, elastically scalable infrastructure that supported 703 billion red‑packet interactions and over a trillion live‑stream views during the 2021 Spring Festival Gala, ensuring zero downtime and seamless user experience.

Cloud NativeEdge Computingdistributed storage
0 likes · 12 min read
How Douyin Handled 70B Red Packet Interactions in 27 Days with Cloud‑Native Magic
Liangxu Linux
Liangxu Linux
Mar 18, 2021 · Operations

Deploying GlusterFS Distributed Replicated Volumes with NFS‑Ganesha

This guide explains the architecture of GlusterFS, compares its volume types, walks through setting up a multi‑node cluster, configuring NFS‑Ganesha for NFSv4 access, and shows how to mount the storage from clients using both GlusterFS and NFS protocols.

GlusterFSLinuxNFS-Ganesha
0 likes · 12 min read
Deploying GlusterFS Distributed Replicated Volumes with NFS‑Ganesha
Tencent Architect
Tencent Architect
Feb 23, 2021 · Artificial Intelligence

Analysis and Optimization of CephFS I/O Performance for AI Training on the Xingchen Compute Platform

This article investigates why AI training tasks on Tencent's Xingchen compute platform experience severe I/O slowdown when using CephFS, analyzes the underlying Ceph‑FUSE and MDS mechanisms, and proposes metadata‑caching and file‑caching optimizations that can accelerate training speed by three to four times.

AI trainingCeph-FUSECephFS
0 likes · 21 min read
Analysis and Optimization of CephFS I/O Performance for AI Training on the Xingchen Compute Platform
Architects' Tech Alliance
Architects' Tech Alliance
Jan 25, 2021 · Fundamentals

Ceph Storage Architecture Overview and Detailed Technical Features

This article provides a comprehensive technical overview of Red Hat Ceph, covering its distributed object storage design, cluster architecture, storage pools, authentication, placement groups, CRUSH algorithm, I/O operations, replication, erasure coding, internal management tasks, high availability, client interfaces, data striping, and encryption mechanisms.

CRUSHCephData Striping
0 likes · 42 min read
Ceph Storage Architecture Overview and Detailed Technical Features
IT Xianyu
IT Xianyu
Jan 18, 2021 · Backend Development

FastDFS Overview, Docker Installation, and Spring Boot Integration Guide

This article introduces FastDFS as a lightweight distributed file system, explains its tracker and storage components, provides step‑by‑step Docker installation commands, demonstrates how to integrate FastDFS with Spring Boot using Maven dependencies and configuration, and includes sample code for uploading, downloading, and deleting files.

FastDFSbackend-developmentdistributed storage
0 likes · 7 min read
FastDFS Overview, Docker Installation, and Spring Boot Integration Guide
Java Captain
Java Captain
Jan 2, 2021 · Backend Development

Design and Implementation of a Lightweight Open‑Source Netdisk System

This article introduces a lightweight, open‑source netdisk system built with a front‑end Vue.js interface and a back‑end micro‑service architecture using SpringBoot, Dubbo, Zookeeper, Redis, Solr, and FastDFS, detailing its deployment, technology stack, and core features for file management and integration.

MicroservicesSpringBootVue.js
0 likes · 5 min read
Design and Implementation of a Lightweight Open‑Source Netdisk System
dbaplus Community
dbaplus Community
Nov 29, 2020 · Databases

Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database

ByteGraph is ByteDance's home‑grown distributed graph storage and computation platform that supports massive social‑graph workloads with directed‑property models, Gremlin queries, multi‑layer architecture, adaptive KV storage, hot‑spot handling, indexing, and a roadmap toward cloud‑native, HTAP‑capable graph processing.

ByteGraphGremlindistributed storage
0 likes · 35 min read
Inside ByteGraph: How ByteDance Built a Scalable Distributed Graph Database
Architects' Tech Alliance
Architects' Tech Alliance
Nov 9, 2020 · Cloud Computing

Ceph Storage Architecture: Overview, Cluster Design, Client Interfaces, and Encryption

This article provides a comprehensive technical overview of Red Hat Ceph, covering its distributed storage architecture, cluster components, storage pool types, authentication, placement algorithms, I/O paths, replication and erasure‑coding strategies, internal management operations, high‑availability mechanisms, client libraries, data striping, and encryption details.

CRUSHCephData Striping
0 likes · 39 min read
Ceph Storage Architecture: Overview, Cluster Design, Client Interfaces, and Encryption
JD Tech Talk
JD Tech Talk
Sep 21, 2020 · Operations

Investigation and Optimization of Ceph Slow‑Request Issues Related to Scrub in the Luminous Release

This article analyzes a frequent slow‑request problem in a Ceph cluster caused by heavy write‑induced RocksDB compaction combined with deep‑scrub activity, explains the underlying scrub mechanisms, and presents parameter‑tuning and custom scheduling strategies to mitigate performance impact while preserving data consistency.

CephScrubdistributed storage
0 likes · 13 min read
Investigation and Optimization of Ceph Slow‑Request Issues Related to Scrub in the Luminous Release
JD Cloud Developers
JD Cloud Developers
Sep 15, 2020 · Databases

How JD’s HoraeDB Tackles Massive Time‑Series Data at Scale

This article introduces JD Cloud’s self‑built time‑series database HoraeDB, explaining its core concepts, typical use cases, architectural layers, high‑performance features, down‑sampling strategies, compression techniques, and stability measures for handling massive, 24‑hour monitoring data at scale.

DownsamplingTime Series Databasecompression
0 likes · 18 min read
How JD’s HoraeDB Tackles Massive Time‑Series Data at Scale
Java Backend Technology
Java Backend Technology
Sep 10, 2020 · Backend Development

Designing a Billion-Scale Feed Stream System: Architecture & Best Practices

This article explains how to design a high‑performance, billion‑user feed stream system, covering product definition, data modeling, storage choices, synchronization modes, metadata handling, commenting, likes, search, sorting, deletion, updates, and practical architecture examples for different feed‑type applications.

BackendScalabilitySystem Architecture
0 likes · 22 min read
Designing a Billion-Scale Feed Stream System: Architecture & Best Practices
Didi Tech
Didi Tech
Aug 28, 2020 · Operations

Ceph Performance Optimization: Lock-Related Issues and Solutions

The article details how Didi’s large‑scale Ceph deployment suffered from high tail latency due to long‑held and coarse‑grained locks, and describes a series of fixes—including asynchronous read threads, fine‑grained object caches, per‑thread lock‑free logging, and lock‑free filestore apply—that cut latency by up to 90 % and more than doubled read throughput.

BlueStoreCephFileStore
0 likes · 12 min read
Ceph Performance Optimization: Lock-Related Issues and Solutions
Full-Stack DevOps & Kubernetes
Full-Stack DevOps & Kubernetes
Aug 3, 2020 · Operations

How to Deploy a Ceph Distributed Storage Cluster from Scratch

This guide walks through installing and configuring a Ceph distributed storage cluster, covering node preparation, hostname and hosts setup, password‑less SSH, required packages, firewall handling, time sync, ceph‑deploy installation, monitor creation, OSD configuration, pool and RBD creation, verification, and common troubleshooting steps.

CephDevOpsLinux
0 likes · 16 min read
How to Deploy a Ceph Distributed Storage Cluster from Scratch
Senior Brother's Insights
Senior Brother's Insights
Jul 30, 2020 · Backend Development

Integrate FastDFS with Spring Boot for File Upload, Download, and Management

This tutorial shows how to integrate the FastDFS distributed file system into a Spring Boot application, covering Maven dependencies, configuration files, utility classes, core client initialization, and controller code for uploading, downloading, retrieving, and deleting files, with complete code examples and a GitHub source link.

BackendFastDFSJava
0 likes · 9 min read
Integrate FastDFS with Spring Boot for File Upload, Download, and Management
MaGe Linux Operations
MaGe Linux Operations
Jul 22, 2020 · Cloud Native

Why Ceph Is the Backbone of Modern Cloud‑Native Storage Solutions

This article traces Ceph’s evolution from its academic origins to widespread adoption in cloud‑native environments, explains the four stages of storage development, details Ceph’s unified object, block, and file interfaces, and describes its architecture—including monitors, OSDs, CRUSH algorithm, placement groups, and high‑availability features.

CephCloud Nativeblock storage
0 likes · 17 min read
Why Ceph Is the Backbone of Modern Cloud‑Native Storage Solutions
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 11, 2020 · Big Data

Alluxio Tiered Metadata Management and Asynchronous Cache Eviction Implementation

The article explains Alluxio's tiered metadata management architecture, describing how the system separates hot and cold metadata into cached and persisted layers, and details the custom asynchronous eviction thread and cache implementation that replace Guava cache for efficient large‑scale metadata handling.

AlluxioCachedistributed storage
0 likes · 15 min read
Alluxio Tiered Metadata Management and Asynchronous Cache Eviction Implementation
Cloud Native Technology Community
Cloud Native Technology Community
Jul 7, 2020 · Cloud Native

Taming etcd Instability: Lessons from Managing Million‑Node Kubernetes Clusters

This article details how Tencent Cloud’s TKE team identified, analyzed, reproduced, and resolved multiple etcd stability and performance issues—including data inconsistency, memory leaks, mvcc deadlocks, and WAL crashes—while sharing the lessons learned and the optimizations applied to support million‑node Kubernetes deployments.

Kubernetescloud-nativedistributed storage
0 likes · 29 min read
Taming etcd Instability: Lessons from Managing Million‑Node Kubernetes Clusters
dbaplus Community
dbaplus Community
Apr 27, 2020 · Cloud Native

How OPPO Cloud Storage Leverages Kubernetes and CSI for Cloud‑Native Solutions

This article explains OPPO's cloud storage journey, covering distributed storage fundamentals, the architecture of its block storage platform, the role of Kubernetes and the Container Storage Interface, sidecar components, and the operational benefits of containerizing storage in a cloud‑native environment.

CSICloud Native StorageKubernetes
0 likes · 9 min read
How OPPO Cloud Storage Leverages Kubernetes and CSI for Cloud‑Native Solutions
dbaplus Community
dbaplus Community
Apr 15, 2020 · Big Data

How Ctrip Scaled Hadoop Across Data Centers: Architecture and Lessons

This article details Ctrip's Hadoop evolution, the challenges of expanding across multiple data centers, the evaluation of multi‑cluster versus single‑cluster designs, and the concrete architectural changes, migration tools, bandwidth monitoring, and future plans that enabled a stable cross‑datacenter big‑data platform.

Big DataCross-DataCenterHDFS
0 likes · 19 min read
How Ctrip Scaled Hadoop Across Data Centers: Architecture and Lessons
vivo Internet Technology
vivo Internet Technology
Mar 18, 2020 · Databases

Vivo Feature Storage Practice: Architecture, Design, and Future Directions Using Nebula Graph

Vivo’s feature‑storage platform, built on Nebula Graph’s Raft‑based, storage‑compute‑separated architecture and exposed via Redis‑compatible proxies, meets massive, low‑latency AI data demands while offering strong consistency, horizontal scalability, backup, active‑active replication, and a roadmap toward general‑purpose KV, cloud‑native integration, and advanced storage engines.

Feature StoreKVNebula Graph
0 likes · 21 min read
Vivo Feature Storage Practice: Architecture, Design, and Future Directions Using Nebula Graph
DataFunTalk
DataFunTalk
Feb 26, 2020 · Databases

ByteGraph: ByteDance’s Distributed Graph Database and Graph Computing System – Architecture, Data Model, and Practices

This article presents an in‑depth technical overview of ByteGraph, ByteDance’s self‑built distributed graph database and its accompanying graph‑computing engine, covering graph data characteristics, the directed‑property graph model, API design, three‑tier system architecture, storage strategies using KV stores and B‑Trees, hotspot handling, indexing, and future research directions.

B+TreeByteGraphGremlin
0 likes · 33 min read
ByteGraph: ByteDance’s Distributed Graph Database and Graph Computing System – Architecture, Data Model, and Practices
Big Data Technology Architecture
Big Data Technology Architecture
Feb 11, 2020 · Databases

JDHBase Multi‑Active Architecture and Asynchronous Replication Practices

This article describes JDHBase’s large‑scale KV storage architecture, its HBase‑based asynchronous replication mechanism, multi‑active cluster design, client‑side routing via Fox Manager, automatic failover strategies, dynamic replication tuning, and serial replication techniques to ensure data consistency across geographically distributed data centers.

ConsistencyDynamic TuningHBase Replication
0 likes · 12 min read
JDHBase Multi‑Active Architecture and Asynchronous Replication Practices
dbaplus Community
dbaplus Community
Jan 15, 2020 · Databases

How Didi Built Fusion-NewSQL: A High‑Throughput, Low‑Latency NewSQL on Distributed KV

Fusion-NewSQL is Didi’s internally‑developed NewSQL system built atop the Fusion distributed KV store, offering MySQL compatibility, high throughput, low latency, flexible schema changes, secondary indexes, and integration with ElasticSearch and Hive, with detailed architecture, data flow, and future roadmap.

MySQL compatibilityNewSQLdistributed storage
0 likes · 16 min read
How Didi Built Fusion-NewSQL: A High‑Throughput, Low‑Latency NewSQL on Distributed KV
JD Retail Technology
JD Retail Technology
Jan 6, 2020 · Backend Development

JDHBase Multi‑Active Architecture and Replication Practices

This article describes JDHBase’s large‑scale KV storage deployment, its HBase‑based asynchronous replication mechanism, the multi‑active architecture with active‑standby clusters, client interaction via Fox Manager, automatic failover strategies, dynamic replication tuning, and serial replication techniques to ensure data consistency across data centers.

Cluster ManagementHBaseReplication
0 likes · 13 min read
JDHBase Multi‑Active Architecture and Replication Practices
UCloud Tech
UCloud Tech
Nov 19, 2019 · Cloud Computing

Why Object Storage Is the Future of Scalable Cloud Data Management

This lesson introduces object storage, explaining its concept, technical principles, market scale, advantages over traditional DAS/NAS/SAN solutions, and key architectural components, while highlighting scalability, cost efficiency, high availability, and design considerations for modern cloud storage systems.

data availabilitydistributed storageobject storage
0 likes · 7 min read
Why Object Storage Is the Future of Scalable Cloud Data Management
58 Tech
58 Tech
Nov 8, 2019 · Databases

Design and Optimization Practices of 58.com Distributed KV Storage System (WTable)

The article details the three‑stage evolution of 58.com’s distributed key‑value storage system WTable, covering its initial architecture, performance and operational optimizations, and the introduction of NewSQL‑style strong consistency and Raft‑based replication to improve scalability and reliability.

NewSQLSystem Architecturedistributed storage
0 likes · 9 min read
Design and Optimization Practices of 58.com Distributed KV Storage System (WTable)
JD Retail Technology
JD Retail Technology
Oct 14, 2019 · Databases

Overview of JDNoSQL Platform and Its Real-Time Advertising Use Cases

The article introduces JDNoSQL, a distributed column‑oriented key‑value store built on HDFS, outlines its core features, describes various business scenarios including real‑time ad computation, details the system architecture with Kafka and Flink, and presents table designs for ad impression and click statistics.

Big DataFlinkKafka
0 likes · 13 min read
Overview of JDNoSQL Platform and Its Real-Time Advertising Use Cases
Didi Tech
Didi Tech
Sep 20, 2019 · Big Data

FastLoad: A One-Click DTS Platform for Online Data Migration

FastLoad, Didi’s one‑click DTS platform, accelerates migration of terabyte‑scale offline data into its Fusion storage by using RocksDB’s IngestFile to import SST files directly, cutting a 1 TB load from twelve hours to one, while supporting thousands of daily tasks with 99.99% stability.

DTS platformData MigrationPerformance Optimization
0 likes · 8 min read
FastLoad: A One-Click DTS Platform for Online Data Migration
Alibaba Cloud Native
Alibaba Cloud Native
Sep 19, 2019 · Databases

How etcd’s Fully Concurrent Read Boosts Kubernetes Performance

This article reviews the evolution of etcd’s read‑write mechanisms, explains the “Fully Concurrent Read” feature introduced in etcd 3.4, and presents experimental results showing how it dramatically reduces expensive read latency and improves overall throughput in Kubernetes clusters.

BoltDBFully Concurrent ReadKubernetes
0 likes · 14 min read
How etcd’s Fully Concurrent Read Boosts Kubernetes Performance
Programmer DD
Programmer DD
Sep 10, 2019 · Databases

Understanding Nebula Graph: Data Model and Architecture Explained

This article introduces Nebula Graph, an open‑source distributed graph database, detailing its directed property graph model, vertex and edge schemas, graph partitioning, storage, metadata, query engine, and client APIs, highlighting its strong schema design, high‑availability architecture, and scalability for trillion‑scale graphs.

Nebula GraphSystem Architecturedata-model
0 likes · 11 min read
Understanding Nebula Graph: Data Model and Architecture Explained
DevOps Cloud Academy
DevOps Cloud Academy
Aug 11, 2019 · Operations

Recovering MFS Master Node Using Metalogger After Node Failure

When the mfsmaster node in a six‑node MFS distributed storage cluster becomes unresponsive, the issue can be resolved by using metalogger on another node to restore the metadata file, copy it to the master directory, adjust permissions, and restart the mfsmaster service.

LinuxMFSNode Recovery
0 likes · 1 min read
Recovering MFS Master Node Using Metalogger After Node Failure
Tencent Cloud Developer
Tencent Cloud Developer
Aug 1, 2019 · Databases

FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges

FeatureKV, WeChat’s high‑performance key‑value store, handles one‑billion queries per second and ingests a billion keys per hour by separating write‑only DataSvr from read‑only KVSvr, supporting in‑memory, indexed, and block‑indexed tables, scaling horizontally, guaranteeing eventual consistency with versioned reads, and delivering up to 11 billion reads per second with sub‑15 ms latency.

Batch ProcessingFeatureKVKey-Value
0 likes · 22 min read
FeatureKV: A High-Performance Key-Value Storage System for WeChat's Billion-Scale Challenges
58 Tech
58 Tech
Jul 25, 2019 · Databases

Design and Evolution of WTable’s Scaling Process Using RocksDB

This article explains how the WTable distributed key‑value store leverages RocksDB’s LSM‑tree architecture and slot‑based data distribution to redesign its scaling workflow, separating full and incremental data migration to reduce compaction overhead and achieve high‑speed, low‑impact cluster expansion.

Data MigrationRocksDBWTable
0 likes · 8 min read
Design and Evolution of WTable’s Scaling Process Using RocksDB
System Architect Go
System Architect Go
Jul 19, 2019 · Big Data

Introduction to HBase: Architecture, Data Model, and Operations

This article provides a comprehensive overview of HBase, covering its distributed column‑oriented architecture, data model components, storage mechanisms, read/write processes, WAL lifecycle, MemStore flushing, region splitting and merging, and failure recovery within the Hadoop ecosystem.

Big DataHBaseHadoop
0 likes · 20 min read
Introduction to HBase: Architecture, Data Model, and Operations
dbaplus Community
dbaplus Community
Jul 18, 2019 · Databases

How JD.com Scales HBase to 90PB: Architecture, Optimizations, and Lessons

This article examines JD.com's massive HBase deployment, detailing its evolution from early adoption to a 90PB, 7,000‑node cluster, the platform's architecture, multi‑active disaster recovery, multi‑tenant isolation, and the integration of Phoenix for SQL‑based access, offering practical insights for large‑scale distributed storage.

Big DataDatabase ArchitectureHBase
0 likes · 15 min read
How JD.com Scales HBase to 90PB: Architecture, Optimizations, and Lessons
DataFunTalk
DataFunTalk
Jul 15, 2019 · Big Data

Key Infrastructure Considerations for Autonomous Driving: Storage, Computing, and Services

The article reviews the essential infrastructure for autonomous driving, covering massive sensor data storage strategies, the role of metadata, offline and real‑time computing platforms, basic micro‑service components, and various business scenarios, highlighting why robust big‑data handling is critical.

Big DataReal‑Time Computingautonomous driving
0 likes · 14 min read
Key Infrastructure Considerations for Autonomous Driving: Storage, Computing, and Services
Architects' Tech Alliance
Architects' Tech Alliance
Jul 14, 2019 · Fundamentals

Comparison of Ceph and GlusterFS Storage Architectures

This article provides an in‑depth comparison of Red Hat’s Ceph and GlusterFS open‑source storage solutions, detailing their underlying architectures—including RADOS, CephFS, RBD, and Gluster’s brick‑based design—while discussing performance, scalability, high‑availability, hardware requirements, and suitability for various cloud environments.

CephComparisonGlusterFS
0 likes · 10 min read
Comparison of Ceph and GlusterFS Storage Architectures
360 Quality & Efficiency
360 Quality & Efficiency
Jun 11, 2019 · Backend Development

NebulasFs: A Distributed High‑Availability Small‑File Storage System Developed by 360 Infrastructure Team

NebulasFs is a self‑developed distributed, highly available, and persistent storage system designed to efficiently store billions of small files, offering simple RESTful APIs, automatic request routing, multi‑tenant isolation, customizable replication, automated scaling, rebalancing, and fault‑tolerant replica recovery for large‑scale unstructured data workloads.

NebulasFsSmall Filescloud
0 likes · 8 min read
NebulasFs: A Distributed High‑Availability Small‑File Storage System Developed by 360 Infrastructure Team
360 Tech Engineering
360 Tech Engineering
Jun 11, 2019 · Databases

NebulasFs: A Distributed High‑Availability Small‑File Storage System

NebulasFs is a self‑developed distributed, highly available and durable storage system designed to efficiently store billions of small files by using a master‑datanode architecture, multi‑tenant isolation, customizable replication, automatic scaling, and automated replica repair, addressing the challenges of massive unstructured data generated by modern applications.

Cloud NativeNebulasFsReplication
0 likes · 7 min read
NebulasFs: A Distributed High‑Availability Small‑File Storage System
21CTO
21CTO
Jun 8, 2019 · Cloud Computing

Unlocking Ceph: Deep Dive into Architecture, IO Flow, Heartbeat, CRUSH & QOS

This article provides a comprehensive overview of Ceph, covering its architecture and use cases, detailed IO processes, heartbeat mechanisms, communication framework, the CRUSH data placement algorithm, and customizable RBD QoS policies, illustrated with diagrams and code snippets to help readers understand and implement Ceph in cloud environments.

CRUSHCephIO Architecture
0 likes · 27 min read
Unlocking Ceph: Deep Dive into Architecture, IO Flow, Heartbeat, CRUSH & QOS
58 Tech
58 Tech
May 21, 2019 · Backend Development

Design and Architecture of WOS: 58 Group's Self‑Developed Object Storage System

This article presents the architecture and key design features of WOS, the 58 Group’s self‑developed object storage system, covering its overall framework, proxy, store, directory, detector modules, fast‑upload “秒传” mechanism, and erasure‑coding solution for efficient, scalable, and reliable unstructured data storage.

Backend ArchitectureWOSdistributed storage
0 likes · 12 min read
Design and Architecture of WOS: 58 Group's Self‑Developed Object Storage System
Architects' Tech Alliance
Architects' Tech Alliance
May 15, 2019 · Fundamentals

Red Hat Ceph Storage Architecture Guide – Overview and Core Concepts

This article provides a comprehensive overview of Red Hat Ceph's distributed object storage architecture, covering storage pools, CRUSH placement, authentication, I/O workflows, internal operations, client interfaces, data striping, erasure coding, high availability, and encryption mechanisms for secure, scalable deployments.

CRUSHCephdistributed storage
0 likes · 40 min read
Red Hat Ceph Storage Architecture Guide – Overview and Core Concepts
Big Data Technology Architecture
Big Data Technology Architecture
May 11, 2019 · Cloud Computing

Open Source Distributed Object Storage Solutions Overview

This article introduces the concepts of block, file, and object storage and reviews several open‑source distributed object storage solutions—including Swift, Ceph, MinIO, HBase MOB, and Hadoop Ozone—highlighting their architectures, features, and typical use cases for large‑scale data handling.

CephHBase MOBMinio
0 likes · 8 min read
Open Source Distributed Object Storage Solutions Overview
Alibaba Cloud Developer
Alibaba Cloud Developer
May 6, 2019 · Backend Development

How Modern IM Systems Achieve Multi-Device Sync, Roaming, and Real-Time Search

Modern instant messaging platforms have evolved from simple CS/P2P designs to complex distributed architectures that store messages before synchronizing, enabling features like multi-device sync, message roaming, and online search, with Alibaba Cloud Tablestore’s Timeline model providing scalable storage, indexing, and flexible synchronization strategies.

cloud computingdistributed storagemessage synchronization
0 likes · 16 min read
How Modern IM Systems Achieve Multi-Device Sync, Roaming, and Real-Time Search
JD Retail Technology
JD Retail Technology
Apr 10, 2019 · Databases

HBase at JD.com: Architecture, Use Cases, and Evolution

This article explains how JD.com leverages the open‑source HBase database for massive, low‑latency data storage across various business lines, detailing its architecture, multi‑tenant isolation, disaster‑recovery mechanisms, and integration with Phoenix SQL for OLTP workloads.

Big DataDatabase ArchitectureHBase
0 likes · 13 min read
HBase at JD.com: Architecture, Use Cases, and Evolution
Tencent Cloud Developer
Tencent Cloud Developer
Mar 29, 2019 · Databases

Design of High‑Availability System and Fast Recovery in Tencent CynosDB

Tencent CynosDB achieves high availability and rapid recovery through an external HA service that combines a co‑located monitoring agent, a ZooKeeper‑backed scheduler for fault detection, decision making, and automated switch/rejoin/rebuild actions, and a VDL‑driven distributed storage recovery mechanism that prevents split‑brain scenarios.

Agent‑SchedulerCynosDBDatabase Architecture
0 likes · 19 min read
Design of High‑Availability System and Fast Recovery in Tencent CynosDB
Tencent Cloud Developer
Tencent Cloud Developer
Mar 26, 2019 · Databases

Technical Overview of Tencent Cloud CynosDB Distributed Storage and Compute‑Storage Separation Architecture

The Beijing technical exchange detailed Tencent Cloud CynosDB’s compute-storage separation architecture, explaining how CynosStore’s block and file-system layers asynchronously persist write-ahead logs via Raft, provide multi-transaction atomicity, continuous snapshots, and rapid recovery, while benchmarks demonstrate high log-throughput and the Q&A covered tenancy, failover, and scaling.

CynosDBDatabase ArchitectureNewSQL
0 likes · 14 min read
Technical Overview of Tencent Cloud CynosDB Distributed Storage and Compute‑Storage Separation Architecture
58 Tech
58 Tech
Feb 26, 2019 · Backend Development

Architecture Evolution of the WeChat‑like Instant Messaging Service at 58 Group

The article details the step‑by‑step evolution of the WeChat‑like instant messaging platform at 58 Group, describing how the system transitioned from a simple PHP short‑connection architecture to a layered micro‑service architecture using Java‑based RPC, Go‑based short‑connection services, distributed storage, message queues, and clear separation of storage, logic, and interface layers.

Backend ArchitectureGoInstant Messaging
0 likes · 12 min read
Architecture Evolution of the WeChat‑like Instant Messaging Service at 58 Group
JD Tech
JD Tech
Feb 18, 2019 · Big Data

Understanding HBase: Advantages, Use Cases, Data Model, and Architecture

This article explains HBase as a high‑performance, column‑oriented distributed storage system, outlines its advantages and limitations, presents real‑world scenarios such as seller operation logs and message logs, and details its data structures, architecture components, and design considerations for big‑data applications.

HBaseNoSQLarchitecture
0 likes · 9 min read
Understanding HBase: Advantages, Use Cases, Data Model, and Architecture
Tencent Database Technology
Tencent Database Technology
Jan 24, 2019 · Databases

Architecture Evolution and Performance Optimization of Tencent's CKV Distributed In-Memory Storage System

This article details the architectural evolution and comprehensive refactoring of Tencent's CKV distributed in-memory key-value storage system, highlighting its transition to a two-tier single-process multi-threaded model, multi-tenant resource isolation, deep Redis protocol compatibility, and optimized memory engine, which collectively deliver significant improvements in performance, scalability, and operational cost efficiency.

Database ArchitectureIn-Memory DatabaseMulti-Tenant Architecture
0 likes · 10 min read
Architecture Evolution and Performance Optimization of Tencent's CKV Distributed In-Memory Storage System
58 Tech
58 Tech
Jan 7, 2019 · Big Data

Comparison of Kuaishou BlobStore and 58 WOS Object Storage Systems

The article summarizes the technical talk from the 58 Group technology salon, detailing the architectures, scalability, high‑availability mechanisms, and storage models of Kuaishou's BlobStore and 58's WOS, and compares their design choices for large‑scale object storage.

BlobStoreHBaseWOS
0 likes · 9 min read
Comparison of Kuaishou BlobStore and 58 WOS Object Storage Systems
Tencent Cloud Developer
Tencent Cloud Developer
Dec 20, 2018 · Databases

CynosDB Architecture and Optimization: A PostgreSQL-Compatible NewSQL Database

CynosDB, Tencent’s PostgreSQL‑compatible NewSQL service, separates compute and storage, uses a log‑based distributed CynosStore with idempotent logs, offloads CRC checks, and implements async table extension, eliminating full‑page writes and dirty‑page flushing to deliver scalable, cost‑effective performance while preserving PostgreSQL features.

CynosDBDatabase ArchitectureLog System Optimization
0 likes · 12 min read
CynosDB Architecture and Optimization: A PostgreSQL-Compatible NewSQL Database
Baidu Intelligent Testing
Baidu Intelligent Testing
Dec 6, 2018 · Fundamentals

Introduction to CDN System Architecture and Workflow

This article provides a beginner-friendly overview of traditional CDN components, their relationships, and a step‑by‑step download workflow, covering entry service, core indexing, core storage, content management, edge storage, and content distribution without delving into advanced features like grouping or P2P.

CDNContent Delivery NetworkEdge Storage
0 likes · 11 min read
Introduction to CDN System Architecture and Workflow
Beike Product & Technology
Beike Product & Technology
Nov 23, 2018 · Backend Development

Elasticsearch Internals: Distributed Document Storage, Real‑time Search, and Translog Mechanics

This article explains the core Elasticsearch architecture—including shard routing, primary‑replica interaction, document CRUD workflows, multi‑document APIs, segment merging, translog durability, and storage file formats—providing a comprehensive view of how near‑real‑time search is achieved on large‑scale data.

ElasticsearchSegment Mergingdistributed storage
0 likes · 20 min read
Elasticsearch Internals: Distributed Document Storage, Real‑time Search, and Translog Mechanics
ITPUB
ITPUB
Nov 2, 2018 · Databases

How Lest Redefines Persistent Caching for Petabyte‑Scale KV Stores

This article presents Lest, a persistent KV‑store cache designed to overcome the limitations of traditional in‑memory caches by combining disk‑based persistence, lock‑free multithreading, custom protocols, and advanced load‑balancing, and it evaluates its performance on petabyte‑scale workloads.

KV StoreLestPerformance Testing
0 likes · 11 min read
How Lest Redefines Persistent Caching for Petabyte‑Scale KV Stores
58 Tech
58 Tech
Nov 1, 2018 · Databases

Insights from the 58 Group Technical Salon: Distributed KV Storage Systems Cellar and WTable

The article summarizes the 58 Group technical salon where experts compared Meituan‑Dianping's Cellar and 58's WTable distributed KV storage systems, detailing their architectures, improvements, scalability, high‑availability mechanisms, and operational considerations, and concludes with a comparative analysis and preview of the next session.

BackendCellarKV Store
0 likes · 10 min read
Insights from the 58 Group Technical Salon: Distributed KV Storage Systems Cellar and WTable
Alibaba Cloud Developer
Alibaba Cloud Developer
Oct 16, 2018 · Operations

Alibaba’s OS, Storage, and Resource Management Highlights from OSDI'18

The 13th OSDI conference in Carlsbad attracted over 650 attendees, featuring 47 accepted papers and three Best Papers—two led by Chinese students—while Alibaba showcased its latest OS kernel (AliKernel), next‑generation distributed storage system Pangu 2.0, and the large‑scale resource manager Sigma, sparking lively discussions among global experts.

AlibabaOSDIOperating Systems
0 likes · 8 min read
Alibaba’s OS, Storage, and Resource Management Highlights from OSDI'18