Tagged articles
Data Reliability
20 articles · Page 1 of 1
This article explains how Erasure Coding (EC) improves data reliability and dramatically reduces storage overhead in Ozone by leveraging hot‑cold data characteristics, intelligent tiering, dynamic EC ratios, and repair throttling, while also discussing performance trade‑offs and limitations.
Data ReliabilityErasure CodingObject Storage
0 likes · 9 min read

This article introduces a newly released comprehensive guide on SSD flash technology while providing an in‑depth analysis of emerging Storage Class Memory (SCM) technologies—such as PCM, ReRAM, MRAM, and NRAM—detailing their principles, current research challenges, and potential system‑level innovations.
Data ReliabilityHardware PrototypingMemory Technologies
0 likes · 18 min read

Didi’s multi‑year data‑stability program for its customer‑service platform progressed through fault‑centered engineering, business‑aligned cross‑team work, and capability normalization, instituting pre‑, mid‑ and post‑fault safeguards, clear ownership, automated alerts and repair tools, which cut fault count by 42 % and more than doubled mean‑time‑to‑repair while boosting team communication and satisfaction.
Data ReliabilityODSautomation
0 likes · 16 min read

This article explains how Apache Pulsar combines messaging, storage, and lightweight function computing into a cloud‑native streaming platform, detailing its architecture, storage‑compute separation, tiered storage, pluggable protocols, reliability guarantees, and rich ecosystem compared with traditional queues and Kafka.
Apache PulsarData Reliabilitycloud-native
0 likes · 10 min read

MySQL replication copies data changes from a primary server to one or more replicas using binlog events—supporting statement, row, or mixed formats and GTID positioning—to provide real‑time backup, read‑write separation, high‑availability failover, and integration pipelines via asynchronous, semi‑synchronous, or centralized binlog services.
Data ReliabilityGTIDMySQL
0 likes · 30 min read

Bilibili reduced petabyte‑scale storage costs by back‑porting erasure‑coding patches to its HDFS 2.8.4 cluster, deploying a parallel EC‑enabled cluster, adding a data‑proxy service, intelligent routing and block‑checking, and automating cold‑data migration, while noting write overhead and planning native acceleration.
Data ReliabilityErasure CodingHDFS
0 likes · 14 min read

This article explains the design, implementation, and future plans of Baidu's log middle platform, detailing its lifecycle management, service architecture, data reliability goals of eliminating duplication and loss, and the technical measures taken across SDKs, servers, and streaming pipelines to achieve near‑100% data integrity.
Data Reliabilitybackend architecturebig data
0 likes · 15 min read

This article describes Baidu's log middle platform architecture, its data lifecycle management, integration status, terminology, service overview, core challenges of ensuring data accuracy, and the implemented optimizations for persistent storage, service decomposition, and SDK reporting to achieve near‑100% no‑repeat no‑loss reliability.
Data Reliabilitybackend architecturebig data
0 likes · 15 min read

This article introduces the current state of Baidu's log platform, explains its lifecycle from data collection to downstream applications, analyzes the challenges of achieving near‑zero duplication and loss, and presents architectural optimizations and best‑practice recommendations to improve data stability and accuracy across the system.
Data Reliabilitybig datadata pipeline
0 likes · 19 min read

This article explains Kafka's message delivery semantics, identifies the exact scenarios where data can be lost in producer, broker, and consumer components, and provides concrete configuration and coding practices to ensure reliable, at‑least‑once or exactly‑once delivery in production environments.
ConsumerData ReliabilityKafka
0 likes · 18 min read

This article analyzes the quantitative model for data reliability in distributed storage, covering factors such as disk count, replication factor, recovery time, annualized failure rate, and copyset configuration, and derives formulas to estimate yearly data loss probability for both replica and erasure‑coding schemes.
AFRData ReliabilityErasure Coding
0 likes · 16 min read

UCloud’s US3 archive storage leverages high‑density SMR drives combined with JBOD architecture and a custom IO‑scheduling algorithm to slash hardware CAPEX by up to 80%, reduce OPEX electricity costs, and maintain data reliability through embedded metadata and dual‑head redundancy.
Cost OptimizationData ReliabilityIO scheduling
0 likes · 11 min read

This article details how 360 adopted Cassandra for its cloud drive, describing Cassandra’s decentralized architecture, the reasons for its selection over HBase, large‑scale deployment challenges, performance optimizations, reliability improvements, disk utilization techniques, and the evolution of the system from 2010 to present.
CassandraData ReliabilityDistributed Database
0 likes · 15 min read

The article explains how OceanBase, a distributed enterprise‑grade database, achieves strong data reliability and rapid service recovery on ordinary PC servers by combining Paxos‑based consensus, enhanced redo‑log verification, periodic checkpoint checks, and fine‑grained fail‑over mechanisms, surpassing traditional hardware‑dependent databases.
Data ReliabilityHigh AvailabilityOceanBase
0 likes · 17 min read

This article presents a comprehensive data quality model derived from ISO 9126, maps its characteristics to data testing, outlines practical testing methods and tool requirements, and demonstrates how to integrate quality checks into the data development lifecycle for reliable, efficient big‑data pipelines.
Data ReliabilityISO 9126Test Automation
0 likes · 28 min read

This article explains the core principles of message queues, outlines common data‑loss scenarios for RabbitMQ and Kafka, and provides practical techniques—such as transactions, confirm mode, persistence settings, and replication configurations—to ensure reliable, loss‑free messaging.
Data ReliabilityKafkaMessage Queue
0 likes · 9 min read

This article explains Kafka's mechanisms for ensuring data reliability through partition replicas, producer acknowledgments, and leader election, and describes how the high‑water‑mark and ISR concepts maintain strong data consistency across the cluster.
ConsistencyData ReliabilityKafka
0 likes · 9 min read

At the 2023 Tencent Cloud + Future summit, Liu Jinming detailed Tencent Cloud COS’s three‑tier EB‑level object storage architecture—covering network, application, and data layers—highlighting its 99.95% availability, 11‑nine durability, end‑to‑end encryption, scalable performance, tiered pricing, and real‑world media and security use cases.
Data ReliabilityObject StorageTencent Cloud
0 likes · 10 min read

The article examines the recurring bugs and reliability issues in Linux file systems—especially ext4—highlighting Linus Torvalds' criticism, POSIX shortcomings, fuzz‑testing results, historical bug statistics, and practical advice for developers to avoid data‑loss pitfalls.
Data ReliabilityEXT4Linux
0 likes · 11 min read

Facing a 0.02% SQLite corruption rate that threatened years of chat history, the WeChat mobile team identified three main causes—insufficient space, power loss, and sync failures—and implemented space management, full sync settings, and master‑table backups, halving damage and doubling repair success.
Data ReliabilitySQLitebackup
0 likes · 9 min read
