Tagged articles
storage architecture
100 articles · Page 1 of 1
The article details how Sugon 8000 overcame engineering challenges to build China's first fully domestic 100,000-card AI supercluster, covering chip architecture, mechanical structure, high-speed networking, storage, and liquid cooling innovations through cross-team collaboration and iterative testing.
100K-card clusterAI superclusterSugon 8000
0 likes · 15 min read

Sugon's FN Neo redefines centralized all-flash storage for AI inference by turning KV Cache into a shared cluster-level asset, delivering native KV semantics with sub-millisecond latency, super-tunnel contention-free data paths, and three-tier load balancing that yields 6–12× context-length speedups on DeepSeek-R1 and Qwen-2.5 across single-node, multi-node, and real-world AI coding workloads.
AI inferenceFN NeoKV Cache
0 likes · 11 min read

Enterprise data platforms face common questions—why add a data warehouse when MySQL or Oracle exist, why introduce a data lake after a warehouse, and what problems NoSQL, ClickHouse, or Doris solve—this article breaks down six storage technologies, their core use‑cases, and how to choose among them.
NoSQLOLAPdata lake
0 likes · 15 min read

This article walks through MySQL InnoDB's five‑level storage hierarchy—record, page, extent, segment, tablespace—explaining page internals, record chaining, page directories, headers, trailers, extent and segment management, and the system tablespace and data dictionary across MySQL 5.7 and 8.0.
InnoDBMySQLdata page
0 likes · 24 min read


ITPUBJun 29, 2026 · Artificial Intelligence
The article analyzes XSKY AIMesh 2026’s layered AI data infrastructure—detailing G1‑G5 tiers, MeshFusion, MeshFS, and MeshSpace—to show how it tackles memory, I/O, and gravity walls, improves performance, and creates a unified data pipeline for inference, training, and long‑term governance.
AIData InfrastructureMeshFS
0 likes · 10 min read

The article explains how database indexes work by comparing them to a book’s table of contents, describes underlying storage mechanisms, demonstrates binary search on sorted records, and discusses the benefits, trade‑offs, and common pitfalls of using clustered and non‑clustered indexes.
Database IndexesQuery PerformanceSQL Optimization
0 likes · 12 min read

This article compares Direct‑Attached Storage (DAS), Network‑Attached Storage (NAS) and Storage Area Networks (SAN), detailing their characteristics, advantages, limitations and typical use cases, to help engineers choose the most suitable storage architecture for performance, cost and complexity requirements.
DASNASSAN
0 likes · 13 min read

This article compares the storage models, data organization, indexing, read/write processes, and performance trade‑offs of three major message queues—Kafka, RocketMQ, and JMQ—providing detailed technical insights for architects and engineers making storage‑related design decisions.
Backend EngineeringJMQKafka
0 likes · 16 min read

This article analyzes the technical principles, architectural innovations, and real‑world scenarios of Data Processing Units (DPUs), showing how they resolve storage‑CPU mismatches, eliminate excessive east‑west traffic, and accelerate failure recovery, thereby becoming a core infrastructure for AI and cloud computing.
AIDPUNVMe
0 likes · 15 min read

Huolala, founded in 2013, runs a massive cross‑cloud hybrid big‑data storage platform of over 40 PB across 3,000+ machines, evolving through four online‑storage phases, robust HA design, performance‑cost optimizations, AI vector storage, and a cost‑governance system that saved more than half of its storage expenses.
AI vector storageCost OptimizationHigh Availability
0 likes · 18 min read

This article explains how databases store data on various storage devices, why indexes dramatically improve query performance through sorted structures and binary search, and outlines practical SQL optimization techniques while warning about the trade‑offs of excessive indexing.
SQL Performancebinary searchclustered index
0 likes · 11 min read

3FS is an open‑source distributed file system that abstracts multiple machines into a single namespace, offering massive scalability, fault tolerance, and high throughput through components like Meta, Mgmtd, Storage, and Client, and leveraging the CRAQ protocol for strong consistency and efficient reads and writes.
3FSCRAQdistributed file system
0 likes · 12 min read

This article provides a concise overview of four key distributed file systems—HDFS, CephFS, GFS, and TFS—explaining their architectures, strengths, weaknesses, and typical application scenarios for large‑scale data storage and processing.
CephFSGFSHDFS
0 likes · 5 min read

RocketMQ tackles the high‑performance, high‑reliability challenges of distributed messaging by combining sequential disk writes, memory caching, and multi‑level indexing, detailing its storage logic, core structures, zero‑copy techniques, replication modes, static topic scaling, and practical tuning guidelines for optimal throughput.
Message QueueRocketMQstorage architecture
0 likes · 8 min read

This article explains how databases store data on various storage media, how indexes—especially clustered indexes—leverage sorted structures and binary search to dramatically reduce query time, and outlines best practices and pitfalls of index usage and SQL optimization.
SQL Optimizationbinary searchclustered index
0 likes · 12 min read

This article provides a detailed, step‑by‑step analysis of RabbitMQ's classic queue storage architecture, covering directory layout, index and data file formats, write and read workflows, file compaction, and practical operational tips such as publisher confirms and manual acknowledgments to improve reliability and throughput.
Classic QueueMessage BrokerRabbitMQ
0 likes · 13 min read

This article explores the principles of data replication, comparing shared memory, shared disk, and non‑shared storage architectures, detailing replication formats, consistency challenges, and various replication strategies such as synchronous, asynchronous, semi‑synchronous, and majority‑based approaches, helping engineers choose the right trade‑offs.
Consistencyasynchronous replicationdata replication
0 likes · 12 min read

Learn how to design, configure, and deploy a Ceph distributed storage cluster using ceph‑deploy, covering storage fundamentals, Ceph architecture, component roles, network planning, OS preparation, mon, mgr, osd setup, and dashboard activation, with detailed commands and best‑practice recommendations for production environments.
CephDashboarddistributed storage
0 likes · 28 min read

This article analyzes the evolution of distributed storage—from traditional compute‑storage separation to edge‑centric, AI‑enabled architectures—covering service models, key technologies such as CXL and erasure coding, reliability strategies, performance optimizations, vendor landscapes, and emerging green and intelligent trends.
AI storageCXLNVMe-oF
0 likes · 13 min read

This article provides a comprehensive overview of Ceph distributed storage, covering storage fundamentals, Ceph architecture, advantages, version lifecycle, and step‑by‑step deployment using ceph‑deploy, including environment preparation, monitor and OSD setup, manager configuration, and dashboard activation.
CephDashboardLinux deployment
0 likes · 28 min read

The article analyzes the challenges of deploying large language models on cloud servers—such as latency, security, and constant connectivity—and explains how near‑memory computing architectures (PNM, PIM, CIM) can integrate storage and processing to enable efficient, high‑performance edge AI deployments, outlining the trade‑offs of each approach.
artificial intelligenceedge AIindustry insights
0 likes · 5 min read

This article compares 3FS, vePFS, and CloudFS across metadata and data planes, presents detailed benchmark results for AI training scenarios, analyzes architectural trade‑offs, and draws insights for future cloud‑native file storage development.
AI Trainingcloud nativedistributed file systems
0 likes · 38 min read

Bilibili’s comment service, a critical component for user interaction, faces massive read‑write traffic that can overwhelm TiDB, so the team built a multi‑level storage architecture using Redis sorted‑sets for indexes and a custom Taishan KV store, adding automatic degradation, consistency mechanisms, and hedging policies to ensure high availability and performance.
Data ConsistencyFailovercomment system
0 likes · 12 min read

This article presents a comprehensive design of Bilibili's comment service architecture, detailing the transition from TiDB to a multi‑level storage system based on Taishan KV, the data models, consistency mechanisms, retry and versioning strategies, and a hedging‑based degradation policy to ensure high availability under heavy traffic.
Taishan KVTiDBcomment system
0 likes · 13 min read

The paper proposes a multi‑level storage architecture for Bilibili’s comment service that replaces TiDB with a custom KV store (Taishan) and Redis caching, introduces unstructured indexes, CAS‑based consistency, real‑time and offline reconciliation, and a hedged degradation strategy to boost reliability, read throughput, and scalability during traffic spikes.
Data ConsistencyNoSQLTiDB
0 likes · 13 min read

Baidu’s Canghai Storage unifies metadata, hierarchical namespace, and data layers into a Meta‑Aware, three‑generation architecture that scales to trillions of metadata items and zettabyte‑scale data, using a distributed transactional KV store, single‑machine‑distributed namespace, and online erasure‑coding micro‑services to deliver high performance, low cost, and seamless scalability.
Erasure CodingNewSQLbig data
0 likes · 18 min read

This article analyzes the technical roadmap for upgrading AI super‑large GPU clusters to support trillion‑parameter multimodal models, covering single‑chip performance, super‑node scaling, DPU‑based compute fusion, energy‑efficient designs, converged storage, high‑throughput networking, and fault‑tolerant checkpoint strategies.
AI computeDPUGPU clusters
0 likes · 18 min read

RocketMQ stores messages in a sequential CommitLog while using a ConsumerQueue index for fast random access, and employs two looping threads—RebalanceService for load‑balancing queue assignments and PullMessageService for continuously pulling messages and dispatching them to user callbacks via a thread‑pooled consumer pipeline.
ConsumerJavaMessage Queue
0 likes · 12 min read

This article thoroughly examines Kafka's storage architecture, analyzing its design motivations, storage mechanisms, log formats, partitioning, indexing, cleanup strategies, and performance optimizations such as sequential log writes, sparse indexing, page cache, and zero‑copy, providing insights applicable to other storage systems.
KafkaLog SegmentsSparse Index
0 likes · 22 min read

This article dissects Pulsar’s storage layer by examining the Bookie component, detailing its architecture, the sequential write‑ahead log, ledger management, journal handling, caching mechanisms, and the complete read/write call chains with concrete code examples and performance considerations.
JavaLedgerPulsar
0 likes · 33 min read

This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.
KafkaMessage QueuePulsar
0 likes · 17 min read

In his 2023 keynote, Baidu Vice President Xie Guangjun unveiled an AI‑native cloud architecture featuring 7th‑gen G7 servers, Kunlun R300 and Ascend 910B instances, a gateway with X86, programmable switches and FPGA, unified storage (TafDB, Aries, BOS, CDS, PFS), Baige 3.0 AI infrastructure, an intelligent computing network, GaiaDB 4.0, BMR Spark 3.2, SugarBot natural‑language analytics, distributed edge and private cloud, and video and low‑code platforms.
Baidu CloudDatabase InnovationIntelligent Computing
0 likes · 19 min read

Software-defined storage (SDS) combines distributed and virtualization technologies to decouple storage software from hardware, offering flexible scaling, higher utilization, and a market dominated by a few Chinese vendors, while outlining its architecture, benefits, and competitive landscape.
SDSSoftware-Defined Storageindustry insights
0 likes · 6 min read

This article presents Baidu Cloud's block storage architecture, comparing replication and erasure‑coding fault‑tolerance methods, detailing the challenges of applying EC to mutable block data, and describing a two‑layer append‑engine solution with selective 3‑replica caching, cost‑benefit compaction, and performance optimizations for low‑cost, high‑throughput storage.
Erasure Codingappend enginebig data
0 likes · 14 min read

This article explains the three core benefits of message queues, compares major MQ products, details RocketMQ's advantages, internal models, reliability mechanisms, load‑balancing, storage design, and practical code examples, guiding engineers to make informed architectural decisions.
Message QueueRocketMQTransactional Messaging
0 likes · 41 min read

This article examines the evolution of big‑data storage from on‑premise HDFS to cloud‑native object storage, compares their architectures and performance, outlines future lakehouse storage requirements, and demonstrates a practical implementation using the JuiceFS distributed file system.
HDFSJuiceFSObject Storage
0 likes · 15 min read

This article provides a comprehensive guide to planning, structuring, securing, and managing a data lake on Azure Data Lake Storage Gen2, covering zone architecture, folder hierarchy, access control, file formats, scalability considerations, and best‑practice recommendations for big‑data workloads.
ADLS Gen2Azurebig data
0 likes · 21 min read

This article introduces RDF and property graph models, explains the core goals of graph database storage, compares mainstream storage solutions such as array, linked‑list and LSM‑Tree approaches, and presents practical deployment experiences of the Galaxybase distributed graph database.
GalaxybaseRDFdistributed-systems
0 likes · 23 min read

This article explains RocketMQ's broker storage architecture, detailing the roles of CommitLog, ConsumeQueue, and IndexFile, and describes how messages are formatted, indexed, and written to disk using memory‑mapped files to achieve high throughput and reliability.
CommitLogConsumeQueueIndexFile
0 likes · 8 min read


ITPUBOct 26, 2022 · Big Data
This article provides an in‑depth technical analysis of Kafka’s storage architecture, covering its design goals, storage mechanisms, log segment layout, sparse indexing, log cleanup policies, and the performance techniques such as sequential writes, page cache, and zero‑copy that enable high‑throughput streaming.
Log CleanupLog SegmentsSparse Index
0 likes · 22 min read

The article explains core flash storage techniques—including global load balancing, multi‑stream partitioning, and end‑to‑end I/O priority—by describing Active‑Active architectures, hash‑based request distribution, cache mirroring, data classification, and latency management to improve performance, reliability, and SSD lifespan.
I/O PriorityMulti‑Stream PartitionSSD
0 likes · 9 min read

The article explains the fully interconnected architecture of an all‑flash storage system, covering redundant FRU modules, RDMA‑based high‑speed networking, intelligent disk enclosures, SSD structure, wear‑leveling, bad‑block management, data redundancy, and the differences between SAS and NVMe protocols.
All-flash storageNVMeRDMA
0 likes · 12 min read

This article introduces graph database fundamentals, explains why graph databases are needed, outlines core storage goals such as index‑free adjacency, compares array, linked‑list and LSM‑tree storage schemes, and presents the design, performance advantages, and real‑world applications of the Galaxybase distributed graph database.
Galaxybasebig datadistributed-systems
0 likes · 20 min read

This article analyzes the storage challenges of high‑performance computing—including traditional HPC, AI‑driven HPC, and high‑performance data analysis—examines Baidu’s internal practices, and presents the Canghai storage platform with its object storage, parallel file system (PFS) and RapidFS solutions that address throughput, latency, and scalability requirements.
AI TrainingHigh-Performance Computingcloud storage
0 likes · 31 min read

The talk outlines IoT data classifications, requirements, and proposes a one‑stop storage product using multi‑model support, columnar formats, compute‑storage separation, tiered storage, and query optimization to achieve ten‑fold cost reduction and ten‑fold performance gains.
IoTcloud nativecolumnar storage
0 likes · 20 min read

FastDFS is a lightweight, open‑source distributed file system written in C that uses a three‑component architecture—client, tracker server for load‑balancing and discovery, and storage servers with push‑based binlog replication—to handle high‑concurrency upload/download of small to medium files, support group‑wide synchronization, optional trunk storage, Nginx anti‑leech integration, and extensible deduplication via FastDHT.
FastDFSSynchronizationUpload Download
0 likes · 15 min read

This article explains RocketMQ’s disk‑based storage design—including CommitLog, ConsumeQueue, and Index files—detailing sequential writes, memory‑mapped I/O, flexible flush strategies, and memory‑level read/write separation that together enable unlimited backlog, high throughput, and low latency messaging.
Memory Mapped FilesRocketMQSequential Write
0 likes · 13 min read

This article provides a comprehensive technical analysis of Apache RocketMQ's storage subsystem, detailing the architecture of CommitLog, ConsumeQueue, and IndexFile, the use of PageCache and mmap, the message write path, flush strategies, and read mechanisms, complete with code excerpts and diagrams.
CommitLogConsumeQueueIndexFile
0 likes · 25 min read

This article examines Android 6.0’s storage architecture, detailing the roles and interactions of MountService, Vold, and the kernel, illustrating module and process structures, communication flows via sockets, and highlighting performance considerations such as thread blocking and ANR issues.
AndroidMountServiceProcess Communication
0 likes · 14 min read


ITPUBSep 9, 2021 · Big Data
This article explains the origins and purpose of data lakes, outlines four key construction goals, details common ingestion methods and storage technologies, and describes essential governance practices such as cataloging, data quality, and regulatory compliance.
Data IngestionETLdata governance
0 likes · 18 min read

This article presents a comprehensive design of a cloud‑native ClickHouse OLAP system, detailing its three‑layer architecture, storage‑compute separation, unified metadata management, high‑availability mechanisms, elastic scaling, cost reductions, and future enhancements for multi‑replica and MPP query support.
ClickHouseElastic ScalingOLAP
0 likes · 19 min read


21CTOSep 1, 2021 · Databases
This article examines the evolving data‑storage demands of large‑scale commercial advertising systems, traces the development of BaikalDB from early MySQL sharding to a heterogeneous, cloud‑native distributed database, and explains its storage, compute, and scheduling designs that deliver high reliability, low cost, and millisecond‑level performance.
Advertising SystemsBaikalDBSQL Compatibility
0 likes · 27 min read

This article examines the requirements of commercial advertising systems for data storage, traces the evolution from single‑node MySQL to BaikalDB’s cloud‑native, MySQL‑compatible distributed architecture, and details its storage, compute, and scheduling designs, highlighting key features such as Raft replication, RocksDB storage, and hybrid OLTP/OLAP support.
BaikalDBRocksDBSQL
0 likes · 25 min read

This article analyzes the evolution of Baidu's advertising data storage, detailing the business-driven requirements, the design and development of the BaikalDB distributed database, its architecture across storage, compute and scheduling layers, key features such as Raft replication and multi‑index support, and the lessons learned for building cloud‑native, high‑performance databases.
BaikalDBDistributed DatabaseRaft replication
0 likes · 27 min read

This article explores the evolution and current landscape of time‑series databases, detailing design principles, storage structures such as B‑Tree, B+Tree, and LSM‑Tree, and providing an in‑depth comparison of OpenTSDB, InfluxDB, and the emerging Apache IoTDB, while also discussing practical deployment considerations and industry use cases.
Apache IoTDBB+TreeInfluxDB
0 likes · 38 min read

This article examines the core concepts, requirements, architectural models, persistence strategies, scalability, high‑availability mechanisms, performance optimizations, security models, and practical considerations of distributed file systems such as HDFS, GFS, and Ceph, offering a comprehensive guide for engineers and researchers.
Data ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read

This article explains the evolution of message‑queue middleware by comparing the basic double‑ended queue implementation, Redis list usage, Kafka’s partitioned log architecture with segment files and sparse indexes, and Pulsar’s compute‑storage separation using BookKeeper, highlighting their designs, strengths, and trade‑offs.
KafkaMessage QueuePulsar
0 likes · 28 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without central nodes, persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and deduplication.
ConsistencyHigh AvailabilitySecurity
0 likes · 21 min read

This article introduces Intel® Optane™ Persistent Memory (PMem), explains its performance characteristics, usage modes, the SNIA NVM programming model, and details the PMDK libraries and related tools that enable developers to build high‑performance, fault‑tolerant applications on persistent memory.
Intel OptaneNVM ProgrammingPMDK
0 likes · 16 min read

This article explores the core concepts, design requirements, architectural models, scalability, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized approaches while highlighting practical solutions for persistence, consistency, and fault tolerance.
Consistencydata replicationdistributed file system
0 likes · 21 min read

This article explains the types, components, operation principles, data layout, read/write process, locality-based prefetching, fragmentation, capacity calculations, and RAID configurations of modern hard disk drives, providing a comprehensive technical overview for anyone interested in storage fundamentals.
RAIDdisk I/Ohard disk
0 likes · 21 min read

This article compares the fundamentals and limitations of Redis list‑based queues with Kafka and Pulsar, explaining how partitions, cursors, replication, segment storage, and consumption models overcome Redis’s drawbacks and enable high‑performance, scalable, and highly available messaging systems.
KafkaMessage QueuePulsar
0 likes · 30 min read

The article explains the concept and advantages of data lakes, outlines the major storage and acceleration challenges they face, provides a checklist for ideal data‑lake solutions, and details how Alibaba Cloud's JindoFS addresses those challenges with object‑storage‑based, high‑performance, scalable features.
Alibaba CloudHadoopJindoFS
0 likes · 9 min read

This article provides a comprehensive overview of Red Hat Ceph’s distributed object‑storage architecture, covering storage pools, authentication, placement groups, the CRUSH algorithm, replication, erasure coding, internal operations, high‑availability mechanisms, client interfaces, and encryption, illustrated with diagrams and practical details.
CRUSHCephErasure Coding
0 likes · 40 min read

The article provides a comprehensive overview of the Lustre parallel file system architecture, its core components, POSIX compliance, scalability, high‑performance networking, security features, data layout mechanisms, and performance considerations for large and small files, along with practical optimization tips for HPC environments.
HPCLustrePOSIX
0 likes · 17 min read

This article provides a comprehensive overview of IBM's General Parallel File System (GPFS), detailing its historical development, architectural models—including SAN, NSD, and Share‑Nothing Cluster—its operational capabilities, performance advantages, scalability, high‑availability features, and its role in large‑scale high‑performance computing environments.
GPFSHigh-Performance ComputingIBM Spectrum Scale
0 likes · 12 min read

WeChat Pay’s Hermes‑based log system ingests trillions of entries daily, storing petabytes across a 200‑node HDFS cluster with four‑nine availability, while LSM‑style writes, separate inverted indexes and hot‑cold tiering cut memory, disk and cost by up to 70 % and keep 95 % of queries under five seconds.
HDFSHermesLog Analytics
0 likes · 7 min read

Data deduplication eliminates redundant data blocks to lower storage and bandwidth costs, while source‑ and transmission‑level encryption safeguards data in transit and at rest; the article also compares hardware vs software deduplication, various storage architectures (DAS, SAN, NAS, object and distributed storage) and their trade‑offs.
NASSANbackup
0 likes · 15 min read

This article presents a comprehensive design and deployment plan for a bank's unstructured data service platform, covering data growth challenges, lifecycle management, three‑tier storage architecture, Elasticsearch indexing, fault‑tolerant disaster recovery, monitoring, and future development directions.
Elasticsearchdisaster recoverystorage architecture
0 likes · 19 min read

This article examines the evolution, core requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional design trade‑offs of distributed file systems, providing a comprehensive overview for architects and engineers.
ConsistencySecuritydistributed file system
0 likes · 19 min read

The article thoroughly examines the Linux SCSI subsystem architecture, illustrating how abstract hardware topology—hosts, channels, IDs, and LUNs—is modeled in software through structs such as scsi_host and scsi_device, and detailing initialization, scanning, path establishment, power management, and block‑layer integration to guide kernel storage driver design.
Block LayerLinux kernelSCSI subsystem
0 likes · 20 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential design requirements, centralized and decentralized architecture models, persistence, scalability, high availability, performance optimization, security, and additional practical aspects such as space allocation, file deletion, small‑file handling, and deduplication.
Data ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and compares centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures, along with strategies for high availability, performance optimization, and handling small files.
ConsistencySecuritydistributed file system
0 likes · 21 min read

This guide explains how to plan Ceph hardware by balancing CPU, memory, storage, network bandwidth, and failure domains, offering practical recommendations for daemons, OSDs, monitors, managers, and SSD vs HDD choices to achieve cost‑effective, high‑performance large‑scale clusters.
CephHardware Planningcluster performance
0 likes · 15 min read

This article explains the fundamental architecture of SSDs, covering their NAND‑flash composition, logical sectors, page‑level operations, flash cell types (SLC, MLC, TLC), and how data is distributed across multiple dies to achieve performance and capacity.
Flash MemoryMLCNAND
0 likes · 7 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without a central node, persistence strategies, scalability, high availability, performance optimizations, security mechanisms, and practical considerations for small‑file workloads.
ConsistencySecuritydistributed file system
0 likes · 20 min read

eMMC is an embedded flash storage standard defined by JEDEC that integrates NAND flash, a controller with FTL, and an 8‑bit parallel interface, offering small size, low power, and high capacity, with defined working modes, six configuration registers, and performance advantages over raw NAND, MMC, and UFS.
FTLJEDECNAND flash
0 likes · 9 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability and security, architectural models with and without a central node, replication strategies, load balancing, high availability, performance optimizations, and practical considerations for small‑file handling.
ConsistencySecuritydistributed file system
0 likes · 20 min read

This article explains how UCloud's performance‑oriented UFS leverages NVMe SSDs, upgrades to NFSv4, redesigns business indexing, and introduces a novel append‑only storage engine with stream/extent architecture to achieve sub‑10 ms latency and high IOPS for demanding AI and analytics workloads.
NFSv4NVMe SSDappend‑only storage
0 likes · 17 min read

This article provides a comprehensive overview of HDFS, covering its purpose, architecture, read/write processes, component roles, command-line tools, replica placement strategies, and the advantages and disadvantages of using Hadoop's distributed file system for large-scale data storage.
HDFSHadoopdata replication
0 likes · 11 min read

The article explains the MFS distributed file system, detailing its four components—Master, Metalogger, Chunkserver, and Client—along with hardware recommendations, metadata handling, replication strategies, and FUSE‑based client mounting, providing a comprehensive guide to building a GoogleFS‑like storage cluster.
MFSbig datachunkserver
0 likes · 5 min read

This article provides a comprehensive technical overview of IBM GPFS (General Parallel File System), detailing its core components, cluster management roles, networking models, and best‑practice building‑block configurations for high‑performance computing environments.
Building BlockGPFSHPC
0 likes · 14 min read

This article explains the NVMe (Non‑Volatile Memory Express) standard, its logical device interface, key attributes, queue architecture, namespace concepts, multi‑path I/O, SR‑IOV support, and how it compares to traditional SCSI storage, providing a comprehensive technical overview for modern data‑center and client systems.
IO QueuesNVMePCIe
0 likes · 10 min read

Alibaba introduces Open Channel SSD and Fusion Engine, a unified storage architecture that addresses modern data‑center challenges by enhancing performance, flexibility, cost efficiency, and determinism through open hardware interfaces and advanced user‑space storage engines.
Alibaba CloudFusion EngineOpen Channel
0 likes · 5 min read

This article explains the fundamentals of NVMe‑oF, the role of InfiniBand components such as HCAs, switches, subnet managers and gateways, and why NetApp chose InfiniBand for its EF570/E5700 storage systems, highlighting performance benefits and protocol coexistence.
InfiniBandNVMe-oFRDMA
0 likes · 9 min read

The explosive growth of big‑data applications is forcing storage vendors to redesign architectures for petabyte‑scale capacity, real‑time latency, high IOPS, security compliance, and cost efficiency, while also addressing flexibility, data longevity, and the needs of both large and small users.
Cost OptimizationIOPSObject Storage
0 likes · 9 min read

The article summarizes PureStorage’s Matt Kixmoeller’s blog, which critiques Dell EMC’s PowerMax as a rebranded, partially NVMe‑optimized VMAX, highlights Pure’s own NVMe‑centric strategy, points out performance and efficiency shortcomings, and discusses the broader impact of NVMe on enterprise storage design.
Dell EMCFlashArrayNVMe
0 likes · 7 min read

This article provides a comprehensive overview of IBM GPFS, covering its architecture, management components, networking models, cluster and storage design, as well as practical guidance on building‑block configurations for performance and capacity scaling in high‑performance computing environments.
Building BlockGPFSHPC
0 likes · 13 min read

With data volumes soaring to petabyte scales, the article explains why object storage is the only viable solution for massive storage needs, outlines procurement considerations, design principles, and operational challenges, and offers practical guidance for building, evaluating, and scaling PB‑level storage systems.
Object Storagebig datacloud computing
0 likes · 38 min read

Distributed file systems extend traditional local storage by partitioning data across multiple servers, using a master node for metadata and coordination, handling namespace, replication, load balancing, caching, and client interfaces, thereby overcoming file size, quantity, and concurrency constraints of ext3, reiserfs, and similar local filesystems.
CachingReplicationdistributed file system
0 likes · 15 min read

The article analyzes HPE’s purchase of Nimble Storage, detailing the company’s flash‑focused product portfolio, underlying CASL architecture, all‑flash AF series specifications, multicloud integration, and the broader market dynamics affecting traditional and startup storage vendors.
All-FlashFlash TechnologyHPE
0 likes · 7 min read

This article shares a series of Sage Weil’s Ceph video lectures that thoroughly explain Ceph’s architecture, design philosophy, key features, and the BlueStore storage engine, providing links to additional resources and background on Ceph’s development history for enthusiasts seeking an in‑depth understanding.
BlueStoreCephObject Store
0 likes · 4 min read

The article explains how traditional centralized backup indexes become performance bottlenecks as data grows, and details Simpana's two‑level distributed indexing architecture—primary and secondary indexes—showing how it enhances backup speed, reduces network load, and simplifies recovery across multi‑site environments.
Simpanabackupdata recovery
0 likes · 7 min read

This article provides a comprehensive overview of flash memory technologies, comparing NOR and NAND flash, explaining cell structures such as SLC, MLC, and TLC, detailing NAND flash architecture, SSD controller functions, wear‑leveling, over‑provisioning, and emerging trends like NVMe and 3D‑Flash.
Cell TypesFlash MemoryNAND
0 likes · 14 min read

This article provides an in‑depth technical overview of Meizu's cloud sync service, covering its business scenarios, the evolution of its core SyncML‑based protocols, modular architecture across multiple data centers, performance optimizations such as traffic reduction and connection handling, as well as high‑availability, security, and future SDK‑API extensions.
High AvailabilityProtocol Designcloud sync
0 likes · 25 min read

The article provides a comprehensive overview of IBM's General Parallel File System (GPFS), detailing its physical and logical architecture, key components such as NSD and quorum mechanisms, scalability, load balancing, and fault‑tolerance features for parallel and serial applications.
GPFSHigh Availabilityparallel file system
0 likes · 11 min read

This article describes the design, architecture, and key features of a Linux‑based distributed file system that emphasizes horizontal scalability, data consistency, fault tolerance, and efficient handling of both small and large files without traditional metadata services.
Data ConsistencySynchronizationdistributed file system
0 likes · 32 min read

Kafka persists messages on disk by organizing each topic into multiple partitions, which are further divided into segment files containing paired .index and .log files; this structure enables efficient storage, offset-based lookup, and fast retrieval of specific messages through binary search across segment indexes.
KafkaMessage Queuestorage architecture
0 likes · 5 min read


21CTONov 13, 2015 · Operations
Taobao’s File System (TFS) is a highly available, high‑performance distributed storage solution built on Linux servers, featuring name‑server and data‑server clusters, block‑level replication, HA mechanisms, client caching, seamless scaling, multi‑data‑center disaster recovery, and open‑source support for C++, Java, and Nginx integration.
High AvailabilityTaobaodistributed file system
0 likes · 15 min read

The article explains FastDFS, an open‑source lightweight distributed file system built from tracker servers, storage servers, and client libraries, detailing its component roles, file upload/download workflows, storage organization, small‑file merging, HTTP support, and current limitations such as data safety and load balancing.
FastDFSdistributed file systemstorage architecture
0 likes · 16 min read
