Tagged articles
distributed file system
94 articles · Page 1 of 1
The article analyzes the limitations of traditional NAS for AI agents—scale ceiling, isolation gaps, and security blind spots—and explains how AgenticFS redesigns the storage paradigm by giving each agent an independent, zero‑trust workspace that scales linearly across regions.
AI storageAgenticFScloud native storage
0 likes · 13 min read

The article details how the WFS Ultra project re‑engineered a traditional TCP‑based distributed file system with a Run‑To‑Completion thread model, ultra‑core async networking, full‑link zero‑copy, and load‑adaptive prefetch, enabling 200 Gbps Fio throughput that exceeds the RDMA‑accelerated 3FS benchmark.
Run-to-CompletionTCP performanceWFS Ultra
0 likes · 10 min read

PoleFS is a high‑performance, cloud‑native distributed file system that combines NVMe‑accelerated hot storage with S3‑based cold storage, offering multiple client access methods, multi‑level metadata and data caches, prefetch/warm‑up strategies, and a Close‑to‑Open consistency model to balance performance and data correctness.
CachingConsistencyStorage
0 likes · 11 min read

This article recounts a critical NFS failure that caused massive loss, then walks through practical high‑availability designs—including Keepalived + DRBD, GlusterFS migration, and cloud‑native CSI storage—while sharing real‑world pitfalls, monitoring strategies, and forward‑looking recommendations for resilient file‑system operations.
High AvailabilityNFSStorage
0 likes · 12 min read

The article reveals that Git originated as a distributed file system with powerful content‑tracking capabilities, explains its underlying architecture, common misconceptions, performance pitfalls, and practical strategies—such as using Git LFS and treating Git as a state‑based system—to fully leverage its strengths beyond traditional version control.
Content TrackingDevOpsGit
0 likes · 9 min read

3FS is an open‑source distributed file system that abstracts multiple machines into a single namespace, offering massive scalability, fault tolerance, and high throughput through components like Meta, Mgmtd, Storage, and Client, and leveraging the CRAQ protocol for strong consistency and efficient reads and writes.
3FSCRAQdistributed file system
0 likes · 12 min read

This article provides a concise overview of four key distributed file systems—HDFS, CephFS, GFS, and TFS—explaining their architectures, strengths, weaknesses, and typical application scenarios for large‑scale data storage and processing.
CephFSGFSHDFS
0 likes · 5 min read

PoleFS is a self‑developed, cloud‑native distributed file system that combines POSIX‑compatible interfaces, a high‑performance NVMe cache layer, and low‑cost S3 object storage to achieve microsecond latency, millions of IOPS, massive directory scalability, multi‑protocol access, and flexible client‑side caching for AI, big data, and container workloads.
NVMe cacheS3 object storagecloud native storage
0 likes · 9 min read

This article explains InfiniFS's speculative path resolution, detailing how predictable directory IDs and parallel lookups transform traditional linear RPC-based path traversal into constant‑time operations, dramatically reducing metadata access latency in large, deep directory trees.
InfiniFSdistributed file systemmetadata service
0 likes · 8 min read

3FS, DeepSeek’s high‑performance parallel file system, is engineered for AI workloads, offering ultra‑low latency, high‑throughput storage via RDMA, CRAQ consistency, and seamless cloud‑native integration, with detailed architecture, deployment steps, performance benchmarks, and cost‑saving strategies for large‑scale model training and inference.
AI storageRDMAdistributed file system
0 likes · 28 min read

This article explains what a Distributed File System (DFS) is, outlines key evaluation criteria such as reliability, availability, performance, scalability, and then compares HDD and SSD performance, investigates whether DFS can surpass local NVMe in large‑IO workloads, and discusses user‑side, cluster‑level, and cache‑level performance assessment methods.
CachingCluster ScalingNVMe
0 likes · 14 min read

This guide explains what JuiceFS is, its cloud‑native architecture separating data and metadata, and provides step‑by‑step instructions—including prerequisites, client installation, formatting, mounting, and verification—to help you deploy the high‑performance distributed file system on object storage and PostgreSQL.
JuiceFSObject Storagecloud-native
0 likes · 7 min read

This article dissects DeepSeek's 3FS parallel file system, detailing its four‑component architecture, high‑throughput RDMA networking, metadata handling with FoundationDB, client access methods, chain replication (CRAQ), custom FFRecord format, and recovery mechanisms, offering a deep technical perspective for storage engineers.
High-performance storageRDMAchain replication
0 likes · 22 min read

This article dives deep into DeepSeek's 3FS parallel file system, detailing its four-component architecture, RDMA‑based high‑speed networking, client options, metadata and storage services, replication protocols, dynamic stripe sizing, and recovery mechanisms that enable efficient AI model training and inference.
AI TrainingRDMAdistributed file system
0 likes · 21 min read

The article provides a comprehensive English overview of 3FS, an AI‑focused distributed file system that leverages FoundationDB for metadata, CRAQ for chunk replication, and a hybrid Fuse/native client architecture, detailing its design, components, fault handling, and performance considerations for large‑scale training workloads.
AI storageCRAQ replicationFoundationDB
0 likes · 25 min read

This article explains how HDFS, the Hadoop Distributed File System, splits large files into blocks, replicates them for fault tolerance, organizes the cluster into NameNode and DataNode components, and provides high‑availability and scalability mechanisms such as standby NameNode and federation, enabling reliable big‑data storage and access.
DataNodeHDFSNameNode
0 likes · 11 min read

This article explains the origins of big‑data technologies, details the architecture and read/write mechanisms of Hadoop's HDFS, describes the MapReduce programming model, and provides complete Java code examples for a simple distributed file‑processing job using Maven dependencies.
HDFSHadoopJava
0 likes · 15 min read

This article details JD's large‑scale HDFS unified storage implementation, covering cross‑region storage challenges, topology design, asynchronous block replication, flow‑control mechanisms, tiered storage strategies, automatic hot‑cold data migration, and the resulting performance and cost improvements for big‑data workloads.
Cross-Region StorageData ManagementHDFS
0 likes · 20 min read

This article presents JD Retail's large‑scale HDFS deployment, detailing its unified storage architecture, cross‑region data replication challenges and solutions, tiered storage strategies for hot, warm and cold data, and the operational modules that together improve performance, reliability and cost efficiency in a big‑data environment.
Cross-Region StorageHDFSbig data
0 likes · 21 min read

This article walks through SeaweedFS 3.46’s source code, showing how to launch Master, Volume, and Filer servers, how Raft leader election is wired, and the exact request‑response flow for uploading and downloading files, complete with command‑line examples and code snippets.
File DownloadFile UploadFiler Server
0 likes · 24 min read

Facing petabyte‑level data, billions of small files, and the need for POSIX, S3, and HDFS compatibility, Didi designed a new generation of non‑structured storage—OrangeFS—by analyzing internal systems, combining multiple storage solutions, reusing GIFT technology, and implementing a high‑performance metadata service, multi‑protocol fusion, and robust scalability features.
AI storageMulti-Protocolbig data
0 likes · 27 min read

The article outlines the challenges of massive AI training data, defines storage performance requirements, and presents Polefs—a cloud‑native distributed cache file system with unified storage, metadata acceleration, and read/write caching designed to improve GPU utilization and reduce data redundancy.
AIGPU utilizationPoleFS
0 likes · 14 min read

This article presents JuiceFS, a cloud‑native distributed file system that bridges the gaps between HDFS and object storage, explaining Data Lake and Lakehouse concepts, comparing storage options, detailing JuiceFS's architecture and performance benefits, and showcasing real‑world user case studies.
JuiceFSObject Storagebig data
0 likes · 23 min read

This tutorial walks you through installing FastDFS, configuring its tracker and storage nodes, integrating it with Nginx, and exposing the service to the internet using cpolar, enabling remote file upload and download over a LAN and public network.
FastDFSLinuxcpolar
0 likes · 13 min read

This step‑by‑step guide shows how to install FastDFS on a Linux server, configure its tracker and storage nodes, integrate it with Nginx, and expose the file service to the Internet using the cpolar tunneling tool, all with detailed commands and configuration files.
FastDFSLinuxNGINX
0 likes · 20 min read

OrangeFS is Didi’s cloud‑native, multi‑protocol distributed data‑lake storage system that unifies POSIX, S3 and HDFS access on a single logical hierarchy, integrates with Kubernetes via a CSI plugin, supports on‑premise and public‑cloud backends, provides multi‑tenant isolation, and dramatically improves elasticity, utilization and latency for petabyte‑scale workloads such as ride‑hailing logs, machine‑learning training, finance and analytics.
CSIFUSEKubernetes
0 likes · 17 min read

The talk from DataFun Summit 2023 explains how Baidu's CFS storage builds a trillion‑file‑scale distributed file system by revisiting file system fundamentals, POSIX limitations, historical storage architectures, and introducing a lock‑free metadata service with single‑shard primitives, data‑layout optimizations, and a simplified client‑centric architecture that achieves high scalability and performance.
CFSPOSIXcloud storage
0 likes · 31 min read

This article examines Alluxio's open-source distributed file system, detailing the core types of metadata—inode, block, mount table, and worker—along with the mechanisms for their storage, management, and optimization in both HEAP and ROCKS modes, and provides practical configuration guidance for scaling large-scale data environments.
AlluxioHEAP modeROCKS mode
0 likes · 15 min read

This article explains Baidu's CFS architecture for building a billion‑file‑scale distributed file system, covering basic file system concepts, POSIX limitations, metadata service modeling, performance metrics, evolution of metadata architectures, and CFS's lock‑free design that achieves high scalability, low latency, and balanced load in cloud storage.
cloud storagedistributed file systemlock-free design
0 likes · 32 min read

ByteFUSE, a user‑space FUSE‑based solution for ByteNAS, delivers low‑latency, high‑throughput, POSIX‑compatible storage across AI training, database backup, and search services by replacing NFS with a cloud‑native architecture that leverages CSI, RDMA, and kernel‑module hot‑upgrade techniques.
FUSEKubernetesRDMA
0 likes · 19 min read

Baidu’s CFS metadata service scales to billions of files by shrinking critical sections through a lock‑free Namespace 2.0 design that confines conflicts to single shards, uses field‑level atomic primitives, and integrates the proxy into the client, delivering up to 76× throughput gains and significant latency reductions in production.
Baidu CFSEuroSys 2023POSIX compatibility
0 likes · 40 min read

This article explains the design and evolution of Baidu Cloud File System's (CFS) metadata service, detailing how a novel lock‑free architecture and strategic data layout enable POSIX‑compatible, highly scalable storage that can handle billions of files while maintaining high performance and consistency.
cloud storagedistributed file systemlock-free
0 likes · 42 min read

The EuroSys 2023 paper on Baidu’s cloud‑native file storage CFS reveals a metadata service design that trims critical sections, achieving 1.76‑75.82× higher throughput and up to 91.71% lower latency compared to HopsFS and InfiniFS, and has been production‑stable for over three years.
EuroSyscloud native storagedistributed file system
0 likes · 6 min read

This article reviews the evolution of file storage, outlines the challenges of the cloud era, and details JuiceFS's design philosophy, architecture, key capabilities, and real‑world use cases such as Kubernetes, AI, big‑data analytics, and NAS migration to the cloud.
AIJuiceFSKubernetes
0 likes · 22 min read

This article reviews the evolution of file storage, outlines challenges of cloud‑native data management, and details JuiceFS’s cloud‑native design philosophy, architecture, and key use cases such as Kubernetes, AI, and big‑data workloads.
AIJuiceFSKubernetes
0 likes · 23 min read


ITPUBOct 20, 2022 · Big Data
The article examines why Hadoop's Distributed File System may become obsolete by detailing its three main shortcomings—deployment complexity, metadata memory limits, and high replication overhead—and explores how newer architectures and erasure coding could address these issues.
Erasure CodingHDFSbig data
0 likes · 8 min read

This article presents Ctrip's experience migrating over 2 PB of cold data to JuiceFS, detailing the system's architecture, metadata engine selection, extensive performance testing, fault‑tolerance analysis, and operational optimizations that reduced storage and maintenance costs while supporting future petabyte‑scale workloads.
Cold Data StorageJuiceFSTiKV
0 likes · 15 min read

This article introduces an open‑source distributed file system built with Spring Boot and Vue CLI, detailing its MIT licensing, UI layout, file operations, multiple upload methods, online preview and editing capabilities, storage options, and the underlying backend and frontend technologies.
Vue.jsdistributed file systemfile-management
0 likes · 9 min read

This presentation provides a comprehensive overview of JuiceFS, an open‑source cloud‑native distributed file system, detailing its role in modern data lake and lakehouse architectures, comparing it with HDFS and object storage, and highlighting its performance, integration, and community ecosystem.
JuiceFSObject Storagebig data
0 likes · 19 min read

FastDFS is a lightweight, open‑source distributed file system written in C that uses a three‑component architecture—client, tracker server for load‑balancing and discovery, and storage servers with push‑based binlog replication—to handle high‑concurrency upload/download of small to medium files, support group‑wide synchronization, optional trunk storage, Nginx anti‑leech integration, and extensible deduplication via FastDHT.
FastDFSSynchronizationUpload Download
0 likes · 15 min read

Bilibili’s HDFS now runs a three‑tier architecture—access, metadata, and data layers—enhanced with a custom MergeFS router, observer NameNode, dynamic load balancing, fast‑failover pipelines, and storage‑aware policies, while future work targets transparent erasure coding, tiered data routing, lock refinements, and a Hadoop 3.x migration.
HDFSbig datadistributed file system
0 likes · 22 min read

This article provides a comprehensive overview of HDFS, covering basic file system concepts, HDFS architecture, high availability, federation, replica placement, storage policies, colocation, data integrity, and key design considerations for large‑scale distributed storage.
HDFSHigh AvailabilityStorage Policies
0 likes · 23 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and comparing centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures along with strategies for high availability, performance optimization, and data consistency.
Consistencydistributed file systemmetadata
0 likes · 19 min read

This article examines the core concepts, requirements, architectural models, persistence strategies, scalability, high‑availability mechanisms, performance optimizations, security models, and practical considerations of distributed file systems such as HDFS, GFS, and Ceph, offering a comprehensive guide for engineers and researchers.
Data ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without central nodes, persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and deduplication.
ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read

This article explores the core concepts, design requirements, architectural models, scalability, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized approaches while highlighting practical solutions for persistence, consistency, and fault tolerance.
Consistencydata replicationdistributed file system
0 likes · 21 min read

This article presents JuiceFS, an open‑source cloud‑native distributed file system that addresses the limitations of object storage for big‑data and AI workloads by providing strong consistency, high‑performance metadata, multi‑protocol support, small‑file management, and deep Kubernetes integration.
artificial intelligencecloud nativedistributed file system
0 likes · 13 min read

This article provides a comprehensive overview of IBM's General Parallel File System (GPFS), detailing its historical development, architectural models—including SAN, NSD, and Share‑Nothing Cluster—its operational capabilities, performance advantages, scalability, high‑availability features, and its role in large‑scale high‑performance computing environments.
GPFSHigh-Performance ComputingIBM Spectrum Scale
0 likes · 12 min read

This article examines the evolution, core requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional design trade‑offs of distributed file systems, providing a comprehensive overview for architects and engineers.
Consistencydistributed file systemscalability
0 likes · 19 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential design requirements, centralized and decentralized architecture models, persistence, scalability, high availability, performance optimization, security, and additional practical aspects such as space allocation, file deletion, small‑file handling, and deduplication.
Data ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read

This article explains the principles behind NFS file‑lock state view consistency, covering NFSv4's stateful design, the SeqId mechanism, SunRPC handling of duplicate requests, and how signal interruptions are managed to keep client and server lock views synchronized.
ConsistencyFile LockNFS
0 likes · 10 min read

This article examines the core concepts, architectural models, scalability, persistence, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized designs such as GFS and Ceph to guide future system design decisions.
ConsistencyHigh Availabilityarchitecture
0 likes · 21 min read

FastDFS is an open‑source lightweight distributed file system that tackles large‑scale file storage and load‑balancing by using dedicated Tracker, Storage, and Client roles, offering high performance, easy horizontal scaling, and clear upload/download workflows for web applications.
FastDFSbackend architecturedistributed file system
0 likes · 9 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and compares centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures, along with strategies for high availability, performance optimization, and handling small files.
Consistencydistributed file systemperformance optimization
0 likes · 21 min read

This article explains the fundamentals of FastDFS, its tracker and storage architecture, the file upload/download workflow, and provides a step‑by‑step Maven‑based Java example—including configuration, dependency setup, and code—to demonstrate how to store and retrieve files in a FastDFS cluster.
BackendFastDFSJava
0 likes · 7 min read

During Kuaishou's 2020 Spring Festival red‑envelope campaign, Tencent Cloud's Cloud File Storage (CFS) provided a high‑throughput, read‑heavy NFS solution that handled up to 5 GB/s across 1 700 pods, delivering 100% availability, low latency, and massive interaction volumes for billions of users.
CFSKuaishouNFS
0 likes · 22 min read

This article provides a comprehensive overview of Hadoop Distributed File System (HDFS), covering its streaming data access model, key characteristics, master‑slave architecture, block storage and replication mechanisms, rack‑aware placement strategy, and how the NameNode manages metadata and checkpoints.
HDFSHadoopRack Awareness
0 likes · 7 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without a central node, persistence strategies, scalability, high availability, performance optimizations, security mechanisms, and practical considerations for small‑file workloads.
Consistencydistributed file systemperformance optimization
0 likes · 20 min read

This article provides a comprehensive overview of HDFS, covering its role as a distributed file system, the concepts of blocks, NameNode and DataNode responsibilities, replication, edit logs, snapshots, high‑availability mechanisms, and practical considerations for managing large‑scale data storage.
DataNodeHDFSNameNode
0 likes · 11 min read

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability and security, architectural models with and without a central node, replication strategies, load balancing, high availability, performance optimizations, and practical considerations for small‑file handling.
Consistencydistributed file systemscalability
0 likes · 20 min read

This article introduces the motivations for using a distributed file system, explains the architecture and core concepts of FastDFS—including tracker, storage, client, and group—covers its upload and download mechanisms, synchronization management, and the design of its file identifiers, providing a comprehensive overview for developers.
BackendFastDFSStorage
0 likes · 16 min read

This article provides a comprehensive overview of HDFS, covering its purpose, architecture, read/write mechanisms, replication strategies, component responsibilities, common command‑line tools, and the advantages and disadvantages of using Hadoop Distributed File System for large‑scale data storage.
Data StorageHDFSHadoop
0 likes · 10 min read

This article explains how UCloud's performance‑oriented UFS leverages NVMe SSDs, upgrades to NFSv4, redesigns business indexing, and introduces a novel append‑only storage engine with stream/extent architecture to achieve sub‑10 ms latency and high IOPS for demanding AI and analytics workloads.
NFSv4NVMe SSDappend‑only storage
0 likes · 17 min read

This article provides a comprehensive overview of HDFS, covering its purpose, architecture, read/write processes, component roles, command-line tools, replica placement strategies, and the advantages and disadvantages of using Hadoop's distributed file system for large-scale data storage.
HDFSHadoopdata replication
0 likes · 11 min read

The article explains the MFS distributed file system, detailing its four components—Master, Metalogger, Chunkserver, and Client—along with hardware recommendations, metadata handling, replication strategies, and FUSE‑based client mounting, providing a comprehensive guide to building a GoogleFS‑like storage cluster.
MFSbig datachunkserver
0 likes · 5 min read

This article provides a comprehensive technical overview of IBM GPFS (General Parallel File System), detailing its core components, cluster management roles, networking models, and best‑practice building‑block configurations for high‑performance computing environments.
Building BlockGPFSHPC
0 likes · 14 min read

This article explains the core concepts of HDFS—including its block, packet, and chunk structures, their roles in data streaming, the detailed write and read workflows, and how checksums ensure data integrity—providing a comprehensive overview for anyone working with Hadoop distributed storage.
HDFSblock storagedata integrity
0 likes · 7 min read

This article explains how HDFS handles file reading and writing, detailing the roles of DFSClient, block selection, hedged reads, packet construction, checksum handling, and the interaction with NameNode and DataNode pipelines to ensure reliability and performance.
DFSClientHDFSRead/Write
0 likes · 7 min read

The article provides a comprehensive overview of the Lustre file system, detailing its cluster storage architecture, core components, scalability, performance optimizations, security mechanisms, high‑availability features, and usage in high‑performance computing environments.
HPCLustrePOSIX
0 likes · 14 min read

PolarFS is a user‑space, ultra‑low‑latency distributed file system designed for POLARDB that leverages RDMA, NVMe SSDs, and a novel ParallelRaft protocol to deliver near‑local‑SSD performance, strong consistency, and seamless failover in a cloud‑native environment.
ParallelRaftPolarDBPolarFS
0 likes · 21 min read

FastDFS is an open‑source lightweight distributed file system written in C, designed for high‑availability file storage and load balancing; this article explains its differences from Hadoop, details its Tracker and Storage architecture, outlines file upload/download workflows, and provides step‑by‑step instructions for installing and configuring FastDFS, Nginx, and related components.
FastDFSNGINXdistributed file system
0 likes · 25 min read

This article provides a comprehensive overview of IBM GPFS, covering its architecture, management components, networking models, cluster and storage design, as well as practical guidance on building‑block configurations for performance and capacity scaling in high‑performance computing environments.
Building BlockGPFSHPC
0 likes · 13 min read

This article reviews the fundamental concepts of HDFS, explains its master‑slave architecture with NameNode and DataNode, describes block replication, and discusses various implementations—including native HDFS, NetApp/Lustre, GPFS/Ceph, and Isilon—as well as HDFS‑to‑NFS gateway integration.
HDFSHadoopNFS
0 likes · 7 min read

Distributed file systems extend traditional local storage by partitioning data across multiple servers, using a master node for metadata and coordination, handling namespace, replication, load balancing, caching, and client interfaces, thereby overcoming file size, quantity, and concurrency constraints of ext3, reiserfs, and similar local filesystems.
CachingReplicationdistributed file system
0 likes · 15 min read

The article describes how JD.com transitioned from open‑source tools to its own distributed file system JFS, completed a massive one‑month migration of billions of images, improved compression with Intel‑optimized pipelines and blind‑watermark technology, and outlines the ongoing scalability and security innovations.
Image storageblind_watermarkdistributed file system
0 likes · 5 min read

This comprehensive tutorial explains HDFS fundamentals, its metadata management and advantages, then walks you through setting up a Hadoop environment, executing core shell commands, and using the Java API with complete code examples, enabling you to confidently operate HDFS in practice.
HadoopJava APIShell Commands
0 likes · 15 min read

This article explains how distributed file systems overcome the limitations of traditional local file systems by using a master‑metadata server, multiple data nodes, and client interfaces, and it details the key architectural components, common problems, and practical engineering solutions such as replication, load balancing, and caching.
Replicationarchitecturedistributed file system
0 likes · 15 min read

This article provides a comprehensive overview of the Lustre parallel distributed file system, detailing its architecture, stripe configuration, I/O performance traits, challenges with small files, and practical optimization techniques for high‑performance computing environments.
HPCLarge FilesLustre
0 likes · 17 min read

Alluxio, formerly Tachyon, is a memory‑centric distributed file system that unifies heterogeneous big‑data storage backends, optimizes small files, and provides a fast, unified data access layer between storage systems like S3 or HDFS and compute frameworks such as Spark or Hadoop.
AlluxioCompute FrameworksData Storage
0 likes · 7 min read

This tutorial guides you through HDFS fundamentals, explaining its purpose and mechanisms, demonstrates command‑line and Java API operations, and walks you through the complete read/write workflow, while providing a ready‑to‑use practice environment for hands‑on learning.
HDFSHadoopJava API
0 likes · 2 min read

This article introduces the open‑source MogileFS distributed file system, explains its server, storage, and client components, outlines its key features and operating principles, and provides step‑by‑step installation, configuration, and Nginx reverse‑proxy load‑balancing instructions for large‑scale image storage.
Backend StorageMogileFSNginx Reverse Proxy
0 likes · 6 min read

This article explains how HDFS, inspired by Google’s GFS, provides a low‑cost, highly reliable, fault‑tolerant, and high‑performance distributed file system for big‑data workloads by using replication, standby NameNodes, block storage, rack awareness, and compute‑close‑to‑data strategies.
HDFSbig datadata replication
0 likes · 7 min read

This article details NetEase’s object storage (NOS) architecture, covering its use cases, core components such as DDB and DFS, rich media processing pipeline, upload acceleration techniques, and practical overload control measures for handling massive unstructured data workloads.
Object StorageUpload Accelerationcloud computing
0 likes · 14 min read

This article provides a comprehensive overview of Hadoop Distributed File System (HDFS), detailing its design goals, architecture components such as NameNode, DataNode and SecondaryNameNode, data block handling, replication strategies, communication protocols, and the read, write, and delete processes.
HDFSHadoopNameNode
0 likes · 18 min read

This article provides a comprehensive overview of GlusterFS, a scale‑out distributed file system, covering its modular architecture, elastic hash algorithm, high‑availability features, global namespace, supported protocols, cloud deployment considerations, and data flow from client to storage nodes.
GlusterFSScale-Out Storagedistributed file system
0 likes · 8 min read

The article explains how traditional SAN storage creates CPU‑I/O bottlenecks, how Google’s distributed file system inspired hyper‑converged designs that fuse compute and storage, and why tiered storage with SSD and HDD offers scalable, high‑performance infrastructure for modern data‑center workloads.
I/O performanceStoragedata center
0 likes · 11 min read


21CTOMar 26, 2016 · Fundamentals
This article describes the design and implementation of a Linux‑based distributed file system (DFS) that targets large‑scale data storage and access, detailing its architecture, goals, storage model, consistency mechanisms, thread‑safety approach, synchronization strategies, and recovery processes to ensure high availability and data integrity.
DFS designLinux storageSynchronization
0 likes · 35 min read

This article describes the design, architecture, and key features of a Linux‑based distributed file system that emphasizes horizontal scalability, data consistency, fault tolerance, and efficient handling of both small and large files without traditional metadata services.
Data ConsistencySynchronizationdistributed file system
0 likes · 32 min read


ITPUBMar 19, 2016 · Big Data
This article explains the fundamentals of distributed file systems, focusing on Hadoop’s HDFS architecture, the separation of metadata and data via NameNode and DataNode, and detailed step‑by‑step write and read processes, including replication, fault recovery, and block splitting across nodes.
DataNodeHDFSNameNode
0 likes · 8 min read

This article explains how HDFS abstracts multiple servers into a single file system, splits files into replicated blocks, manages metadata via NameNode and DataNode, and provides linear capacity scaling and high reliability for big data workloads.
HDFSbig datadata replication
0 likes · 5 min read


21CTONov 14, 2015 · Fundamentals
This article explains the design, storage mechanisms, high‑availability architecture, scaling strategies, multi‑data‑center disaster recovery, operational management, and future plans of Taobao’s distributed file system (TFS), a highly available and scalable storage solution for massive unstructured data.
High AvailabilityTFSTaobao
0 likes · 14 min read


21CTONov 13, 2015 · Operations
Taobao’s File System (TFS) is a highly available, high‑performance distributed storage solution built on Linux servers, featuring name‑server and data‑server clusters, block‑level replication, HA mechanisms, client caching, seamless scaling, multi‑data‑center disaster recovery, and open‑source support for C++, Java, and Nginx integration.
High AvailabilityTaobaodistributed file system
0 likes · 15 min read

The article explains FastDFS, an open‑source lightweight distributed file system built from tracker servers, storage servers, and client libraries, detailing its component roles, file upload/download workflows, storage organization, small‑file merging, HTTP support, and current limitations such as data safety and load balancing.
FastDFSdistributed file systemstorage architecture
0 likes · 16 min read

This step‑by‑step guide shows how to build a MogileFS distributed file system cluster on CentOS, configure MariaDB, set up trackers, storage nodes and domains, compile Nginx with the MogileFS module, create init scripts, and verify fault‑tolerant file access through Nginx reverse proxy.
LinuxMogileFSNGINX
0 likes · 15 min read
