BestHub
Sign in
Tagged articles
93 articles
Page 1 of 1
Tencent Technical Engineering
Tencent Technical Engineering
Feb 11, 2026 · Backend Development

How WFS Ultra Achieved 200 Gbps TCP Throughput, Surpassing RDMA‑Based 3FS

The article details how the WFS Ultra project re‑engineered a traditional TCP‑based distributed file system with a Run‑To‑Completion thread model, ultra‑core async networking, full‑link zero‑copy, and load‑adaptive prefetch, enabling 200 Gbps Fio throughput that exceeds the RDMA‑accelerated 3FS benchmark.

Distributed File SystemRun-to-CompletionTCP performance
0 likes · 10 min read
How WFS Ultra Achieved 200 Gbps TCP Throughput, Surpassing RDMA‑Based 3FS
360 Smart Cloud
360 Smart Cloud
Nov 25, 2025 · Cloud Native

How PoleFS Achieves Microsecond I/O with Multi‑Layer Caching and CTO Consistency

PoleFS is a high‑performance, cloud‑native distributed file system that combines NVMe‑accelerated hot storage with S3‑based cold storage, offering multiple client access methods, multi‑level metadata and data caches, prefetch/warm‑up strategies, and a Close‑to‑Open consistency model to balance performance and data correctness.

ConsistencyDistributed File Systemcaching
0 likes · 11 min read
How PoleFS Achieves Microsecond I/O with Multi‑Layer Caching and CTO Consistency
Ops Community
Ops Community
Sep 19, 2025 · Operations

From Midnight Outage to Zero Downtime: Mastering NFS High‑Availability

This article recounts a critical NFS failure that caused massive loss, then walks through practical high‑availability designs—including Keepalived + DRBD, GlusterFS migration, and cloud‑native CSI storage—while sharing real‑world pitfalls, monitoring strategies, and forward‑looking recommendations for resilient file‑system operations.

Distributed File SystemNFShigh availability
0 likes · 12 min read
From Midnight Outage to Zero Downtime: Mastering NFS High‑Availability
IT Services Circle
IT Services Circle
Sep 14, 2025 · Fundamentals

Why Git Is Really a Distributed File System, Not Just a VCS

The article reveals that Git originated as a distributed file system with powerful content‑tracking capabilities, explains its underlying architecture, common misconceptions, performance pitfalls, and practical strategies—such as using Git LFS and treating Git as a state‑based system—to fully leverage its strengths beyond traditional version control.

Content TrackingDevOpsDistributed File System
0 likes · 9 min read
Why Git Is Really a Distributed File System, Not Just a VCS
Efficient Ops
Efficient Ops
Sep 9, 2025 · Fundamentals

Inside 3FS: How Distributed File Systems Hide Complexity and Scale

3FS is an open‑source distributed file system that abstracts multiple machines into a single namespace, offering massive scalability, fault tolerance, and high throughput through components like Meta, Mgmtd, Storage, and Client, and leveraging the CRAQ protocol for strong consistency and efficient reads and writes.

3FSCRAQDistributed File System
0 likes · 12 min read
Inside 3FS: How Distributed File Systems Hide Complexity and Scale
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Sep 2, 2025 · Cloud Native

Unlock Microsecond I/O: Inside PoleFS’s NVMe‑Accelerated, S3‑Backed Cloud‑Native File System

PoleFS is a self‑developed, cloud‑native distributed file system that combines POSIX‑compatible interfaces, a high‑performance NVMe cache layer, and low‑cost S3 object storage to achieve microsecond latency, millions of IOPS, massive directory scalability, multi‑protocol access, and flexible client‑side caching for AI, big data, and container workloads.

Cloud Native StorageDistributed File SystemNVMe cache
0 likes · 9 min read
Unlock Microsecond I/O: Inside PoleFS’s NVMe‑Accelerated, S3‑Backed Cloud‑Native File System
Big Data Technology Tribe
Big Data Technology Tribe
Jul 28, 2025 · Fundamentals

How Speculative Path Resolution Cuts Metadata Latency in InfiniFS

This article explains InfiniFS's speculative path resolution, detailing how predictable directory IDs and parallel lookups transform traditional linear RPC-based path traversal into constant‑time operations, dramatically reducing metadata access latency in large, deep directory trees.

Distributed File SystemInfiniFSmetadata service
0 likes · 8 min read
How Speculative Path Resolution Cuts Metadata Latency in InfiniFS
Instant Consumer Technology Team
Instant Consumer Technology Team
Jun 30, 2025 · Operations

How 3FS Revolutionizes AI Storage with High‑Throughput Distributed Filesystem

3FS, DeepSeek’s high‑performance parallel file system, is engineered for AI workloads, offering ultra‑low latency, high‑throughput storage via RDMA, CRAQ consistency, and seamless cloud‑native integration, with detailed architecture, deployment steps, performance benchmarks, and cost‑saving strategies for large‑scale model training and inference.

AI storageDistributed File SystemHigh Throughput
0 likes · 28 min read
How 3FS Revolutionizes AI Storage with High‑Throughput Distributed Filesystem
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Jun 30, 2025 · Fundamentals

Can Distributed File Systems Outperform Local NVMe? A Deep Performance Evaluation

This article explains what a Distributed File System (DFS) is, outlines key evaluation criteria such as reliability, availability, performance, scalability, and then compares HDD and SSD performance, investigates whether DFS can surpass local NVMe in large‑IO workloads, and discusses user‑side, cluster‑level, and cache‑level performance assessment methods.

Distributed File SystemNVMePerformance Evaluation
0 likes · 14 min read
Can Distributed File Systems Outperform Local NVMe? A Deep Performance Evaluation
Linux Ops Smart Journey
Linux Ops Smart Journey
Jun 19, 2025 · Cloud Native

How to Deploy JuiceFS: A Cloud‑Native Distributed File System Tutorial

This guide explains what JuiceFS is, its cloud‑native architecture separating data and metadata, and provides step‑by‑step instructions—including prerequisites, client installation, formatting, mounting, and verification—to help you deploy the high‑performance distributed file system on object storage and PostgreSQL.

Distributed File SystemJuiceFScloud-native
0 likes · 7 min read
How to Deploy JuiceFS: A Cloud‑Native Distributed File System Tutorial
ByteDance Cloud Native
ByteDance Cloud Native
Mar 13, 2025 · Backend Development

Inside DeepSeek 3FS: Architecture of a High‑Performance Parallel File System

This article dissects DeepSeek's 3FS parallel file system, detailing its four‑component architecture, high‑throughput RDMA networking, metadata handling with FoundationDB, client access methods, chain replication (CRAQ), custom FFRecord format, and recovery mechanisms, offering a deep technical perspective for storage engineers.

Distributed File SystemHigh-performance storageRDMA
0 likes · 22 min read
Inside DeepSeek 3FS: Architecture of a High‑Performance Parallel File System
Volcano Engine Developer Services
Volcano Engine Developer Services
Mar 7, 2025 · Operations

Inside 3FS: How DeepSeek’s Parallel File System Powers AI Training

This article dives deep into DeepSeek's 3FS parallel file system, detailing its four-component architecture, RDMA‑based high‑speed networking, client options, metadata and storage services, replication protocols, dynamic stripe sizing, and recovery mechanisms that enable efficient AI model training and inference.

AI trainingDistributed File SystemRDMA
0 likes · 21 min read
Inside 3FS: How DeepSeek’s Parallel File System Powers AI Training
AntData
AntData
Mar 4, 2025 · Big Data

Design and Analysis of 3FS: An AI‑Optimized Distributed File System

The article provides a comprehensive English overview of 3FS, an AI‑focused distributed file system that leverages FoundationDB for metadata, CRAQ for chunk replication, and a hybrid Fuse/native client architecture, detailing its design, components, fault handling, and performance considerations for large‑scale training workloads.

AI storageCRAQ replicationDistributed File System
0 likes · 25 min read
Design and Analysis of 3FS: An AI‑Optimized Distributed File System
IT Services Circle
IT Services Circle
Feb 9, 2025 · Big Data

Understanding HDFS: Architecture, Data Blocks, Fault Tolerance, and High Availability

This article explains how HDFS, the Hadoop Distributed File System, splits large files into blocks, replicates them for fault tolerance, organizes the cluster into NameNode and DataNode components, and provides high‑availability and scalability mechanisms such as standby NameNode and federation, enabling reliable big‑data storage and access.

Big DataDataNodeDistributed File System
0 likes · 11 min read
Understanding HDFS: Architecture, Data Blocks, Fault Tolerance, and High Availability
JD Retail Technology
JD Retail Technology
Oct 29, 2024 · Big Data

JD Unified Storage Practice: Cross‑Region and Tiered Storage on HDFS

This article details JD's large‑scale HDFS unified storage implementation, covering cross‑region storage challenges, topology design, asynchronous block replication, flow‑control mechanisms, tiered storage strategies, automatic hot‑cold data migration, and the resulting performance and cost improvements for big‑data workloads.

Big DataCross-Region StorageData Management
0 likes · 20 min read
JD Unified Storage Practice: Cross‑Region and Tiered Storage on HDFS
DataFunSummit
DataFunSummit
Oct 4, 2024 · Big Data

JD Retail HDFS Unified Storage: Cross‑Region and Tiered Storage Practices

This article presents JD Retail's large‑scale HDFS deployment, detailing its unified storage architecture, cross‑region data replication challenges and solutions, tiered storage strategies for hot, warm and cold data, and the operational modules that together improve performance, reliability and cost efficiency in a big‑data environment.

Big DataCross-Region StorageDistributed File System
0 likes · 21 min read
JD Retail HDFS Unified Storage: Cross‑Region and Tiered Storage Practices
Didi Tech
Didi Tech
Sep 5, 2024 · Industry Insights

How Didi Built a Multi‑Protocol, Petabyte‑Scale Storage System for AI Training

Facing petabyte‑level data, billions of small files, and the need for POSIX, S3, and HDFS compatibility, Didi designed a new generation of non‑structured storage—OrangeFS—by analyzing internal systems, combining multiple storage solutions, reusing GIFT technology, and implementing a high‑performance metadata service, multi‑protocol fusion, and robust scalability features.

AI storageBig DataCloud Native
0 likes · 27 min read
How Didi Built a Multi‑Protocol, Petabyte‑Scale Storage System for AI Training
360 Smart Cloud
360 Smart Cloud
May 15, 2024 · Cloud Native

Polefs: A Cloud‑Native Distributed Cache File System for AI Training Workloads

The article outlines the challenges of massive AI training data, defines storage performance requirements, and presents Polefs—a cloud‑native distributed cache file system with unified storage, metadata acceleration, and read/write caching designed to improve GPU utilization and reduce data redundancy.

AICloud NativeDistributed File System
0 likes · 14 min read
Polefs: A Cloud‑Native Distributed Cache File System for AI Training Workloads
DataFunTalk
DataFunTalk
Jan 27, 2024 · Big Data

JuiceFS: A Cloud‑Native Distributed File System for Data Lake and Lakehouse

This article presents JuiceFS, a cloud‑native distributed file system that bridges the gaps between HDFS and object storage, explaining Data Lake and Lakehouse concepts, comparing storage options, detailing JuiceFS's architecture and performance benefits, and showcasing real‑world user case studies.

Big DataDistributed File SystemJuiceFS
0 likes · 23 min read
JuiceFS: A Cloud‑Native Distributed File System for Data Lake and Lakehouse
Didi Tech
Didi Tech
Sep 19, 2023 · Cloud Native

OrangeFS: A Cloud‑Native Multi‑Protocol Distributed Data Lake Storage System

OrangeFS is Didi’s cloud‑native, multi‑protocol distributed data‑lake storage system that unifies POSIX, S3 and HDFS access on a single logical hierarchy, integrates with Kubernetes via a CSI plugin, supports on‑premise and public‑cloud backends, provides multi‑tenant isolation, and dramatically improves elasticity, utilization and latency for petabyte‑scale workloads such as ride‑hailing logs, machine‑learning training, finance and analytics.

CSICloud Native StorageData Lake
0 likes · 17 min read
OrangeFS: A Cloud‑Native Multi‑Protocol Distributed Data Lake Storage System
DataFunTalk
DataFunTalk
Sep 15, 2023 · Cloud Computing

Design and Architecture of Baidu CFS Large‑Scale Distributed File System and Metadata Service

The talk from DataFun Summit 2023 explains how Baidu's CFS storage builds a trillion‑file‑scale distributed file system by revisiting file system fundamentals, POSIX limitations, historical storage architectures, and introducing a lock‑free metadata service with single‑shard primitives, data‑layout optimizations, and a simplified client‑centric architecture that achieves high scalability and performance.

CFSDistributed File SystemPOSIX
0 likes · 31 min read
Design and Architecture of Baidu CFS Large‑Scale Distributed File System and Metadata Service
Programmer DD
Programmer DD
Sep 15, 2023 · Big Data

How Alluxio Manages Massive Metadata: Inode, Block, MountTable, and Worker Insights

This article examines Alluxio's open-source distributed file system, detailing the core types of metadata—inode, block, mount table, and worker—along with the mechanisms for their storage, management, and optimization in both HEAP and ROCKS modes, and provides practical configuration guidance for scaling large-scale data environments.

AlluxioBig DataDistributed File System
0 likes · 15 min read
How Alluxio Manages Massive Metadata: Inode, Block, MountTable, and Worker Insights
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Aug 29, 2023 · Cloud Computing

How Baidu CFS Scales to Billions of Files with a Lock‑Free Metadata Service

This article explains Baidu's CFS architecture for building a billion‑file‑scale distributed file system, covering basic file system concepts, POSIX limitations, metadata service modeling, performance metrics, evolution of metadata architectures, and CFS's lock‑free design that achieves high scalability, low latency, and balanced load in cloud storage.

Distributed File SystemScalabilitycloud storage
0 likes · 32 min read
How Baidu CFS Scales to Billions of Files with a Lock‑Free Metadata Service
ByteDance SYS Tech
ByteDance SYS Tech
Aug 1, 2023 · Cloud Native

How ByteFUSE Revolutionizes High‑Performance Cloud‑Native Storage with FUSE and RDMA

ByteFUSE, a user‑space FUSE‑based solution for ByteNAS, delivers low‑latency, high‑throughput, POSIX‑compatible storage across AI training, database backup, and search services by replacing NFS with a cloud‑native architecture that leverages CSI, RDMA, and kernel‑module hot‑upgrade techniques.

Distributed File SystemFUSEKubernetes
0 likes · 19 min read
How ByteFUSE Revolutionizes High‑Performance Cloud‑Native Storage with FUSE and RDMA
Baidu Geek Talk
Baidu Geek Talk
May 29, 2023 · Backend Development

CFS: Scaling Metadata Service for Distributed File System via Pruned Scope of Critical Sections - Baidu's Implementation Journey

Baidu’s CFS metadata service scales to billions of files by shrinking critical sections through a lock‑free Namespace 2.0 design that confines conflicts to single shards, uses field‑level atomic primitives, and integrates the proxy into the client, delivering up to 76× throughput gains and significant latency reductions in production.

Baidu CFSDistributed File SystemEuroSys 2023
0 likes · 40 min read
CFS: Scaling Metadata Service for Distributed File System via Pruned Scope of Critical Sections - Baidu's Implementation Journey
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
May 25, 2023 · Cloud Native

How Baidu’s CFS Achieved Billion‑File Scale with a Lock‑Free Metadata Service

This article explains the design and evolution of Baidu Cloud File System's (CFS) metadata service, detailing how a novel lock‑free architecture and strategic data layout enable POSIX‑compatible, highly scalable storage that can handle billions of files while maintaining high performance and consistency.

Distributed File SystemScalabilitycloud storage
0 likes · 42 min read
How Baidu’s CFS Achieved Billion‑File Scale with a Lock‑Free Metadata Service
DataFunSummit
DataFunSummit
Jan 20, 2023 · Cloud Native

Design and Architecture of JuiceFS: A Cloud‑Native Distributed File System

This article reviews the evolution of file storage, outlines the challenges of the cloud era, and details JuiceFS's design philosophy, architecture, key capabilities, and real‑world use cases such as Kubernetes, AI, big‑data analytics, and NAS migration to the cloud.

AICloud NativeDistributed File System
0 likes · 22 min read
Design and Architecture of JuiceFS: A Cloud‑Native Distributed File System
ITPUB
ITPUB
Oct 20, 2022 · Big Data

Will HDFS Be Replaced? Analyzing Its Drawbacks and Future Alternatives

The article examines why Hadoop's Distributed File System may become obsolete by detailing its three main shortcomings—deployment complexity, metadata memory limits, and high replication overhead—and explores how newer architectures and erasure coding could address these issues.

Big DataDistributed File SystemHDFS
0 likes · 8 min read
Will HDFS Be Replaced? Analyzing Its Drawbacks and Future Alternatives
Ctrip Technology
Ctrip Technology
Aug 4, 2022 · Cloud Native

Case Study of Using JuiceFS for Cold Data Storage at Ctrip: Architecture, Performance Evaluation, and Optimization

This article presents Ctrip's experience migrating over 2 PB of cold data to JuiceFS, detailing the system's architecture, metadata engine selection, extensive performance testing, fault‑tolerance analysis, and operational optimizations that reduced storage and maintenance costs while supporting future petabyte‑scale workloads.

Cold Data StorageDistributed File SystemJuiceFS
0 likes · 15 min read
Case Study of Using JuiceFS for Cold Data Storage at Ctrip: Architecture, Performance Evaluation, and Optimization
IT Architects Alliance
IT Architects Alliance
Jun 3, 2022 · Backend Development

Open‑Source Distributed File System Based on Spring Boot and Vue CLI – Features and Technical Overview

This article introduces an open‑source distributed file system built with Spring Boot and Vue CLI, detailing its MIT licensing, UI layout, file operations, multiple upload methods, online preview and editing capabilities, storage options, and the underlying backend and frontend technologies.

Distributed File SystemSpring BootVue.js
0 likes · 9 min read
Open‑Source Distributed File System Based on Spring Boot and Vue CLI – Features and Technical Overview
DataFunTalk
DataFunTalk
May 17, 2022 · Big Data

Exploring JuiceFS in Data Lake Storage Architecture

This presentation provides a comprehensive overview of JuiceFS, an open‑source cloud‑native distributed file system, detailing its role in modern data lake and lakehouse architectures, comparing it with HDFS and object storage, and highlighting its performance, integration, and community ecosystem.

Big DataData LakeDistributed File System
0 likes · 19 min read
Exploring JuiceFS in Data Lake Storage Architecture
vivo Internet Technology
vivo Internet Technology
Apr 20, 2022 · Backend Development

FastDFS Overview: Principles, Architecture, Upload/Download Process, Synchronization, and Storage Management

FastDFS is a lightweight, open‑source distributed file system written in C that uses a three‑component architecture—client, tracker server for load‑balancing and discovery, and storage servers with push‑based binlog replication—to handle high‑concurrency upload/download of small to medium files, support group‑wide synchronization, optional trunk storage, Nginx anti‑leech integration, and extensible deduplication via FastDHT.

Distributed File SystemFastDFSSynchronization
0 likes · 15 min read
FastDFS Overview: Principles, Architecture, Upload/Download Process, Synchronization, and Storage Management
Bilibili Tech
Bilibili Tech
Mar 30, 2022 · Big Data

HDFS Architecture, Optimizations, and Future Plans at Bilibili

Bilibili’s HDFS now runs a three‑tier architecture—access, metadata, and data layers—enhanced with a custom MergeFS router, observer NameNode, dynamic load balancing, fast‑failover pipelines, and storage‑aware policies, while future work targets transparent erasure coding, tiered data routing, lock refinements, and a Hadoop 3.x migration.

Big DataDistributed File SystemHDFS
0 likes · 22 min read
HDFS Architecture, Optimizations, and Future Plans at Bilibili
Top Architect
Top Architect
Aug 3, 2021 · Fundamentals

Design and Considerations of Distributed File Systems

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and comparing centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures along with strategies for high availability, performance optimization, and data consistency.

ConsistencyDistributed File SystemScalability
0 likes · 19 min read
Design and Considerations of Distributed File Systems
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jun 18, 2021 · Fundamentals

What Makes Distributed File Systems Tick? Design Principles and Architecture Explained

This article examines the core concepts, requirements, architectural models, persistence strategies, scalability, high‑availability mechanisms, performance optimizations, security models, and practical considerations of distributed file systems such as HDFS, GFS, and Ceph, offering a comprehensive guide for engineers and researchers.

Data ConsistencyDistributed File SystemScalability
0 likes · 21 min read
What Makes Distributed File Systems Tick? Design Principles and Architecture Explained
IT Architects Alliance
IT Architects Alliance
Jun 6, 2021 · Fundamentals

Design Principles and Architecture of Distributed File Systems

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without central nodes, persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and deduplication.

ConsistencyDistributed File SystemScalability
0 likes · 21 min read
Design Principles and Architecture of Distributed File Systems
Programmer DD
Programmer DD
Jun 1, 2021 · Fundamentals

What Makes Distributed File Systems Tick? Design Principles and Architecture Explained

This article explores the core concepts, design requirements, architectural models, scalability, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized approaches while highlighting practical solutions for persistence, consistency, and fault tolerance.

ConsistencyDistributed File SystemScalability
0 likes · 21 min read
What Makes Distributed File Systems Tick? Design Principles and Architecture Explained
DataFunTalk
DataFunTalk
Apr 30, 2021 · Cloud Native

JuiceFS: A Cloud‑Native Distributed File System for Big Data and AI Workloads

This article presents JuiceFS, an open‑source cloud‑native distributed file system that addresses the limitations of object storage for big‑data and AI workloads by providing strong consistency, high‑performance metadata, multi‑protocol support, small‑file management, and deep Kubernetes integration.

Artificial IntelligenceCloud NativeDistributed File System
0 likes · 13 min read
JuiceFS: A Cloud‑Native Distributed File System for Big Data and AI Workloads
Architects' Tech Alliance
Architects' Tech Alliance
Dec 3, 2020 · Fundamentals

IBM GPFS (Spectrum Scale) Overview: History, Architecture, Features, and High‑Performance Computing Use Cases

This article provides a comprehensive overview of IBM's General Parallel File System (GPFS), detailing its historical development, architectural models—including SAN, NSD, and Share‑Nothing Cluster—its operational capabilities, performance advantages, scalability, high‑availability features, and its role in large‑scale high‑performance computing environments.

Distributed File SystemGPFSHigh‑performance computing
0 likes · 12 min read
IBM GPFS (Spectrum Scale) Overview: History, Architecture, Features, and High‑Performance Computing Use Cases
Architects' Tech Alliance
Architects' Tech Alliance
Nov 7, 2020 · Fundamentals

Design Considerations and Architecture of Distributed File Systems

This article examines the evolution, core requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional design trade‑offs of distributed file systems, providing a comprehensive overview for architects and engineers.

ConsistencyDistributed File SystemScalability
0 likes · 19 min read
Design Considerations and Architecture of Distributed File Systems
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 14, 2020 · Big Data

Distributed File Systems: Overview, Design Requirements, Architecture Models, and Key Considerations

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential design requirements, centralized and decentralized architecture models, persistence, scalability, high availability, performance optimization, security, and additional practical aspects such as space allocation, file deletion, small‑file handling, and deduplication.

Data ConsistencyDistributed File SystemScalability
0 likes · 21 min read
Distributed File Systems: Overview, Design Requirements, Architecture Models, and Key Considerations
Alibaba Cloud Developer
Alibaba Cloud Developer
Aug 4, 2020 · Fundamentals

How NFSv4 Guarantees Consistent File Locks Across Clients

This article explains the principles behind NFS file‑lock state view consistency, covering NFSv4's stateful design, the SeqId mechanism, SunRPC handling of duplicate requests, and how signal interruptions are managed to keep client and server lock views synchronized.

ConsistencyDistributed File SystemFile Lock
0 likes · 10 min read
How NFSv4 Guarantees Consistent File Locks Across Clients
IT Architects Alliance
IT Architects Alliance
Jul 25, 2020 · Fundamentals

What Makes Distributed File Systems Tick? Design Principles and Trade‑offs

This article examines the core concepts, architectural models, scalability, persistence, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized designs such as GFS and Ceph to guide future system design decisions.

ConsistencyDistributed File SystemScalability
0 likes · 21 min read
What Makes Distributed File Systems Tick? Design Principles and Trade‑offs
Senior Brother's Insights
Senior Brother's Insights
Jul 21, 2020 · Backend Development

How FastDFS Powers Scalable File Storage for Web Services

FastDFS is an open‑source lightweight distributed file system that tackles large‑scale file storage and load‑balancing by using dedicated Tracker, Storage, and Client roles, offering high performance, easy horizontal scaling, and clear upload/download workflows for web applications.

Backend ArchitectureDistributed File SystemFastDFS
0 likes · 9 min read
How FastDFS Powers Scalable File Storage for Web Services
Architects' Tech Alliance
Architects' Tech Alliance
Jul 15, 2020 · Fundamentals

Design Considerations and Architecture of Distributed File Systems

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and compares centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures, along with strategies for high availability, performance optimization, and handling small files.

ConsistencyDistributed File SystemScalability
0 likes · 21 min read
Design Considerations and Architecture of Distributed File Systems
Full-Stack Internet Architecture
Full-Stack Internet Architecture
May 10, 2020 · Backend Development

Introduction to FastDFS and a Java Integration Example

This article explains the fundamentals of FastDFS, its tracker and storage architecture, the file upload/download workflow, and provides a step‑by‑step Maven‑based Java example—including configuration, dependency setup, and code—to demonstrate how to store and retrieve files in a FastDFS cluster.

BackendDistributed File SystemFastDFS
0 likes · 7 min read
Introduction to FastDFS and a Java Integration Example
Tencent Cloud Developer
Tencent Cloud Developer
Apr 27, 2020 · Cloud Computing

High‑Throughput Cloud File Storage (CFS) Design for Kuaishou's Spring Festival Red‑Envelope Campaign

During Kuaishou's 2020 Spring Festival red‑envelope campaign, Tencent Cloud's Cloud File Storage (CFS) provided a high‑throughput, read‑heavy NFS solution that handled up to 5 GB/s across 1 700 pods, delivering 100% availability, low latency, and massive interaction volumes for billions of users.

CFSDistributed File SystemHigh Throughput
0 likes · 22 min read
High‑Throughput Cloud File Storage (CFS) Design for Kuaishou's Spring Festival Red‑Envelope Campaign
Big Data Technology Architecture
Big Data Technology Architecture
Apr 20, 2020 · Big Data

Introduction to HDFS: Architecture, Features, Replication, Rack Awareness, and Metadata Management

This article provides a comprehensive overview of Hadoop Distributed File System (HDFS), covering its streaming data access model, key characteristics, master‑slave architecture, block storage and replication mechanisms, rack‑aware placement strategy, and how the NameNode manages metadata and checkpoints.

Distributed File SystemHDFSHadoop
0 likes · 7 min read
Introduction to HDFS: Architecture, Features, Replication, Rack Awareness, and Metadata Management
Architects' Tech Alliance
Architects' Tech Alliance
Mar 6, 2020 · Fundamentals

Design Principles and Architecture of Distributed File Systems

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without a central node, persistence strategies, scalability, high availability, performance optimizations, security mechanisms, and practical considerations for small‑file workloads.

ConsistencyDistributed File SystemScalability
0 likes · 20 min read
Design Principles and Architecture of Distributed File Systems
Sohu Tech Products
Sohu Tech Products
Mar 4, 2020 · Big Data

Introduction to HDFS: Architecture, Components, and Operations

This article provides a comprehensive overview of HDFS, covering its role as a distributed file system, the concepts of blocks, NameNode and DataNode responsibilities, replication, edit logs, snapshots, high‑availability mechanisms, and practical considerations for managing large‑scale data storage.

DataNodeDistributed File SystemHDFS
0 likes · 11 min read
Introduction to HDFS: Architecture, Components, and Operations
Architects' Tech Alliance
Architects' Tech Alliance
Feb 16, 2020 · Fundamentals

Design Principles and Architecture of Distributed File Systems

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability and security, architectural models with and without a central node, replication strategies, load balancing, high availability, performance optimizations, and practical considerations for small‑file handling.

ConsistencyDistributed File SystemScalability
0 likes · 20 min read
Design Principles and Architecture of Distributed File Systems
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Nov 15, 2019 · Fundamentals

Understanding FastDFS: A Lightweight Distributed File System

This article introduces the motivations for using a distributed file system, explains the architecture and core concepts of FastDFS—including tracker, storage, client, and group—covers its upload and download mechanisms, synchronization management, and the design of its file identifiers, providing a comprehensive overview for developers.

BackendDistributed File SystemFastDFS
0 likes · 16 min read
Understanding FastDFS: A Lightweight Distributed File System
360 Tech Engineering
360 Tech Engineering
Sep 19, 2019 · Big Data

Understanding HDFS: Architecture, Read/Write Operations, Component Roles, Commands, and Pros & Cons

This article provides a comprehensive overview of HDFS, covering its purpose, architecture, read/write mechanisms, replication strategies, component responsibilities, common command‑line tools, and the advantages and disadvantages of using Hadoop Distributed File System for large‑scale data storage.

Distributed File SystemHDFSHadoop
0 likes · 10 min read
Understanding HDFS: Architecture, Read/Write Operations, Component Roles, Commands, and Pros & Cons
UCloud Tech
UCloud Tech
Sep 2, 2019 · Backend Development

How UFS’s Performance‑Optimized Architecture Cuts I/O Latency to Sub‑10ms

This article explains how UCloud's performance‑oriented UFS leverages NVMe SSDs, upgrades to NFSv4, redesigns business indexing, and introduces a novel append‑only storage engine with stream/extent architecture to achieve sub‑10 ms latency and high IOPS for demanding AI and analytics workloads.

Distributed File SystemNFSv4NVMe SSD
0 likes · 17 min read
How UFS’s Performance‑Optimized Architecture Cuts I/O Latency to Sub‑10ms
DevOps Cloud Academy
DevOps Cloud Academy
Aug 11, 2019 · Big Data

Overview of MFS Distributed File System Architecture Similar to GoogleFS

The article explains the MFS distributed file system, detailing its four components—Master, Metalogger, Chunkserver, and Client—along with hardware recommendations, metadata handling, replication strategies, and FUSE‑based client mounting, providing a comprehensive guide to building a GoogleFS‑like storage cluster.

Big DataDistributed File SystemMFS
0 likes · 5 min read
Overview of MFS Distributed File System Architecture Similar to GoogleFS
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Aug 14, 2018 · Big Data

Understanding HDFS Read and Write Mechanisms

This article explains how HDFS handles file reading and writing, detailing the roles of DFSClient, block selection, hedged reads, packet construction, checksum handling, and the interaction with NameNode and DataNode pipelines to ensure reliability and performance.

DFSClientDistributed File SystemHDFS
0 likes · 7 min read
Understanding HDFS Read and Write Mechanisms
Architecture Digest
Architecture Digest
Jan 20, 2018 · Operations

FastDFS Overview, Architecture, and Deployment Guide

FastDFS is an open‑source lightweight distributed file system written in C, designed for high‑availability file storage and load balancing; this article explains its differences from Hadoop, details its Tracker and Storage architecture, outlines file upload/download workflows, and provides step‑by‑step instructions for installing and configuring FastDFS, Nginx, and related components.

Distributed File SystemFastDFSInstallation
0 likes · 25 min read
FastDFS Overview, Architecture, and Deployment Guide
Architects' Tech Alliance
Architects' Tech Alliance
Dec 18, 2017 · Fundamentals

GPFS Technical Practice Sharing and Building‑Block Design Overview

This article provides a comprehensive overview of IBM GPFS, covering its architecture, management components, networking models, cluster and storage design, as well as practical guidance on building‑block configurations for performance and capacity scaling in high‑performance computing environments.

Building BlockCluster ManagementDistributed File System
0 likes · 13 min read
GPFS Technical Practice Sharing and Building‑Block Design Overview
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jul 1, 2017 · Fundamentals

Designing Distributed File Systems: Solving Local FS Limits

Distributed file systems extend traditional local storage by partitioning data across multiple servers, using a master node for metadata and coordination, handling namespace, replication, load balancing, caching, and client interfaces, thereby overcoming file size, quantity, and concurrency constraints of ext3, reiserfs, and similar local filesystems.

Distributed File SystemReplicationcaching
0 likes · 15 min read
Designing Distributed File Systems: Solving Local FS Limits
JD Retail Technology
JD Retail Technology
Jun 21, 2017 · Backend Development

From Open‑Source to Self‑Developed: JD.com’s One‑Month Migration of Hundreds of Millions of Images

The article describes how JD.com transitioned from open‑source tools to its own distributed file system JFS, completed a massive one‑month migration of billions of images, improved compression with Intel‑optimized pipelines and blind‑watermark technology, and outlines the ongoing scalability and security innovations.

Distributed File SystemImage storageblind_watermark
0 likes · 5 min read
From Open‑Source to Self‑Developed: JD.com’s One‑Month Migration of Hundreds of Millions of Images
dbaplus Community
dbaplus Community
May 16, 2017 · Big Data

Master HDFS: Theory, Shell Commands, and Java API Hands‑On Guide

This comprehensive tutorial explains HDFS fundamentals, its metadata management and advantages, then walks you through setting up a Hadoop environment, executing core shell commands, and using the Java API with complete code examples, enabling you to confidently operate HDFS in practice.

Distributed File SystemHadoopJava API
0 likes · 15 min read
Master HDFS: Theory, Shell Commands, and Java API Hands‑On Guide
ITFLY8 Architecture Home
ITFLY8 Architecture Home
May 8, 2017 · Fundamentals

Designing Scalable Distributed File Systems: Architecture, Challenges, and Solutions

This article explains how distributed file systems overcome the limitations of traditional local file systems by using a master‑metadata server, multiple data nodes, and client interfaces, and it details the key architectural components, common problems, and practical engineering solutions such as replication, load balancing, and caching.

Distributed File SystemReplicationarchitecture
0 likes · 15 min read
Designing Scalable Distributed File Systems: Architecture, Challenges, and Solutions
Architects' Tech Alliance
Architects' Tech Alliance
May 3, 2017 · Fundamentals

Lustre Distributed File System: Overview, Stripe Mechanism, I/O Performance Characteristics, and Optimization Practices

This article provides a comprehensive overview of the Lustre parallel distributed file system, detailing its architecture, stripe configuration, I/O performance traits, challenges with small files, and practical optimization techniques for high‑performance computing environments.

Distributed File SystemHPCLustre
0 likes · 17 min read
Lustre Distributed File System: Overview, Stripe Mechanism, I/O Performance Characteristics, and Optimization Practices
MaGe Linux Operations
MaGe Linux Operations
Feb 5, 2017 · Backend Development

How MogileFS Powers Scalable Distributed File Storage: Architecture & Deployment Guide

This article introduces the open‑source MogileFS distributed file system, explains its server, storage, and client components, outlines its key features and operating principles, and provides step‑by‑step installation, configuration, and Nginx reverse‑proxy load‑balancing instructions for large‑scale image storage.

Backend StorageDistributed File SystemMogileFS
0 likes · 6 min read
How MogileFS Powers Scalable Distributed File Storage: Architecture & Deployment Guide
MaGe Linux Operations
MaGe Linux Operations
Nov 7, 2016 · Big Data

How HDFS Achieves Low Cost, High Reliability, and Fault Tolerance

This article explains how HDFS, inspired by Google’s GFS, provides a low‑cost, highly reliable, fault‑tolerant, and high‑performance distributed file system for big‑data workloads by using replication, standby NameNodes, block storage, rack awareness, and compute‑close‑to‑data strategies.

Big DataDistributed File SystemHDFS
0 likes · 7 min read
How HDFS Achieves Low Cost, High Reliability, and Fault Tolerance
dbaplus Community
dbaplus Community
Oct 17, 2016 · Cloud Computing

How NetEase’s Object Storage Powers Scalable Media and Data Services

This article details NetEase’s object storage (NOS) architecture, covering its use cases, core components such as DDB and DFS, rich media processing pipeline, upload acceleration techniques, and practical overload control measures for handling massive unstructured data workloads.

Distributed File SystemUpload Accelerationcloud computing
0 likes · 14 min read
How NetEase’s Object Storage Powers Scalable Media and Data Services
Qunar Tech Salon
Qunar Tech Salon
May 13, 2016 · Big Data

Overview and Architecture of Hadoop Distributed File System (HDFS)

This article provides a comprehensive overview of Hadoop Distributed File System (HDFS), detailing its design goals, architecture components such as NameNode, DataNode and SecondaryNameNode, data block handling, replication strategies, communication protocols, and the read, write, and delete processes.

Big DataDistributed File SystemHDFS
0 likes · 18 min read
Overview and Architecture of Hadoop Distributed File System (HDFS)
Architects' Tech Alliance
Architects' Tech Alliance
Apr 6, 2016 · Big Data

An Overview of GlusterFS: Architecture, Scalability, High Availability, and Data Management

This article provides a comprehensive overview of GlusterFS, a scale‑out distributed file system, covering its modular architecture, elastic hash algorithm, high‑availability features, global namespace, supported protocols, cloud deployment considerations, and data flow from client to storage nodes.

Distributed File SystemGlusterFSScale-Out Storage
0 likes · 8 min read
An Overview of GlusterFS: Architecture, Scalability, High Availability, and Data Management
Architecture Digest
Architecture Digest
Mar 31, 2016 · Operations

Why Hyper‑Converged Architecture Improves I/O Performance: From Traditional SAN to Tiered Storage

The article explains how traditional SAN storage creates CPU‑I/O bottlenecks, how Google’s distributed file system inspired hyper‑converged designs that fuse compute and storage, and why tiered storage with SSD and HDD offers scalable, high‑performance infrastructure for modern data‑center workloads.

Distributed File SystemI/O performanceVirtualization
0 likes · 11 min read
Why Hyper‑Converged Architecture Improves I/O Performance: From Traditional SAN to Tiered Storage
21CTO
21CTO
Mar 26, 2016 · Fundamentals

How a Custom Linux‑Based Distributed File System Achieves Scalability and Consistency

This article describes the design and implementation of a Linux‑based distributed file system (DFS) that targets large‑scale data storage and access, detailing its architecture, goals, storage model, consistency mechanisms, thread‑safety approach, synchronization strategies, and recovery processes to ensure high availability and data integrity.

DFS designDistributed File SystemLinux storage
0 likes · 35 min read
How a Custom Linux‑Based Distributed File System Achieves Scalability and Consistency
ITPUB
ITPUB
Mar 19, 2016 · Big Data

Inside HDFS: How NameNode and DataNode Manage Big Data Writes and Reads

This article explains the fundamentals of distributed file systems, focusing on Hadoop’s HDFS architecture, the separation of metadata and data via NameNode and DataNode, and detailed step‑by‑step write and read processes, including replication, fault recovery, and block splitting across nodes.

Big DataDataNodeDistributed File System
0 likes · 8 min read
Inside HDFS: How NameNode and DataNode Manage Big Data Writes and Reads
21CTO
21CTO
Nov 14, 2015 · Fundamentals

Inside Taobao’s High‑Performance Distributed File System (TFS): Architecture & Scaling

This article explains the design, storage mechanisms, high‑availability architecture, scaling strategies, multi‑data‑center disaster recovery, operational management, and future plans of Taobao’s distributed file system (TFS), a highly available and scalable storage solution for massive unstructured data.

Distributed File SystemTFSTaobao
0 likes · 14 min read
Inside Taobao’s High‑Performance Distributed File System (TFS): Architecture & Scaling
21CTO
21CTO
Nov 13, 2015 · Operations

Inside Taobao’s High‑Performance Distributed File System (TFS): Architecture & Scaling

Taobao’s File System (TFS) is a highly available, high‑performance distributed storage solution built on Linux servers, featuring name‑server and data‑server clusters, block‑level replication, HA mechanisms, client caching, seamless scaling, multi‑data‑center disaster recovery, and open‑source support for C++, Java, and Nginx integration.

Distributed File SystemTaobaohigh availability
0 likes · 15 min read
Inside Taobao’s High‑Performance Distributed File System (TFS): Architecture & Scaling
Architect
Architect
Nov 4, 2015 · Backend Development

Design Principles and Architecture of the FastDFS Distributed File System

The article explains FastDFS, an open‑source lightweight distributed file system built from tracker servers, storage servers, and client libraries, detailing its component roles, file upload/download workflows, storage organization, small‑file merging, HTTP support, and current limitations such as data safety and load balancing.

Distributed File SystemFastDFSstorage architecture
0 likes · 16 min read
Design Principles and Architecture of the FastDFS Distributed File System
MaGe Linux Operations
MaGe Linux Operations
Jul 13, 2015 · Operations

Deploy MogileFS with Nginx Reverse Proxy for Scalable File Storage

This step‑by‑step guide shows how to build a MogileFS distributed file system cluster on CentOS, configure MariaDB, set up trackers, storage nodes and domains, compile Nginx with the MogileFS module, create init scripts, and verify fault‑tolerant file access through Nginx reverse proxy.

Distributed File SystemLinuxMogileFS
0 likes · 15 min read
Deploy MogileFS with Nginx Reverse Proxy for Scalable File Storage