An in‑depth overview compares Hadoop’s HDFS and the open‑source Ceph object storage, detailing their architectures, replication mechanisms, scalability, strengths, limitations, and real‑world enterprise adoption for handling massive large datasets and unstructured data.
This article explains the fundamentals of Content Delivery Networks (CDN), how they cache object‑storage files, the DNS resolution process with CNAME records, origin‑pull behavior on cache misses, performance trade‑offs, and guidelines for when CDN integration is beneficial or unnecessary.
This article reviews the evolution of file storage, outlines challenges of cloud‑native data management, and details JuiceFS’s cloud‑native design philosophy, architecture, and key use cases such as Kubernetes, AI, and big‑data workloads.
This guide explains the fundamentals of distributed file systems, compares them with traditional storage, introduces MinIO’s architecture and features, and provides step‑by‑step instructions for deploying a multi‑node MinIO cluster with Nginx load balancing on Linux.
As global data volumes surge toward 163 zettabytes by 2025, traditional file and block storage struggle to scale, making object storage—offering unique IDs, metadata binding, and effortless scalability—the preferred solution for handling massive, mostly unstructured data in modern cloud environments.
UCloud’s US3 object storage metadata service, originally built on a chained MongoDB architecture, faced scalability, performance, and cost challenges, prompting a redesign that introduces a high‑compatibility DB‑Gateway, a distributed KV store (UKV) with custom RocksDB, delivering faster reads, zero list‑service latency, 80% cost reduction, and simpler operations.
This article explores Ceph's origins, its RADOS-based cluster architecture, the CRUSH data placement algorithm, the transition from FileStore to BlueStore, and the unified object, block, and file interfaces that make Ceph a versatile solution for modern cloud storage platforms.
This article introduces MinIO as a high‑performance, cloud‑native object storage solution, explains its key features and use cases, provides step‑by‑step Docker installation commands, and demonstrates how to integrate MinIO with Spring Boot through Maven dependencies, configuration, utility classes, service, and controller code.
This article explains how Ozone, the Hadoop community’s new distributed object‑storage system, overcomes HDFS’s small‑file limitations with a hierarchical Volume‑Bucket‑Object model, detailing its architecture, components, data flow for creating and reading objects, and the benefits of its scalable, fault‑tolerant design.
This tutorial explains how to deploy a MinIO object‑storage service using Docker on Linux/macOS and Windows, configure a Spring Boot backend with MinIO client libraries, and build a Vue + Element‑UI front‑end for various file‑upload methods, providing complete code examples and deployment steps.
This article introduces Ceph's distributed storage architecture, explains the CRUSH data placement algorithm, compares FileStore and BlueStore backends, and discusses performance optimizations and emerging storage technologies for cloud‑native environments, including considerations of scalability, fault tolerance, and hardware integration.
This tutorial walks through installing MinIO with Docker on Linux/macOS and Windows, configuring a Spring Boot backend with MinIO Java client, creating a reusable component for file upload, and implementing three Vue/Element‑UI front‑end upload methods, complete with code snippets and screenshots.
This article describes the design goals, technology selection, architecture, high‑availability mechanisms, performance testing, cost optimization, and seamless migration strategy of a new object storage service built on SeaweedFS to support billions of files with low latency and high reliability.
This guide introduces MinIO, an open‑source, S3‑compatible object storage written in Go, outlines its key features and deployment via Docker, and demonstrates step‑by‑step integration with Spring Boot to upload files through the MinIO SDK, including configuration, controller code, and testing.
This guide walks through MinIO client (mc) usage, bucket management, user and policy administration, and two practical methods for expanding a MinIO distributed cluster—peer‑to‑peer scaling and federation with etcd—providing step‑by‑step commands, scripts, and configuration details for cloud‑native object storage.
This article introduces MinIO, a high‑performance, GNU‑AGPL‑licensed object storage compatible with Amazon S3, explains its core concepts such as buckets, drives, sets and erasure coding, and provides a detailed step‑by‑step guide for deploying a distributed MinIO cluster on Linux, including configuration, systemd service setup, Nginx load balancing, and client (mc) usage.
This article introduces the open‑source MinIO object storage system, outlines its high‑performance and cloud‑native features, provides step‑by‑step Docker installation and UI configuration, and demonstrates how to integrate MinIO with Spring Boot for secure file uploads via SDKs.
The article explains the core concepts, advantages, and limitations of file storage, block storage, and object storage, illustrating how each format organizes data, interacts with operating systems, and fits various application scenarios such as databases, cloud services, and unstructured data handling.
The article chronicles Bilibili’s 13‑day development of BOSS, a custom object storage service, detailing how it replaces MySQL‑based routing and ID generation with replicated etcd or Raft KV stores, models metadata via protobuf, adopts erasure coding and a Bitcask‑style engine, and implements safe delete, replica repair, and horizontal scaling for a resilient large‑scale system.
This tutorial introduces MinIO, an open‑source object storage solution, explains its features and application scenarios, provides step‑by‑step Docker installation, shows how to configure it, and demonstrates integrating MinIO with Spring Boot to upload files via SDK.
This presentation provides a comprehensive overview of JuiceFS, an open‑source cloud‑native distributed file system, detailing its role in modern data lake and lakehouse architectures, comparing it with HDFS and object storage, and highlighting its performance, integration, and community ecosystem.
In just 13 days Bilibili transformed a simple MySQL‑based S3 prototype into the BOSS distributed object storage system by separating metadata and data, adding an RPC abstraction layer, implementing two‑level sharding, switching to RocksDB, and deploying a three‑replica, multi‑zone high‑availability architecture.
This article outlines the historical development of storage systems—from single-node and centralized architectures to distributed, cloud, and cloud‑native solutions—detailing storage types (block, file, object), data models, system components, and key design principles for modern storage infrastructures.
This tutorial explains how to deploy MinIO with Docker on Linux/macOS and Windows, configure a Spring Boot backend with MinIO client libraries, create reusable Java components for signed policies and file uploads, and build a Vue/Element‑UI frontend that demonstrates three upload methods, all with complete code examples.
This article introduces object storage as a cost‑effective solution for massive unstructured data and reviews five open‑source platforms—LakeFS, Ceph, MinIO, OpenIO, and Apache Ozone—highlighting their features, scalability, and suitability for modern data‑lake and cloud‑native environments.
This article explains how front-end developers can directly download resources from CDN or object storage using HTML5's download attribute and JavaScript workarounds, addressing CORS limitations and providing code examples for implementation.
Kuaishou reduced half of its massive object storage expenses by redesigning its architecture to use HBase indexing, HDFS large‑file storage, MemoryCache, and a cross‑IDC LRC erasure‑coding warm layer that maintains disaster‑recovery while dynamically moving data from hot to warm to cold tiers.
This article explains what object storage services are, introduces MinIO as a compatible, high‑performance open‑source alternative to AWS S3, and provides step‑by‑step guidance for single‑node and distributed deployments using Docker on various operating systems.
This article explains how QingStor object storage uses SSL and server‑side AES256 encryption, details the client‑side tagging process, walks through the full encrypt‑and‑decrypt workflow, and provides practical API request and response examples for secure data handling.
This article explains how object storage emerged as a cloud-native solution that surpasses traditional DAS, SAN, and NAS architectures by offering virtually unlimited capacity, robust metadata handling, and simple RESTful APIs for modern applications and large‑scale data workloads.
This article explains the concepts of file, object, and block storage, compares their architectures and access methods, and outlines practical scenarios where each storage type best fits, helping you choose the optimal solution for your workload.
This article explains QingStor's object storage concepts, core advantages, global data model, subsystem design, massive small‑file optimizations, key features like lifecycle management and cross‑region replication, and showcases a traffic‑industry use case, highlighting its scalability, reliability, and ease of integration.
This guide walks you through installing MinIO with Docker, using its visual Console, managing buckets and files via the MinIO Client, and leveraging S3 compatibility to integrate with third‑party tools, providing a complete self‑hosted object storage solution.
This article presents a comprehensive overview of using Apache Iceberg with object storage to construct scalable data lake solutions, covering lake architecture, Iceberg table organization, Flink‑based write and read workflows, catalog abstractions, object storage versus HDFS comparisons, append‑upload and atomic‑commit challenges, a demonstration setup, and ideas for storage optimization.
This article provides a comprehensive overview of Ceph, covering its origins, cluster architecture, CRUSH data placement algorithm, unified access interfaces, the transition from FileStore to BlueStore, and emerging storage back‑ends such as SeaStore and PFStore, highlighting performance characteristics and design trade‑offs.
The article explains the rapid growth of global data, compares traditional file and block storage with modern object storage, and highlights why object storage is better suited for massive, unstructured data and cloud scalability.
This article examines the limitations of s3fs and goofys for large‑file backup scenarios, explains the design and implementation of the US3FS FUSE‑based file system, and presents benchmark results showing its superior performance and lower resource consumption.
The article explains the concept and advantages of data lakes, outlines the major storage and acceleration challenges they face, provides a checklist for ideal data‑lake solutions, and details how Alibaba Cloud's JindoFS addresses those challenges with object‑storage‑based, high‑performance, scalable features.
As data volumes explode, Intel’s open‑source Distributed Asynchronous Object Storage (DAOS) offers a low‑latency, high‑bandwidth, NVM‑optimized platform that bridges the gap between high‑performance computing, big data analytics, and artificial intelligence workloads.
The article analyzes the challenges of moving traditional on‑premise big‑data platforms to the cloud, outlines the cost‑saving opportunities of cloud‑native data lakes, presents three core architectural principles, and reviews Tencent Cloud's data lake product suite and its key use cases.
This article outlines practical techniques for multi‑cloud data migration, covering challenges, resource categories, migration phases, and detailed synchronization and verification methods for MySQL databases, file systems, and object storage using tools such as UDTS, pt‑table‑checksum, sync_diff_inspector and US3SYNC.
This article explains the challenges of cross‑cloud migration, outlines three resource categories, details MySQL, file and object storage synchronization methods, presents practical tools and validation techniques, and shares real‑world case studies to ensure data integrity and efficiency.
This article outlines the evolution of storage systems—from early single‑node solutions to modern cloud‑native architectures—detailing key storage types (block, file, object), models, and design principles of centralized, distributed, and cloud storage, providing a concise roadmap for newcomers.
This guide explains the differences between block, file, and object storage, outlines their advantages and drawbacks, introduces distributed storage concepts, and provides a detailed, code‑rich walkthrough for deploying a Ceph cluster—including environment setup, node configuration, CRUSH map tuning, and testing.
By parallelizing Spark’s driver‑side commit, trash, and move phases—previously single‑threaded operations that caused costly copy‑on‑rename when writing massive files to object storage—the Tencent Cloud EMR case achieved over a tenfold (1,100 %) speedup, making object storage a viable alternative to HDFS.
This guide explains how to set up, configure, and run the Ceph s3-tests compatibility suite—including installation, virtualenv setup, config.yaml creation, HTML reporting with nose-html-reporting, and custom test extensions—to reliably validate S3 API behavior in object‑storage environments.
This article explains how to pick the most suitable cloud storage—cloud disk, file service, or object storage—by using four easy-to-remember mnemonics that cover functionality, performance, concurrency, and latency, helping readers avoid complex parameter analysis.
This article explains the fundamentals of object storage, compares it with block and file storage, outlines its architecture, components, advantages, use cases, and limitations, showing why it has become the dominant storage model in modern cloud environments.
This article traces Ceph’s evolution from its academic origins to widespread adoption in cloud‑native environments, explains the four stages of storage development, details Ceph’s unified object, block, and file interfaces, and describes its architecture—including monitors, OSDs, CRUSH algorithm, placement groups, and high‑availability features.
This guide explains the background of object storage services, shows SDK upload examples for Alibaba Cloud, Huawei Cloud, and Qiniu, introduces the Amazon S3 protocol as a universal solution, lists compatible Chinese cloud providers, and demonstrates how to configure and use a Spring Boot OSS starter—including support for self‑hosted MinIO—by setting appropriate parameters and invoking OssTemplate.
This article explains the fundamentals of Ceph, a high‑performance, highly available and scalable distributed storage system, covering its architecture, core components, data placement algorithms, storage interfaces, and typical deployment scenarios in cloud environments.
This article provides a concise yet comprehensive overview of the backend development technology stack, covering load balancing, microservice ecosystems, RPC frameworks, service discovery, relational and NoSQL databases, caching strategies, message queues, object storage, and search engines, while highlighting practical configurations and real‑world trade‑offs.
This article presents a comprehensive offline big‑data processing framework that leverages scalable object storage for PB‑level data, details storage and compute engine requirements, compares cost options, describes data pipeline design, and showcases an e‑commerce case study with Spark‑driven analytics.
This guide explains how to deploy MinIO object storage on Kubernetes using a PersistentVolume and Helm, configure GitLab Runner to use MinIO as an S3 cache, troubleshoot common issues, and verify the setup with a sample CI pipeline.
This article introduces Apache Ozone, a scalable distributed object storage system for Hadoop, covering its background, core components, design principles, architecture, deployment steps, configuration examples, and basic command‑line operations for managing volumes, buckets, and keys.
This article provides a detailed technical overview of EMC's Elastic Cloud Storage (ECS), covering its historical evolution, layered architecture, supported protocols, data protection mechanisms, performance characteristics, limitations, and future roadmap within the context of cloud object storage.
This lesson introduces object storage, explaining its concept, technical principles, market scale, advantages over traditional DAS/NAS/SAN solutions, and key architectural components, while highlighting scalability, cost efficiency, high availability, and design considerations for modern cloud storage systems.
This article provides a comprehensive technical overview of MinIO, an open‑source, S3‑compatible object storage system, covering its design philosophy, data organization, distributed architecture, erasure‑coding, lock management, lambda notifications, backup strategies, performance optimizations, and a comparative analysis with Ceph, highlighting its suitability for AI, big‑data, and cloud‑native deployments.
Despite recent market turbulence and negative headlines, Hadoop's revenue continues to grow, driven by cloud migration, evolving storage solutions, and increasing adoption of related projects like Spark and Kafka, positioning it as a leading data‑lake technology.
This article explains Ceph's distributed storage design, detailing how clients use RADOS to map files to objects, objects to placement groups, and placement groups to OSDs, while covering component roles, metadata handling, Xattr limits, and the impact of PGs on OSD communication overhead.
This article presents the architecture and key design features of WOS, the 58 Group’s self‑developed object storage system, covering its overall framework, proxy, store, directory, detector modules, fast‑upload “秒传” mechanism, and erasure‑coding solution for efficient, scalable, and reliable unstructured data storage.
This article provides a comprehensive overview of Red Hat Ceph's distributed object storage architecture, covering storage pools, CRUSH placement, authentication, I/O workflows, internal operations, client interfaces, data striping, erasure coding, high availability, and encryption mechanisms for secure, scalable deployments.
This article introduces the concepts of block, file, and object storage and reviews several open‑source distributed object storage solutions—including Swift, Ceph, MinIO, HBase MOB, and Hadoop Ozone—highlighting their architectures, features, and typical use cases for large‑scale data handling.
OpenSDS Gelato is an open‑source multi‑cloud data management platform that unifies object storage across public clouds and on‑premises, offers S3‑compatible APIs, supports backend pooling and policy‑driven cross‑cloud data flow, and outlines a roadmap for lifecycle management and advanced migration tools.
This article explains block, object, and file storage concepts based on Ceph, describes how front‑end JavaScript resources are migrated to an S3‑compatible object store, outlines the supporting Lua/Nginx combo tool, caching strategies, and step‑by‑step e‑commerce migration procedures.
The article summarizes the technical talk from the 58 Group technology salon, detailing the architectures, scalability, high‑availability mechanisms, and storage models of Kuaishou's BlobStore and 58's WOS, and compares their design choices for large‑scale object storage.
This article introduces Zhuanzhuan's multi‑service storage layer—including MySQL, TiDB, ZZRedis, ZZKV/ZZList, and ZZOS—detailing their applicable scenarios, architectural designs, and the improvements made to address scalability, security, and resource‑usage challenges.
This article provides a comprehensive English translation of the Red Hat Ceph Storage Architecture Guide, covering Ceph's distributed object storage concepts, cluster architecture, storage pools, CRUSH algorithm, replication and erasure‑coding I/O, internal operations, high‑availability mechanisms, client interfaces, and encryption considerations for cloud environments.
The explosive growth of big‑data applications is forcing storage vendors to redesign architectures for petabyte‑scale capacity, real‑time latency, high IOPS, security compliance, and cost efficiency, while also addressing flexibility, data longevity, and the needs of both large and small users.
The article explains Git’s fundamental architecture, describing how trees, blobs, and commits are stored as objects identified by hashes, how version history forms a linked list, and how optimizations such as directory sharding and garbage collection fit into this design.
iQiyi improved OpenStack Swift’s video‑streaming performance by redesigning its storage engine to merge numerous tiny objects into larger files, replacing Swift’s per‑object directory structure with RocksDB‑managed metadata, eliminating file‑ID overhead, using prefix scans for fast listings, and adding lazy deletion, achieving dramatic speedups with only about 3,000 lines of new code.
At the 2023 Tencent Cloud + Future summit, Liu Jinming detailed Tencent Cloud COS’s three‑tier EB‑level object storage architecture—covering network, application, and data layers—highlighting its 99.95% availability, 11‑nine durability, end‑to‑end encryption, scalable performance, tiered pricing, and real‑world media and security use cases.
This article chronicles JD.com's journey from recognizing massive storage demands to designing, building, and evolving a self‑developed distributed storage platform—JFS—that handles small and large files, powers a custom image system, object storage, and future container‑native workloads.
The article explains how to integrate the open‑source Ceph storage system with Windows using the Ceph Object Gateway or SUSE Enterprise Storage iSCSI target, describes Ceph’s architecture and access methods, and discusses file‑sharing options such as NFS and RBD.
This article details Tongcheng's adoption of Ceph for handling massive unstructured data, describing the motivations, architecture, specific uses of RGW, block storage, and CephFS, the challenges encountered, optimization measures, monitoring practices, and planned future improvements.
The article provides a comprehensive overview of Ceph, covering its object storage concepts, architecture, core technologies, performance testing results versus Swift, and concludes with a summary of findings and future outlook for this distributed storage solution.
This article explains the fundamental architectures of Ceph and GlusterFS, describes their core components such as RADOS, librados, CephFS, RBD, and Gluster bricks, and compares their scalability, high‑availability, hardware requirements, decentralization, and performance characteristics for distributed storage deployments.
With data volumes soaring to petabyte scales, the article explains why object storage is the only viable solution for massive storage needs, outlines procurement considerations, design principles, and operational challenges, and offers practical guidance for building, evaluating, and scaling PB‑level storage systems.
The article provides a detailed technical overview of Scality Ring 6.0, explaining its three‑layer software stack, X86‑based scale‑out hardware, diverse connectors, storage node design, management tools, routing protocol, data durability, multi‑site deployment models, and consistency guarantees.
Data durabilityMulti-site DeploymentObject Storage
This article details NetEase’s object storage (NOS) architecture, covering its use cases, core components such as DDB and DFS, rich media processing pipeline, upload acceleration techniques, and practical overload control measures for handling massive unstructured data workloads.
This article provides a systematic summary of Ceph's architecture, including its core services, RADOS layer, libraries, high‑level storage interfaces, client interactions, data placement algorithms, and deployment considerations, while also comparing its ecosystem and enterprise features.
This article provides a comprehensive overview of Ceph's distributed storage architecture, its integration with OpenStack, internal data mapping mechanisms, advanced features like SSD caching and global object storage, and examines the key factors behind its widespread adoption in cloud environments.
This article compares the open‑source Ceph storage system and Huawei's OceanStor 9000 commercial SDS, detailing their service components, software stacks, OpenStack compatibility, and differing strengths in scalability, client architecture, and management for cloud and enterprise storage scenarios.
This article introduces Ceph as an open‑source distributed storage system that supports object, block, and file interfaces, explains its core RADOS components, outlines its advantages such as unified storage, high scalability, reliability and performance, and details how Ceph handles object and block storage within cloud environments.
This guide consolidates Huawei OBS eSDK storage training, explains the service’s core capabilities, lists bucket and object operations, and walks through setting up a C# development environment with step‑by‑step demos for creating buckets, uploading, downloading, and hosting static sites.
This article presents a systematic performance and cost‑effectiveness comparison of six major cloud object‑storage providers—AWS, Azure, Qiniu, Huawei Cloud, Alibaba Cloud, and Kingsoft Cloud—using JMeter‑driven upload/download tests across various file sizes and analyzing price data to rank their overall value.
This article examines the problems and trade‑offs of metadata protection in cloud object storage, highlighting the importance of reliability, availability, and consistency, and comparing primary‑secondary and multi‑replica models, including the WRN algorithm and practical operational considerations.
In this presentation, seasoned R&D engineer Bian Xuebin, who leads Suning’s object storage platform, shares the design principles and architectural strategies behind Suning’s multi‑tenant object storage system, illustrated through a series of detailed slides.
This article provides a comprehensive overview of OpenStack Swift, covering its background, consistent hashing, quorum-based consistency model, ring data structure, hierarchical data model, system architecture, component responsibilities, and RESTful API usage for scalable object storage.