The article explains why copying a seemingly 100 GB file with the cp command finishes instantly by analyzing file size versus allocated blocks, sparse file concepts, inode structures, direct and indirect block indexing, and how Linux file systems manage storage space efficiently.
The article explains how the ASTRA high‑performance computing platform, built for translational medicine, integrates a three‑tier storage architecture with Intel's DAOS to overcome data‑intensive challenges, improve AI and big‑data workloads, and achieve top rankings on the IO500 benchmark.
The whitepaper introduces a comprehensive DPU performance benchmark methodology, detailing evaluation criteria across network, storage, compute, and security dimensions, and provides guidance on building test systems, selecting workloads, and designing metrics to enable fair, open, and objective assessment of diverse DPU products.
This article explains how IO performance becomes the bottleneck in modern systems, detailing the layered IO architecture, file system and buffer cache roles, various IO models, key metrics such as IOPS and latency, and the impact of storage technologies like LVM, RAID, and SAN/NAS solutions.
This article examines why Git struggles with monolithic repositories exceeding 100 GB, outlines storage, performance, stability, and reliability challenges, and presents preventive, mid‑process, and post‑process strategies—including LFS, shallow and partial clones, commit‑graph, bitmap, multi‑pack index, and bundle techniques—to efficiently manage and maintain large Git monorepos.
This article explains Linux storage hierarchy, the roles of user‑space and kernel caches, the three‑layer I/O stack, page‑cache synchronization policies, file‑operation atomicity, locking mechanisms, and performance testing techniques for HDD and SSD devices.
Music FeatureBox is NetEase Cloud Music’s custom feature store that centralizes metadata, unifies offline and online feature storage across multiple engines, provides a cross‑language DSL for extraction, ensures training‑inference data consistency, and offers built‑in monitoring, thereby streamlining feature engineering and accelerating the platform’s machine‑learning lifecycle.
This article provides a comprehensive overview of Alluxio's metadata and data synchronization mechanisms, covering its unified namespace, mounting strategies, consistency models, various write modes, read workflows, metadata sync techniques, performance optimizations, and recommended configurations for different deployment scenarios.
This article provides a detailed overview of large‑scale RDMA technology, covering basic concepts, major protocols, network‑level techniques such as congestion control, lossless‑to‑lossy evolution and multipath, virtualization, communication libraries for AI training and storage, performance tuning, monitoring, and real‑world deployment experiences.
This article provides a comprehensive technical overview of Apache BookKeeper, covering its role as a distributed append‑only log service, core concepts, high‑availability mechanisms, storage‑media evolution, comparisons with Raft, and community resources, while illustrating its use in Pulsar and large‑scale data platforms.
This article provides a comprehensive overview of Kubernetes, covering its revolutionary role in cloud computing, master‑worker architecture, core concepts such as Pods and Controllers, storage and networking models, workload types, and extensibility through CRI, CNI, and CSI standards.
This article explains why solid‑state drives outperform mechanical hard drives by detailing their distinct operating principles, covering HDD magnetic storage, SSD floating‑gate transistors, and how these differences affect data writing, reading, erasing, and overall performance, especially for random access.
This article explains the fundamental differences between mechanical hard drives and solid‑state drives, covering HDD structure, magnetic data storage, SSD floating‑gate transistors, write/read/erase mechanisms, performance advantages, endurance limits, and why SSDs deliver superior speed and reliability.
The article explains the challenges of running stateful applications on Kubernetes—including complexity, vendor lock‑in, elasticity limits, and bloated infrastructure—and presents multi‑cloud, one‑click storage platforms that provide resilient, portable workloads without data loss.
Kubernetes 1.24, released in early 2022, introduces 46 enhancements—including the removal of dockershim, default‑off test APIs, signed release artifacts, OpenAPI v3 support, GA storage capacity tracking and volume expansion, new CNI requirements, and numerous API graduations—while also deprecating or deleting several legacy components.
This guide walks through configuring an NFS server, installing required utilities, creating a dedicated namespace, installing Helm, and deploying the nfs-client-provisioner chart to provide dynamic NFS storage for workloads in a Kubernetes cluster.
The article explains how the new NVMe ZNS (Zoned Namespace) standard, likened to a 5G upgrade for storage, improves SSD performance and lifespan, outlines its technical background, and describes how major hardware vendors and cloud providers are adopting it for data‑center and future consumer storage solutions.
This article explains the structure of Linux Ext (2/3/4) file systems, the role of superblocks and block groups, compares hardware and software RAID, and introduces LVM as a flexible volume management solution, highlighting its benefits and potential risks.
This article outlines the historical development of storage systems—from single-node and centralized architectures to distributed, cloud, and cloud‑native solutions—detailing storage types (block, file, object), data models, system components, and key design principles for modern storage infrastructures.
The article provides a comprehensive overview of Oracle Real Application Cluster (RAC), detailing its two‑part architecture of Clusterware and RAC instances, shared storage configurations, key Clusterware processes, storage options, benefits such as load balancing and high availability, as well as its management and planning considerations.
This guide walks through the step‑by‑step process of restoring a Ceph 16 cluster when a node's system disk fails, covering host removal, node re‑initialization, Docker and Cephadm installation, host addition, labeling, OSD recreation, and final verification.
This guide explains why databases like TiDB have specific hardware requirements and provides detailed recommendations for CPU selection, memory tuning, storage choices, and network configuration to achieve optimal performance and cost efficiency.
This guide provides a comprehensive overview of Azure Data Lake Storage Gen2, covering when to use it, key design considerations, data organization strategies, access control models, file formats, cost‑optimization techniques, monitoring approaches, and performance‑tuning tips for large‑scale big‑data workloads.
The 2021 China DPU Industry Whitepaper outlines how NVIDIA’s BlueField DPU series—BlueField‑2, the upcoming 400 Gb/s BlueField‑3, and future BlueField‑4—offload and accelerate networking, storage, security, and AI workloads, offering programmable ARM cores, high‑performance NICs, and a rich DOCA ecosystem that reshapes data‑center infrastructure.
Vivo’s self‑built database and storage platform evolved from a fragmented 2018 system into a two‑layer DaaS solution offering unified management, automated security, high availability, and cost‑effective scaling across MySQL, Redis, TiDB, object storage, and future autonomous services within a hybrid‑cloud architecture.
This article describes Yiche's cloud‑native transformation that unifies virtualization and containerization by evaluating Kata, Virtlet and KubeVirt, selecting KubeVirt, detailing its architecture, storage integration, custom enhancements, deployment challenges, and future work for a hybrid infrastructure.
This article dissects Optuna’s internal design by building three miniature versions (Minituna v1‑v3) that illustrate its main components, storage layer, sampling APIs, pruning mechanisms, and joint‑sampling concepts, while comparing them with Optuna’s full implementation.
This article reviews the progress of Kubernetes' in‑tree storage to CSI migration from v1.17 through v1.23, explains the purpose and benefits of CSI migration, outlines new feature gates, observability metrics, bug fixes, collaboration efforts, the release roadmap, and provides guidance for users and contributors.
The article provides an in‑depth analysis of NVIDIA’s BlueField DPU series—detailing the roadmap from BlueField‑2 to BlueField‑4, the technical capabilities of BlueField‑3 across networking, security, and storage, and the DOCA ecosystem that enables programmable, hardware‑accelerated data‑center services, positioning DPUs as a core pillar of modern cloud infrastructure.
This article explains Spark's storage architecture, detailing the roles and interactions of BlockManager, MemoryStore, and DiskStore, including their initialization, data management mechanisms, code implementations, and eviction strategies, to help readers grasp how Spark efficiently handles in‑memory and on‑disk data.
This step‑by‑step guide shows how to initialize a CentOS 7 host, disable firewalls, configure time sync, prepare storage disks, pull the official Ceph Docker image, and launch MON, OSD, MGR, RGW, and MDS containers to build a functional single‑node Ceph cluster with a dashboard.
The guide explains Linux’s unified VFS architecture, detailing how inodes and dentries map files to disk regions, describes superblock and data block layout, outlines ZFS’s pool, copy‑on‑write and ARC caching, covers block device types, I/O scheduling queues, and key performance metrics.
This article explores how to design a cloud‑native distributed database architecture by examining TiDB’s current structure, proposing a storage‑compute separation that leverages cloud services like S3 and EBS, and discussing implications for cost, scalability, fault‑tolerance, and multi‑tenant deployment.
This article explores how to design a cloud‑native distributed database architecture using TiDB as a case study, discussing cost reduction, storage choices, compute‑storage separation, fault tolerance, scaling strategies, and the trade‑offs of caching and persistent storage in modern cloud environments.
The article details Tencent Cloud's TDSQL‑C cloud‑native database, explaining traditional database bottlenecks, its four standout features—PB‑level storage, second‑level scaling, rapid snapshot backup, and serverless pay‑as‑you‑go—and the deep kernel optimizations that enable these capabilities while outlining future performance and cross‑region goals.
The article analyzes how hyper‑converged data center networks use lossless Ethernet to unify general compute, high‑performance computing and storage, enabling lifecycle automation, intelligent operation, and significant compute capacity gains without expanding server count.
This comprehensive guide explains the NVMe specification, covering terminology, SSD architecture, PCIe register layout, queue structures, arbitration mechanisms, data addressing methods, command formats, controller operation, reset procedures, interrupt handling, and advanced features such as firmware updates and end‑to‑end data protection.
This article explains the basic principles of computer architecture, covering CPU operation, memory organization, binary representation, instruction sets, caching levels, storage hierarchy, compilers, script languages, and the impact of open‑source software on system design.
This article explains how Kubernetes CSI enables out‑of‑tree persistent storage by detailing the PV lifecycle—Create, Attach, Detach, Mount, Unmount, Delete—and describing the roles of core components such as apiserver, PVController, external‑provisioner, external‑attacher, and kubelet, with code examples and diagrams.
The article explains why the Linux cp command appears to copy a 100 GB file in less than a second by exploring sparse files, the difference between logical file size and physical block allocation, and how inode direct and indirect indexing enable efficient storage and fast copying.
This article explains the concept of data storage, introduces Fibre Channel SAN architecture, outlines its components and advantages, and compares DAS, NAS, and SAN in terms of connection methods and typical application scenarios.
An unexpected fast copy of a 100 GB file using the cp command reveals the concept of sparse files, where the logical size differs from physical disk usage, and explains how file systems employ inodes, block allocation, and multi‑level indexing to manage storage efficiently.
This article explains the purpose and principles of RAID arrays, compares common RAID levels (0, 1, 5, 6, 10) and JBOD, and outlines advanced configurations such as RAID N+N, helping readers choose the right storage solution for performance and reliability.
This article examines the business storage requirements of large‑scale advertising platforms and details how BaikalDB was architected as a cloud‑native, MySQL‑compatible distributed database that combines OLTP, OLAP, KV and full‑text capabilities while ensuring high reliability, low cost and linear scalability.
This article explains Kafka's storage architecture, including segment files and indexing, and details the reliability mechanisms such as ISR, OSR, LEO, HW, producer acknowledgment levels, and leader election strategies to ensure data consistency and availability.
This article explains Kafka's storage architecture—including topics, partitions, segments, and their naming rules—along with how data is read, and details the system's reliability features such as ISR/OSR replication, leader election, producer acknowledgment levels, and delivery guarantees.
This article explains the concepts of file, object, and block storage, compares their architectures and access methods, and outlines practical scenarios where each storage type best fits, helping you choose the optimal solution for your workload.
This article provides a comprehensive overview of storage fundamentals, covering the concepts, architectures, and use cases of Direct‑Access Storage (DAS), Network‑Attached Storage (NAS), Storage Area Networks (SAN), and detailed explanations of RAID levels, their advantages, limitations, and implementation options.
This guide walks through deploying the NFS‑subdir‑external‑provisioner on Kubernetes, covering migration to the new repository, configuring storage classes with subdirectory support, applying RBAC resources, creating PVCs, enabling high‑availability with leader election, and troubleshooting common mount errors.
This article provides a comprehensive overview of storage fundamentals, covering the three main storage architectures—DAS, NAS, and SAN—their advantages and limitations, followed by an in‑depth explanation of disk arrays and the various RAID levels, their principles, classifications, and practical use cases.
This article investigates the meaning of the number in parentheses for MySQL INT types, demonstrates through storage tests that INT(4) and INT(11) have identical storage size and range, and explains how the ZEROFILL attribute changes only the displayed width.
This article explains how to design private and hybrid cloud architectures, outlines essential open‑source technologies, discusses scalability, availability, manageability and feasibility, and provides practical guidance on resource measurement, queue load‑balancing, and error‑data handling for robust cloud systems.
This article explains how databases store data on physical storage devices, describes the role of indexes and binary search in accelerating query performance, discusses clustered versus non‑clustered indexes, outlines the trade‑offs of excessive indexing, and lists common SQL optimization techniques.
This article examines typical storage options for stateful applications in cloud‑native environments, compares mainstream cloud‑native storage products through performance testing, discusses the challenges of multi‑cloud deployments, outlines criteria for selecting storage solutions, and highlights the advantages of distributed and cloud‑native storage systems for modern workloads.
This article examines the difficulties of running stateful services such as Redis, etcd, and MySQL on Kubernetes and presents practical solutions—including workload selection, CRD/operator extensions, scheduling strategies, high‑availability mechanisms, performance‑optimized networking and storage, and chaos‑engineering validation—to achieve reliable, high‑performance containerized deployments.
This article explains the definitions, key features, and drawbacks of Direct Attached Storage (DAS), Network Attached Storage (NAS), and Storage Area Network (SAN), helping readers choose the most suitable storage architecture for their environment.
This article explains the principles of Linux file systems using clear diagrams and the simple MINIX file system as a teaching example, covering hard‑drive basics, file and directory structures, inode layouts, formatting layout, and step‑by‑step file‑reading procedures.
The article explains dual‑active data‑center disaster‑recovery architecture, covering SAN vs NAS storage options, distance, network, performance, true active‑active versus active‑passive designs, multipathing considerations, and provides a downloadable comprehensive guide to implementation for practitioners.
This article provides a comprehensive overview of database system design, covering the historical evolution, classification of relational and NoSQL databases, common architectures, consistency models, indexing techniques, storage formats, compression, and practical considerations for selecting and optimizing database solutions.
ByteDance’s cloud‑native migration of stateful services uses a custom SolarService extending StatefulSet with Budset CRD to handle versioned data, shard‑aware routing, NUMA‑aware scheduling, advanced storage, eBPF monitoring, and automated PDB eviction, delivering efficiency, cost savings, and reliable rolling upgrades.
This article details ByteDance's journey of migrating stateful services to a cloud‑native Kubernetes platform, covering challenges in state management, infrastructure enhancements, storage solutions, monitoring, and automated operations that together improve efficiency and reduce costs at massive scale.
The article explains the active‑active dual‑data‑center solution as the preferred disaster‑recovery option, detailing its storage choices, distance, network, performance, true versus pseudo active‑active modes, and multipathing requirements, and offers a downloadable technical guide.
The article explains how Nutanix's hyper‑converged infrastructure combines general‑purpose servers, local storage, and intelligent software to replace traditional siloed data‑center components, offering scalable, fault‑tolerant, and cloud‑like performance for enterprise IT environments.
This article explains why setting MySQL columns to NOT NULL improves query optimization, reduces index complexity, saves storage space, and prevents unexpected behavior in calculations, while also detailing default values, handling of NULL in aggregates, and the internal row format differences between NULLable and NOT NULL columns.
When the root partition is too small for Docker's default storage at /var/lib/docker, you can reclaim space by moving the directory to a larger disk and updating Docker's storage path via symbolic links, daemon options, or systemd drop‑in files, ensuring the service continues to run smoothly.
This article provides a technical overview of DRAM memory architecture, its refresh mechanism, comparison with SRAM, market dominance by Samsung, Hynix and Micron, product families such as DDR, LPDDR, GDDR and HBM, and outlines investment logic and domestic development prospects.
The article presents a comprehensive cloud computing reference architecture, illustrating private and hybrid cloud designs, discussing platform choices, storage solutions, scalability, elasticity, and key considerations such as availability, manageability, and feasibility for building robust, cost‑effective systems.
This article explains the Linux virtual file system architecture, the differences between storage media such as HDD, SSD, NAND and NOR flash, introduces NVDIMM‑N/F/P types, their hardware and software handling, and shows how to configure and use persistent memory (PMEM) with DAX, BTT and kernel memmap on Linux.
This article explores how KubeVirt extends Kubernetes to manage virtual machines, compares it with OpenStack, details technical selection, core components, CRDs, storage and networking options, and shares practical SDK usage and migration experiences for cloud‑native virtualization.
This article explains how Kubernetes handles data persistence through concepts such as Volumes, PersistentVolumes, PersistentVolumeClaims, and StorageClasses, compares in‑tree, FlexVolume, and CSI plugin models, and provides practical YAML examples for each component.
This article reviews the evolution of storage systems from single‑node to cloud‑native solutions, explains data classifications and models, and details the main storage types—block, file, and object—while summarizing key concepts of centralized, distributed, and cloud storage architectures.
This article explains how KubeVirt extends Kubernetes to manage virtual machines, covering the background of OpenStack‑to‑K8s migration, technical selection, KubeVirt architecture, CRDs, components, common operations, storage and networking choices, SDK usage, and practical deployment experiences within a private cloud platform.
This article examines the challenges of running both VMs and containers in a hybrid OpenStack‑Kubernetes environment, evaluates KubeVirt against alternatives such as Virtlet and Kata, and provides detailed guidance on KubeVirt architecture, CRDs, storage, networking, SDK usage, and integration with internal platforms.
This tutorial walks you through configuring three CentOS 7.9 nodes, disabling firewalls, setting hostnames, enabling time sync, establishing password‑less SSH, installing required packages, and using Ansible to clone, configure, and deploy a Ceph Nautilus cluster in about ten minutes.
This article provides a comprehensive, step‑by‑step tutorial for setting up a three‑node Ceph storage cluster on CentOS 7.9, covering host configuration, firewall and SELinux settings, package installation, monitor, manager, OSD, MDS, and RGW deployment, along with required keyrings, configuration files, and troubleshooting tips.
This article analyzes why the Linux commands du and df often report inconsistent disk usage, detailing the underlying file system structures, inode and block allocation, how each tool gathers statistics, and the impact of mounted partitions and deleted-but‑still‑referenced files.
The article explains how Twitter built the Account Activity Replay API by repurposing Kafka as a storage layer, detailing the system’s architecture, partitioning strategy, request handling, deduplication, and performance optimizations to provide reliable event recovery for developers.
Kubernetes v1.20 promotes the Volume Snapshot feature to GA, detailing the new default enablement, added validation webhook, supported CSI drivers, deployment steps, usage patterns, and current limitations for creating and restoring snapshots in any Kubernetes cluster.
This article explains what a data center is, its core components of compute, storage, and networking, the operational and architectural considerations for reliability and security, and reviews industry standards and emerging technologies such as edge computing, cloud integration, SDN, HCI, containers, NVMe, and GPU acceleration.
This article explains Kubernetes storage fundamentals—including Volumes, PersistentVolumes, PersistentVolumeClaims, StorageClasses, and the evolution from in‑tree plugins to FlexVolume and CSI—while providing practical YAML examples and highlighting how to mount persistent data onto cluster nodes.
This article explains how real‑world business requirements drive storage performance configuration, details I/O aggregation and RAID write penalties, compares OLTP, OLAP, VDI and SPC‑1 workloads, and discusses the impact of SSD, SAS, FC links, cache, and RAID levels on throughput and latency.
The article explains common storage I/O characteristics—such as I/O size, read/write ratio, sequential versus random access, locality, stability, and multithreading—across various application workloads and shows how tools like Iometer can be used to model and benchmark these patterns.
Elasticsearch combines Lucene’s inverted index with a distributed cluster of master‑eligible, data, and coordinating nodes, using Zen discovery for node election and split‑brain prevention, while indexing writes to primary shards, replicating to replicas, storing immutable segments that are periodically merged for efficient search.
This article explains the concept and benefits of data lakes, outlines the storage and acceleration challenges they pose, presents an ideal checklist for selecting a data lake solution, and evaluates Alibaba Cloud's JindoFS against that checklist, highlighting its capabilities for big‑data and AI workloads.
The article explains how B+ trees serve as efficient disk-based indexes, contrasting disk and memory I/O, describing node and leaf structures, retrieval methods, and dynamic insertion and deletion operations, highlighting their advantages for large-scale data storage and query performance.
This article provides a concise, visual explanation of the fundamental differences between cache and buffer, illustrating their distinct roles—cache for speeding data retrieval and buffer for smoothing data flow—through real‑world analogies, hardware examples, and practical usage scenarios.
This article explains the principles, data models, storage choices, synchronization strategies, metadata handling, and scaling considerations for building a high‑performance feed‑stream system that can support millions to billions of users, with practical guidance on using NoSQL or relational databases.
This guide walks through entering the BIOS, selecting system settings, configuring the storage controller, creating RAID 0/1/5 volumes with specific capacities, and verifying the status of each logical volume using step‑by‑step screenshots.
This article provides a comprehensive overview of server fundamentals, covering definitions, high‑performance characteristics, core components such as CPU, memory, disks and RAID, various CPU architectures, server classifications, C/S and B/S models, and detailed explanations of network interface cards, their functions, classifications, port types, and cable options.
This article explains how the Network File System (NFS) enables file sharing across heterogeneous operating systems, describes the limitations of traditional NFS for high‑performance computing, and introduces pNFS (NFSv4.1) with its three protocols, layout types, and performance advantages for large‑scale parallel storage.
This guide provides a concise overview of core storage concepts—including RAID levels, COW/ROW snapshot mechanisms, backup strategies, continuous data protection, clustering architectures, and virtual tape library (VTL) implementations—helping readers understand how each technology contributes to data reliability and availability.
This article breaks down the five major data replication approaches—application‑level, host‑level, database‑level, storage‑gateway, and storage‑media—detailing their principles, advantages, drawbacks, and typical use cases to help professionals design effective disaster‑recovery solutions.
The article introduces the 2021 China Disaster Recovery Whitepaper and explains five layers of data replication—application, host, database, storage‑gateway, and storage‑media—detailing their mechanisms, advantages, limitations, and use cases in modern backup and business continuity solutions.
This article explains the principles and differences of file‑level and block‑level backup, remote file copy, remote volume imaging, snapshot mechanisms (including CoFW and RoFW), clone techniques, and various backup destinations, paths, and strategies used to ensure data reliability and redundancy.
This article explains the fundamentals and classifications of data backup technologies—including file‑level and block‑level protection, remote file copy, remote volume imaging, snapshot mechanisms, CoFW vs RoFW strategies, and volume clone methods—while also covering backup destinations, paths, and common backup strategies.
Hard drives (HDD) and solid‑state drives (SSD) differ dramatically in latency; the article explains the orders‑of‑magnitude gap between CPU cache, memory, and disk access, details the I/O mechanisms (programmed I/O, interrupt‑driven I/O, DMA), and shows why mechanical drives are inherently slow.
In an interview, ByteDance edge‑computing architect Zhou Chaoyong explains how CDN and storage architectures have evolved with rising data demands, predicts NVMe's dominance in the storage market, and previews his upcoming SACC 2020 talk on high‑performance XFS storage systems.
This article provides a comprehensive guide to building private cloud infrastructures, covering core principles of stability, scalability and redundancy, storage options, network design, compute resource planning, operating‑system choices, security mechanisms, cloud‑ification techniques, and practical OpenStack deployment examples.
This article explains how to design a high‑performance feed‑stream system—covering product definition, data categories, storage options, synchronization modes, metadata handling, commenting, likes, search, sorting, deletion, and update—so you can build a solution that scales to millions or billions of users.
This article explains Prometheus's built‑in time‑series database (TSDB), covering its concepts, storage configuration, block structure, write‑ahead log, mmap reads, inverted indexing, data compression, and remote storage integration for scalable monitoring.
