Learn how to recognize Linux sparse files, use commands like du, ls, tar, and find to accurately assess their true disk usage, and efficiently copy or compress massive log files without consuming unnecessary space.
The article explains why a 100 GB file can be copied in under a second by examining the difference between logical file size and physical block usage, demonstrating sparse file behavior, inode structure, direct and indirect block indexing, and how these mechanisms affect copy performance on Linux.
A recent poll revealed that MySQL with the InnoDB engine can support up to four billion tables, a limit dictated by the underlying file system rather than MySQL itself, and the official documentation confirms this massive capacity.
This article explains the structure of Linux Ext2/3/4 file systems, the role of superblocks and block groups, compares hardware and software RAID, and introduces LVM for flexible storage management, highlighting both benefits and risks.
This article introduces Cloudreve, an open‑source public cloud file system supporting multiple storage backends, outlines its key features such as direct client uploads, Aria2 integration, WebDAV, and provides step‑by‑step deployment and build instructions for Linux and Windows environments.
This article explains the structure of Linux Ext (2/3/4) file systems, the role of superblocks and block groups, compares hardware and software RAID, and introduces LVM as a flexible volume management solution, highlighting its benefits and potential risks.
This comprehensive guide explores the core components of a Linux system—including the kernel, memory management, process scheduling, file system architecture, device drivers, networking, and shell—while also covering disk partitions, mounting, and practical command examples for system administration.
This guide explains the differences between hard links and soft (symbolic) links in Linux, how to create and manage them with the ln command, their filesystem constraints, practical use‑cases, and common pitfalls such as dangling links and path issues.
Even when a Linux system reports a full disk, deleted files may still occupy space because active processes keep them open; using commands like df, du, and lsof reveals hidden usage, and understanding the virtual file system, inode structures, and link management explains how to release the space.
This article explains why a Linux system can report a full disk even after files are deleted, covering the role of open file handles, the lsof command, and the virtual file system architecture that manages inodes, dentries, and link counts.
Even when the `df` command shows a full disk, hidden deleted files held open by processes can consume space, and understanding this requires diving into Linux’s virtual file system architecture, including superblocks, inodes, file and dentry objects, as well as link types and file‑process interactions.
This article explains the purpose and functions of file systems, describes logical and physical file structures, introduces Linux's virtual file system architecture and its core data structures such as superblocks, inodes, dentries and file objects, and details the path‑lookup process used by the kernel when opening files.
This article explores Linux file system fundamentals, covering physical disk layout, partition strategies, how the OS reduces random‑read latency, the real space cost of empty files and directories, inode usage, block allocation, and practical tips for improving disk I/O performance.
The guide explains Linux’s unified VFS architecture, detailing how inodes and dentries map files to disk regions, describes superblock and data block layout, outlines ZFS’s pool, copy‑on‑write and ARC caching, covers block device types, I/O scheduling queues, and key performance metrics.
The article explains how a Linux file can appear to be 100 GB yet copy in under a second because the file system uses inodes, direct and indirect block pointers, and sparse allocation, separating logical size from actual physical storage.
This article explains the physical structure of mechanical disks, partitioning strategies, inode and block allocation, and how filename length and directory layout affect Linux file system performance, providing practical experiments and tips for improving disk I/O.
The article explains why the Linux cp command appears to copy a 100 GB file in less than a second by exploring sparse files, the difference between logical file size and physical block allocation, and how inode direct and indirect indexing enable efficient storage and fast copying.
A colleague was amazed when the cp command copied a 100 GB file in less than a second, prompting an investigation that reveals the difference between logical file size and physical block usage, the role of inodes, direct and indirect block indexing, and how sparse files make such copies appear instantaneous.
The article explains how Linux file systems use inodes and multi‑level block indexing to separate a file's logical size from its physical storage, illustrating why copying a seemingly huge file with the cp command can complete instantly when the file is sparse.
A colleague copied a 100 GB file in under a second, prompting an investigation that reveals how Linux file systems distinguish logical file size from physical storage using inodes, block allocation, direct and indirect indexing, and sparse file techniques that make such copies appear instantaneous.
An unexpected fast copy of a 100 GB file using the cp command reveals the concept of sparse files, where the logical size differs from physical disk usage, and explains how file systems employ inodes, block allocation, and multi‑level indexing to manage storage efficiently.
This article explains why copying a seemingly 100 GB file with the cp command finishes almost instantly by exploring sparse files, the difference between file size and physical block usage, inode structures, multi‑level block indexing, and how modern file systems efficiently manage storage.
This article explains core Linux I/O concepts—including file systems, I/O types, caching mechanisms, copy‑on‑write, zero‑copy, and performance metrics—then presents practical application, file‑system, and disk‑level optimization techniques to improve throughput and latency.
This article demystifies the three common Linux commands—ln, mv, and cp—explaining how they differ in handling inodes, dirents, and data blocks, when they create links or move files, and why their performance varies across file systems and file types.
This article shows how to use Python on Windows to obtain the system drive, enumerate and delete files and directories with os.walk, copy the script to the startup folder, generate a BAT wrapper, and silently shut down the computer, illustrating a basic malicious automation technique.
This article examines the limitations of s3fs and goofys for large‑file backup scenarios, explains the design and implementation of the US3FS FUSE‑based file system, and presents benchmark results showing its superior performance and lower resource consumption.
Linux’s Virtual File System (VFS) acts as a glue layer between user space and various file systems, providing a unified interface and object model—including superblocks, inodes, file and dentry objects—and employs caching mechanisms like dcache to efficiently manage directory entries.
This tutorial walks through building a tiny 1 MB file system using 1 KB blocks, block bitmaps, 128‑byte inodes, superblocks, directory structures, and single‑ and double‑indirect indexing, culminating in a design comparable to Linux's ext2 filesystem.
This article explains how a seemingly simple read‑of‑one‑byte in user code triggers a complex Linux I/O stack involving the I/O engine, system calls, VFS, page cache, file system implementations, generic block layer and I/O scheduler, and clarifies when actual disk I/O occurs and its granularity.
This article explains Python's os.path functions—os.path.exists, os.path.isfile, and os.path.isdir—demonstrating how to determine whether a given path points to an existing file or directory, with example code snippets and notes on related functions like islink and ismount.
The PHP dir() function opens a directory and returns a Directory class instance, accepting a directory path and an optional context resource, with detailed parameter descriptions, return values, and a complete example demonstrating how to read entries and close the handle.
This comprehensive guide covers over forty operating system interview questions, explaining core concepts such as OS fundamentals, process and thread management, memory handling, file systems, I/O mechanisms, scheduling algorithms, and deadlock prevention, providing detailed answers and diagrams to help candidates excel in technical interviews.
This article explains the Linux I/O stack, detailing how data moves from user‑space buffers through libc and page cache to the disk, covering system calls, synchronization primitives, scheduler behavior, consistency, safety, and performance considerations.
This article explains the layered design of Linux file I/O, describes how data moves from application buffers through libc, page cache, and kernel I/O queues to disk, discusses synchronization primitives, consistency issues, and performance factors such as scheduling algorithms and hardware characteristics.
This article provides a comprehensive overview of Linux I/O fundamentals, covering the layered I/O stack, buffer interactions, system call flow, scheduler algorithms, consistency and safety considerations, and performance characteristics, supplemented with code examples and illustrative diagrams.
This article explains the Composite (part‑whole) design pattern in Go, illustrates its concept with natural and file‑system analogies, and provides complete Go code demonstrating how files and folders can be treated uniformly as tree nodes.
This article analyzes why the Linux commands du and df often report inconsistent disk usage, detailing the underlying file system structures, inode and block allocation, how each tool gathers statistics, and the impact of mounted partitions and deleted-but‑still‑referenced files.
This guide walks through the concepts, volume types, environment setup, cluster creation, and client mounting steps required to build a highly available GlusterFS distributed replicated storage system integrated with NFS‑Ganesha on Linux servers.
The article explains why Linux’s du and df commands often report inconsistent disk usage, detailing the underlying file‑system structures such as inodes, block maps, and superblocks, the processes of file creation and deletion, and how mounted partitions, open file handles, and stat calls affect each tool’s calculations.
The article thoroughly explains Linux kernel file cache, detailing how read() and write() operations traverse VFS to the block layer, the adaptive read‑ahead algorithm that pre‑loads sequential pages, the internal functions such as generic_file_buffered_read and ondemand_readahead, and outlines write handling and future topics on mmap and cache reclamation.
This article explains the design of Linux file systems, focusing on ext4's inode layout, block allocation, extents, directory storage, journaling modes, and the kernel's cached and direct I/O paths, complete with code snippets and structural diagrams.
Journal file systems use write‑ahead logging to record each write operation as a transaction, ensuring that after power loss or crashes the system can replay logs and maintain metadata and user‑data consistency, avoiding corruption and space waste through techniques like data, ordered, and metadata journaling.
This article provides an in‑depth technical guide to Linux, covering its Unix heritage, process and thread creation, kernel architecture, virtual memory, scheduling algorithms, file‑system structures, I/O handling, networking, and built‑in security mechanisms such as user IDs, permissions, and set‑UID features.
This article explains how the Network File System (NFS) enables file sharing across heterogeneous operating systems, describes the limitations of traditional NFS for high‑performance computing, and introduces pNFS (NFSv4.1) with its three protocols, layout types, and performance advantages for large‑scale parallel storage.
This article explains Linux's file system architecture, covering basic directory structures, absolute and relative paths, symbolic links, the Virtual File System layer, core system calls such as creat, open, read, write, lseek, locking mechanisms, and the design of ext2, ext4, /proc, and NFS file systems.
A comprehensive guide walks through Linux fundamentals, covering the UNIX heritage, kernel structure, system calls, process and thread creation, inter‑process communication, memory management, file‑system design, I/O handling, scheduling algorithms, modules, and built‑in security mechanisms.
This article provides a comprehensive overview of Linux file system architecture, covering basic concepts, directory structures, absolute and relative paths, linking, VFS abstraction, ext2/ext4 implementations, locking mechanisms, key system calls, and network file systems like NFS, illustrating how they interact within the kernel.
NVFS is a Red Hat‑led Linux file system designed for DAX‑based persistent‑memory devices, mapping the whole device into a linear address space to bypass the kernel block layer, and delivering performance far superior to EXT2/EXT4/XFS and the earlier NOVA file system.
This article explains the core components of Linux file systems—including inodes, directory entries, virtual file system layers, storage allocation methods, free‑space management, block‑group layout, directory handling, hard and symbolic links, and various I/O models—providing a comprehensive guide for developers and system engineers.
The article explains why a Linux system may report a full disk even after deleting files, detailing how open file handles keep space occupied, and walks through the virtual file system architecture—including superblocks, inodes, file and dentry objects—while demonstrating diagnostic commands like df, du, lsof, and illustrating link types and file‑process interactions.
This article explains the PHP is_dir() function, describing its purpose to check whether a given filename refers to an existing directory, detailing its parameter, return values, and providing example code snippets demonstrating true and false outcomes for various path inputs.
This article provides a comprehensive overview of Linux file systems, covering VFS, directory hierarchy, absolute and relative paths, symbolic links, mounting, file locking, and the essential system calls such as creat, open, read, write, lseek, stat, pipe, and fcntl.
This guide shows how a file can appear deleted on Linux yet still occupy disk space, explains the inode reference‑count mechanism, demonstrates how to reproduce the issue with dd and a C program, and provides commands like lsof and /proc inspection to locate and release such hidden usage.
This tutorial explains Linux file system hierarchy, absolute and relative paths, common directory purposes, command‑line navigation, file creation, copying, moving, deleting, and provides practical Vim editing tips with example commands and code snippets.
This article walks through using Python's os and shutil modules to obtain the system drive, list and delete files, create a startup entry, and force a Windows shutdown, illustrating a lightweight, educational example of a hacking‑style tool.
This article explains why removing a file with rm on Linux sometimes leaves disk space unchanged, demonstrates the behavior with a random file and a C program that keeps the file open, and shows how to identify and release such lingering file allocations using inode reference counts and lsof.
This article explains how a simple read of a single byte in user space triggers a complex Linux I/O stack involving the read system call, VFS, page cache, generic block layer, and I/O scheduler, and clarifies when actual disk I/O occurs and how many bytes are transferred.
Microsoft's latest Windows 10 build lets users open the Linux subsystem's root directory straight from File Explorer, offering a penguin icon shortcut, command‑line access via \\wsl$, and guidance on safe usage while highlighting upcoming full‑kernel integration.
This article explains the concept of inodes, their contents, size, numbering, relationship with directories, hard and soft links, special behaviors in Unix/Linux, and provides a real‑world troubleshooting example for inode exhaustion.
This comprehensive guide explains the fundamentals of operating systems, covering hardware components, CPU architecture, memory hierarchy, process and address space management, file systems, system calls, and various OS structures such as monolithic, layered, microkernel, and client‑server designs, providing clear examples and diagrams for each concept.
This guide explains the concepts, differences, and practical commands for creating hard links and symbolic (soft) links in Linux, illustrating their behavior with examples, showing their advantages, limitations, and how link counts change across files and directories.
F2FS is a log‑structured, flash‑aware file system that transforms random writes into sequential ones, uses segment‑based layout, NAT indexing, and hot‑cold data segregation with sophisticated garbage‑collection to improve SSD performance, and is now deployed in Android devices with ongoing enhancements such as online resizing and cold‑data compression.
This guide explains the differences between hard and symbolic links in Linux, demonstrates how to create and inspect them with the ln command, and covers link counts, limitations, and practical examples for managing files and directories.
This article explains Unix/Linux inodes—the metadata structures that store file information—covering their purpose, contents, size considerations, inode numbers, directory handling, hard and soft links, and special inode-related operations, with practical command examples and visual illustrations.
This comprehensive guide explains the core components of a Linux system—including the kernel, memory management, process scheduling, device drivers, networking, shells, file‑system types, VFS architecture, mounting procedures, and link handling—providing clear examples, commands, and diagrams for each topic.
Microsoft has opened the exFAT file‑system specifications and pledged defensive‑patent protection, enabling native exFAT support in the Linux kernel and sparking community excitement while hinting at future cross‑platform storage improvements.
Journal file systems protect against data corruption caused by power loss or crashes by recording each write operation as a transaction in a dedicated log, then committing the changes only after the log is safely stored, enabling replay to restore consistency.
The April 2019 Windows 10 update introduces a 9P‑based file server in WSL, allowing users to safely browse Linux files from Windows Explorer, export and import distributions, and use new command‑line features while noting remaining limitations and best‑practice warnings.
This article provides a comprehensive overview of Linux’s core architecture, detailing the kernel, memory management, process scheduling, file systems, VFS, device drivers, networking, shells, partitioning, mounting, and essential commands, offering readers a solid foundation for understanding and working with Linux systems.
This guide walks you through Python’s os module, showing how to create, inspect, rename, and delete files and directories, manipulate paths, work with environment variables, execute system commands, and traverse directory trees using os.walk, all illustrated with clear examples and diagrams.
This article explains how Linux models all resources—including disks, devices, and inter‑process communication—as files, describes the physical storage mechanisms of hard drives, and walks through the steps of partitioning, formatting, mounting, and navigating the Linux file system hierarchy.
This article provides a comprehensive overview of Linux fundamentals, covering the kernel architecture, virtual memory management, process scheduling and inter‑process communication, the hierarchical file system and VFS layer, device drivers, networking, shell variants, partitioning, mounting, and essential command‑line tools.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and fingerprint‑based deduplication.
This article reviews the history of Sun and Solaris, explains ZFS’s core design as a combined file system and volume manager, and details its key features such as metadata integrity, copy‑on‑write, snapshots, clones, storage pools, RAID‑Z, caching, compression, deduplication, and tunability.
This article explains Linux's hierarchical file system, detailing how the root directory, partitions, boot and super blocks, inodes, and data blocks interact to store and retrieve files, and it also covers file descriptors, sharing, and process forking.
This article provides a comprehensive overview of NFSv4’s design goals, security and performance improvements, and details the four major advantages of NFS‑Ganesha, its modular architecture, memory and thread management, caching mechanisms, and a practical Ceph RGW integration example.
When accidental deletion removes files from the /data/webapps/.../upload directory, this guide shows how to install extundelete, locate the relevant inodes, and safely recover the lost data using read‑only mounts and specific restore commands.
This article offers a gentle, language‑first introduction to Linux for .NET developers, explaining why Linux lacks a Windows‑style registry, the advantages of the command line over graphical interfaces, and demystifying the purpose of key directories such as /, /boot, /bin, /usr, /etc, /var, and /home.
This article explains the standard Unix/Linux directory hierarchy, the seven file types, how to inspect file types and permissions with ls, the meaning of permission bits, octal codes, and practical chmod examples for managing file access.
This article provides a comprehensive overview of Linux system architecture, covering the kernel, shell, file system, memory management, process scheduling, device drivers, networking, partitioning, mounting, and common command‑line tools, offering readers a solid foundation in operating‑system fundamentals.
This article provides a comprehensive guide to Node.js's path module, covering how to retrieve directories, filenames, and extensions, combine and resolve paths, normalize and parse paths, format and decompose file paths, compute relative paths, and handle platform-specific differences.
This article provides an end‑to‑end overview of storage I/O performance, covering fundamental concepts, the flow of I/O through host, OS, file system, cache, block, and scheduler layers, and practical techniques for monitoring, tuning, and designing systems for optimal disk throughput.
This article compares Linux and Windows file systems, explains the concept of a virtual file system, details how Linux organizes directories, and describes the purpose of key system directories such as /boot, /bin, /etc, and /proc, helping beginners grasp fundamental OS storage structures.
This article presents five practical Python scripts that demonstrate how to traverse directories and list files, draw text triangles, create a number‑guessing game, log disk usage, and analyze IP access counts from log files, each explained with clear code examples.
This article explains how Linux organizes files through its file system, covering the role of the kernel and shell, the classification of file systems, detailed insights into ext4 and XFS, and the purpose of each standard directory in the hierarchy.
Intel announced it will cease providing its proprietary Lustre file‑system version, contribute all features to the open‑source community, designate the community build as the long‑term‑stable release, and release related Hadoop and HPC adapters, signaling a strategic shift in its high‑performance‑computing operations.
When power is lost mid‑write, the write command, device caches, and ordering become uncertain, and file systems employ various strategies—ranging from ignoring errors to copy‑on‑write and journaling—to mitigate data loss, each with trade‑offs in performance and reliability.
This guide walks through the concepts of distributed file systems, introduces MogileFS, and provides step‑by‑step instructions—including environment setup, MariaDB and MogileFS configuration, Nginx compilation with the MogileFS module, and testing—to create a scalable small‑file storage solution.
This article examines the inherent risks of distributed storage, emphasizes that data safety outweighs raw performance, and explains storage types, file system structures, media characteristics, and practical solutions such as SSD caching and SRVSAN architecture to mitigate those risks.
This article explains the importance of Linux file and directory permissions, details how permission bits work, demonstrates using octal and symbolic notation with chmod, chown, and chgrp, and shows how umask determines default permissions for newly created files and directories.
This article explains the principles of distributed storage, the CAP theorem and consistency models, compares popular distributed file systems, and provides a detailed overview of MogileFS architecture, features, and operation workflow.
This article provides a comprehensive technical overview of InnoDB's physical file architecture, detailing how fixed‑size blocks, page types, extents, file‑list structures, segment creation, page allocation, recycling mechanisms, index page composition, and both traditional and transparent compression methods work together to manage MySQL storage efficiently.
This article explains how Node.js implements synchronous file system and child‑process APIs on top of its asynchronous libuv core, covering the underlying C/C++ bindings, error handling, debugging techniques, and the practical trade‑offs of using sync methods in real‑world scripts.
This article explains the concepts, usage, and performance characteristics of AUFS, a Union File System used in Linux and Docker, providing examples of mounting, branch permissions, whiteout handling, and configuration options, while discussing its role in container image layering.
This article explains the Unix principle that everything is a file, describes regular and special file types, outlines why and how disks are partitioned, introduces mount points, and details the standard Linux filesystem hierarchy and inode metadata.
This article explains the physical components of hard disk drives, the concepts of tracks, cylinders and sectors, compares MBR and GPT partition schemes, outlines common file system types, and provides practical command‑line examples using fdisk and parted for creating, modifying, and deleting partitions on Linux.
This article explains the concept of an inode, its role in storing file metadata, its size and structure on disk, how inode numbers are used, and the differences between hard links, soft links, and directory entries in Unix/Linux systems.
