Can You Recover Files After an rm -rf Mistake? What Works and What Doesn’t

What rm -rf Actually Does

The rm command removes ordinary files by calling unlink or unlinkat, and removes directories via rmdir. The directory entry is deleted and the inode link count is decreased. When the count reaches zero and no process still holds the file open, the filesystem can reclaim the inode and data blocks. rm marks space as reusable but does not overwrite data immediately.

If a process still has the file open, the kernel keeps the file until the last descriptor is closed.

Each subsequent write may reuse the freed blocks, reducing recovery probability over time.

The -r flag makes the operation recursive, and -f ignores missing files and suppresses prompts. Neither flag provides a recycle bin or undo record.

Common Confusions

Hard links: deleting one name does not delete the data while other links exist.

Soft links: deleting the link only removes the pointer, not the target.

Mount points: recursive deletion may cross into a mounted filesystem depending on the command and directory layout.

Containers: the path may reside on an overlay writable layer, a bind mount, a named volume, or a Kubernetes PersistentVolume, each requiring a different recovery approach.

Network storage (NFS, CephFS, SMB): deletions are persisted on the server side, so local block‑level tools are usually ineffective.

First Five Minutes After Accidental Deletion

2.1 Stop All New Writes Immediately

Notify relevant teams to halt publishing, syncing, building, log archiving, and batch processing. If the deleted directory still receives live writes, isolate the instance or switch the write target before shutting down the service.

Do NOT install recovery tools (e.g., testdisk, extundelete) on the affected filesystem.

Do NOT create placeholder files or redeploy the application in the same path.

Do NOT write recovered files back to the original filesystem.

Do NOT run fsck or xfs_repair as generic undelete tools.

Do NOT execute fstrim; it may trigger TRIM on SSD/NVMe devices, making data unrecoverable.

Do NOT reboot or power‑off the server; this can close open file descriptors and corrupt the filesystem.

2.2 Record Time, Command, and Operator

date -Is
id
pwd
findmnt -T /path/to/deleted

Replace /path/to/deleted with the actual path. Even if the path no longer exists, findmnt -T can often locate the underlying filesystem.

2.3 Identify Filesystem and Underlying Structure

findmnt -T /path/to/deleted -o TARGET,SOURCE,FSTYPE,OPTIONS
lsblk -o NAME,TYPE,FSTYPE,SIZE,MOUNTPOINTS
df -hT /path/to/deleted

Key questions:

Is the filesystem ext4, XFS, Btrfs, ZFS, or a network/distributed FS?

Is the storage local HDD/SSD/NVMe, a hardware/software RAID, or LVM?

Are mount options like discard or periodic fstrim enabled?

Are there pre‑deletion snapshots (LVM, cloud, storage array)?

Is the directory provided by a container volume, bind mount, or PersistentVolume?

Is data encrypted and are the keys still available?

Recovery Path with Highest Success Probability

Application‑level backups, artifact repositories, and object‑storage versioning.

Pre‑deletion filesystem, LVM, cloud‑disk, or storage‑array snapshots.

Files that are still open by a process.

Copies from other nodes, read‑only replicas, caches, or container images.

File‑system‑level or signature‑based recovery on an offline image.

3.1 Verify Backups Are Usable

Ensure the backup timestamp precedes the deletion and that retention policies have not removed the data. For database directories, use the database’s physical or logical backup/restore procedures instead of copying raw files. mkdir -p /recovery/restore-test Restore into an isolated directory on a separate disk; never overwrite the production path directly.

3.2 Snapshots Must Pre‑date Deletion

Post‑deletion snapshots only capture the state after the delete and cannot magically restore lost data. If a pre‑deletion snapshot exists, clone it to a new volume and mount read‑only for extraction.

Be aware of the snapshot’s scope (full volume vs. sub‑volume).

Validate the snapshot before mounting.

3.3 ext4 Specifics

Ext4 may reuse directory entries, inodes, and data blocks quickly. SSD/NVMe devices with online discard or fstrim can zero out blocks, drastically lowering recovery chances.

Tools such as extundelete work only on an unmounted or read‑only image and may not support newer ext4 features.

3.4 XFS Specifics

XFS lacks an official undelete utility. xfs_repair fixes metadata consistency but does not restore deleted files and may alter the on‑disk state.

3.5 Btrfs, ZFS, LVM, Cloud‑Disk Snapshots

Recovery relies on snapshots taken before the delete. Clone the snapshot, mount it read‑only, and extract the needed path. Never roll back a live volume directly; always use a cloned volume.

Open Files via /proc

4.1 Find Deleted but Still Open Files

lsof +L1

Output shows entries with (deleted) in the name and NLINK 0. Copy the file descriptor to another filesystem:

install -d -m 700 /recovery/open-files
cp --sparse=always /proc/18472/fd/5 /recovery/open-files/access.log.recovered
sync /recovery/open-files/access.log.recovered

Replace the PID, FD number, and target path with actual values. Verify the copy with sha256sum and stat.

Block‑Level Imaging

5.1 Create a Safe Image Before Any Further Writes

If no reliable backup or open‑file copy exists, stop all writes, unmount the filesystem, and image the block device.

ddrescue -f -n /dev/mapper/vg-data /recovery/vg-data.img /recovery/vg-data.map
# Second pass with retries
ddrescue -d -f -r3 /dev/mapper/vg-data /recovery/vg-data.img /recovery/vg-data.map

Replace /dev/mapper/vg-data with the actual device discovered via findmnt or lsblk. Store the image on a separate disk.

5.2 Analyze the Image

file /recovery/vg-data.img
blkid -p /recovery/vg-data.img

Confirm the filesystem type (e.g., ext4) before attempting recovery tools.

5.3 Filesystem‑Specific Recovery on the Image

For ext4:

mkdir -p /recovery/extundelete-output
extundelete /recovery/vg-data.img --restore-all --output-dir /recovery/extundelete-output

For XFS, Btrfs, ZFS, use the respective snapshot or clone mechanisms; avoid xfs_repair as a generic undelete tool.

Special Data Types

Database Directories

Never treat database files as ordinary documents. Use physical backups, WAL/redo logs, or replica copies. Recovery steps include:

Preserve the entire volume or block image.

Locate database‑specific backups or logs.

Restore to a new instance using the database’s official procedure.

Validate consistency before switching traffic.

Configuration, Code, and Release Artifacts

Prefer version‑control repositories, configuration centers, CI build records, or artifact registries over filesystem scans. Verify commit IDs, artifact hashes, and environment differences before redeployment.

Log Files

If a log file is still open, copy it via /proc as described earlier. Otherwise, retrieve logs from centralized logging platforms; do not fabricate timestamps.

Media, Archives, and VM Images

Signature‑based tools (e.g., PhotoRec) can recover file fragments but lose original names and metadata. Always verify recovered archives with integrity checks (e.g., tar -tzf, zip -T).

Post‑Recovery Verification and Re‑deployment

Check file count, directory hierarchy, and total size against backup manifests.

Validate hashes (SHA‑256) against trusted sources.

Confirm ownership, permissions, ACLs, extended attributes, and SELinux contexts.

Ensure hard‑ and soft‑link relationships are intact.

Run application‑specific syntax checks on configuration files.

Test data files in read‑only or validation mode before allowing writes.

Perform a dry‑run sync to compare restored data with production targets:

rsync -aHAXn --delete /recovery/restored/ /data/target/

Only after confirming the diff and business impact should the -n flag be removed and the actual sync executed.

Rollback Plan

Keep the original directory or volume untouched until the restored data passes all checks. If the application fails to start, validation errors appear, or data timestamps are mismatched, revert by renaming directories, switching mounts, or rolling back traffic.

Documentation and Auditing

Record every action: operator, timestamp, source device, image hash, tool version, parameters, output location, and result. Compute a SHA‑256 hash of any block image for forensic reproducibility:

sha256sum /recovery/vg-data.img > /recovery/vg-data.img.sha256

Store logs, manifests, and verification reports in a secure, read‑only location for future audits.

Incident Checklist

Record exact time, host, user, command, and target path.

Stop all non‑essential writes; do not install software or reboot.

Identify the real filesystem, device, and any container or network‑storage boundaries.

Search for backups, pre‑deletion snapshots, artifact stores, and other node copies.

Immediately list open deleted files with lsof +L1 and copy them elsewhere.

If data value is high and no reliable copy exists, isolate the node, unmount, and create a block‑level image.

Run all recovery experiments on the image only; output to an isolated disk.

Choose the appropriate tool based on the filesystem (ext4, XFS, Btrfs, ZFS, LVM, etc.).

Validate recovered content, hashes, permissions, timestamps, and application consistency.

Before production restore, perform dry‑run, gray‑scale, and backup of the current state; have a clear rollback procedure.

Post‑mortem: review permissions, cleanup scripts, snapshot policies, audit logging, and recovery drills.