Recover Accidentally Deleted Linux Files: Kernel Secrets & Practical Tools

Kernel‑level deletion mechanism

When rm is executed the kernel performs:

Unlink : remove the directory entry that maps the filename to an inode.

Link‑count decrement : decrease the inode's link count.

Data‑block release : if the link count reaches zero and no process holds an open file descriptor, the data blocks are marked free.

Inode reclamation : the inode becomes available for reuse.

Data blocks are not overwritten immediately; they remain recoverable until new data overwrites them. Hard links share the same inode, so deleting one link does not erase the data. Soft links are independent files that store the target path and become dangling when the target is removed. If a process keeps an open file descriptor, the file data stays on disk and can be accessed via /proc/<pid>/fd/<fd>. The command lsof +L1 lists such deleted‑but‑still‑open files.

Recovery tools for ext* filesystems

extundelete

extundelete recovers files from ext3/ext4 by scanning the journal and the inode table. It works because deleted inodes are not immediately reclaimed and their data blocks may still be intact.

# Install (CentOS/DNF)
sudo dnf install extundelete -y
# Install (Ubuntu/apt)
sudo apt update && sudo apt install extundelete -y
# Build from source (latest version)
cd /tmp
wget https://nchc.dl.sourceforge.net/project/extundelete/extundelete/0.2.4/extundelete-0.2.4.tar.bz2
tar xjf extundelete-0.2.4.tar.bz2
cd extundelete-0.2.4
./configure && make && sudo make install
extundelete --version

Typical workflow

# Create a test image (do not run on production disks)
sudo dd if=/dev/zero of=/tmp/testdisk.img bs=1M count=100
sudo mkfs.ext4 -F /tmp/testdisk.img
sudo mount -o loop /tmp/testdisk.img /mnt/testmount
# Populate data
mkdir -p /mnt/testmount/data
echo "Important config" | sudo tee /mnt/testmount/data/config.yaml
echo "DB password: MySecretPass123" | sudo tee /mnt/testmount/data/db.conf
# Simulate accidental deletion
sudo rm -rf /mnt/testmount/data/*
sudo umount /mnt/testmount
# Recover with extundelete
sudo extundelete /tmp/testdisk.img --restore-all
sudo extundelete /tmp/testdisk.img --restore-file data/config.yaml
sudo extundelete /tmp/testdisk.img --restore-directory /data

Recovery principle : extundelete reads the ext4 journal to locate delete entries and scans the inode table for entries marked "deleted" whose data blocks have not been overwritten.

testdisk and photorec combination

Installation

# CentOS/RHEL
sudo dnf install testdisk -y
# Ubuntu/Debian
sudo apt install testdisk -y
# Verify
testdisk --version
photorec --version

testdisk – partition table recovery

Run sudo testdisk /dev/sdX and follow the interactive steps: select disk → partition table type → Analyse → Quick Search → Write → run partprobe to reload the table.

photorec – signature based file recovery

Photorec ignores filesystem metadata and recovers files by their signatures. It is useful when the filesystem is severely damaged.

# Interactive mode
sudo photorec /dev/sdX1
# Batch mode (example)
sudo photorec /d /mnt/recovered /log photorec.log /cmd /dev/sdX1 options,search

debugfs recovery for ext4

debugfs is an interactive debugger from the e2fsprogs suite.

# Verify installation
which debugfs
debugfs -V
# Open the filesystem (read‑only is safer)
sudo debugfs -w /tmp/testdisk.img
# Core commands (example)
debugfs:  ls -d /data                # list deleted inodes
debugfs:  stat <1310731>            # show inode details
debugfs:  rdump /data/config.yaml /tmp/recovered/   # extract file
debugfs:  cat <1310731> > /tmp/recovered/config.yaml   # alternative extraction
debugfs:  quit

A sample batch script iterates over recent inodes and attempts rdump for each.

XFS filesystem recovery

xfsrestore – backup‑based recovery

# Install xfsdump (provides xfsrestore)
sudo dnf install xfsdump -y
# Create a backup image
sudo xfsdump -l 0 -f /backup/xfs_backup.img /dev/sda1
# Restore a single file or the whole filesystem
sudo xfsrestore -f /backup/xfs_backup.img -i /mnt/restored
sudo xfsrestore -f /backup/xfs_backup.img -i   # interactive

xfs_repair – emergency repair

# Check metadata consistency
sudo xfs_check /dev/sda1
# Repair (requires unmount)
sudo xfs_repair -v /dev/sda1
# Repair a corrupted log
sudo xfs_repair -L /dev/sda1

Advanced recovery techniques

Disk imaging and data fidelity

Always create a raw image before any recovery work to avoid further writes.

# Full disk image (preserve all sectors)
sudo dd if=/dev/sdb of=/backup/sdb_raw.img bs=4M status=progress
# Compressed image
sudo dd if=/dev/sdb bs=4M | gzip -9 > /backup/sdb.img.gz
# Restore from compressed image
sudo gzip -dc /backup/sdb.img.gz | sudo dd of=/dev/sdb bs=4M status=progress
# Image a single partition
sudo dd if=/dev/sdb1 of=/backup/sdb1.img bs=4M status=progress

Ext4 journal replay

Inspect journal entries to understand which inodes and blocks were modified before deletion.

sudo debugfs -R "logdump -a" /tmp/testdisk.img

Manual inode and block recovery

Calculate the block group and offset of a target inode, then extract the raw inode with dd and analyse it (e.g., with hexdump).

# Example script (simplified)
DEVICE="/tmp/testdisk.img"
TARGET_INODE=1310731
INODE_SIZE=256
INODES_PER_GROUP=8192
BLOCK_GROUP=$(( (TARGET_INODE - 1) / INODES_PER_GROUP ))
INODE_OFFSET=$(( (TARGET_INODE - 1) % INODES_PER_GROUP * INODE_SIZE ))
INODE_ADDR=$(( 1024 + BLOCK_GROUP * 8192 * 4096 + 2 * 4096 + INODE_OFFSET ))
sudo dd if=$DEVICE of=/tmp/inode_data.bin bs=1 skip=$INODE_ADDR count=$INODE_SIZE 2>/dev/null
hexdump -C /tmp/inode_data.bin | head -20

Fragment identification and reassembly

When a file is partially overwritten, identify its block fragments before attempting reconstruction.

# Show fragment layout (ext4)
sudo debugfs -R "frag /data/largefile.dat" /tmp/testdisk.img
# Check fragmentation ratio
sudo e4defrag -c /mnt/testmount
# Map physical blocks
sudo debugfs -R "bmap data/largefile.dat 0" /tmp/testdisk.img

Prevention strategy system

Interactive deletion protection

# Add safe aliases to ~/.bashrc
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Optional warning alias for dangerous removals
alias dangerous-rm='echo "WARNING: This will delete data!" && rm'

Trash implementation

A Bash wrapper moves files to $HOME/.trash, logs actions, enforces size limits, and provides a restore function.

#!/bin/bash
TRASH_DIR="$HOME/.trash"
LOG_FILE="$TRASH_DIR/.trash_log"
MAX_TRASH_SIZE=10G
RETENTION_DAYS=30
[ ! -d "$TRASH_DIR" ] && mkdir -p "$TRASH_DIR"
[ ! -f "$LOG_FILE" ] && touch "$LOG_FILE"

safe_delete() {
  local file="$1"
  local ts=$(date +%s)
  local name="${ts}_$(basename "$file")"
  mv "$file" "$TRASH_DIR/$name"
  echo "$ts|$file|$name" >> "$LOG_FILE"
  # Size check (simplified)
  local cur=$(du -sb "$TRASH_DIR" | cut -f1)
  local max=$((MAX_TRASH_SIZE*1024*1024*1024))
  if [ $cur -gt $max ]; then
    echo "Trash exceeds $MAX_TRASH_SIZE, consider cleaning."
  fi
}

restore() {
  local name="$1"
  local entry=$(grep "|$name$" "$LOG_FILE")
  [ -z "$entry" ] && { echo "Not found in trash log"; return 1; }
  local orig=$(echo "$entry" | cut -d'|' -f2)
  mv "$TRASH_DIR/$name" "$orig"
  sed -i "/^$entry$/d" "$LOG_FILE"
}

Filesystem snapshots

LVM snapshots (read‑only):

# Create a 10 GB snapshot of /dev/vg00/lv_data
sudo lvcreate -L 10G -s -n data_snap /dev/vg00/lv_data
sudo mount -o ro /dev/vg00/lv_data_snap /mnt/snapshot
# Merge snapshot back after verification
sudo umount /mnt/snapshot
sudo lvconvert --merge /dev/vg00/lv_data_snap

Btrfs subvolume snapshots :

# Writable snapshot
sudo btrfs subvolume snapshot /data /data/snap_$(date +%Y%m%d)
# Read‑only snapshot
sudo btrfs subvolume snapshot -r /data /data/ro_snap_$(date +%Y%m%d)

Real‑time sync and backup

A Bash script uses inotifywait to watch a source directory and synchronises changes to a remote host via rsync.

#!/bin/bash
SOURCE_DIR="/data"
BACKUP_SERVER="192.168.1.100"
BACKUP_DIR="/backup/data"
SSH_PORT=22
INOTIFY=/usr/bin/inotifywait

install_dependencies() {
  command -v inotifywait >/dev/null || sudo dnf install -y inotify-tools || sudo apt install -y inotify-tools
}

initial_sync() {
  rsync -avz -e "ssh -p $SSH_PORT" --delete "$SOURCE_DIR/" "backup@$BACKUP_SERVER:$BACKUP_DIR/"
}

incremental_sync() {
  $INOTIFY -m -r -e create,modify,delete,move --format '%w%f' "$SOURCE_DIR" |
  while read file; do
    rsync -avz -e "ssh -p $SSH_PORT" --delete "$SOURCE_DIR/" "backup@$BACKUP_SERVER:$BACKUP_DIR/"
  done
}

case "$1" in
  install) install_dependencies ;;
  full)    initial_sync ;;
  watch)   initial_sync && incremental_sync ;;
  *)       echo "Usage: $0 {install|full|watch}" ;;
esac

Common failure scenarios and troubleshooting

Log file deletion (e.g., Nginx)

Identify the deleted but still‑open file with lsof +L1, then copy its contents from /proc/<pid>/fd/<fd> or recreate the log file and signal the daemon to reopen it.

# Find the deleted log descriptor
lsof +L1 | grep nginx
# Example: pid 12345, fd 4
sudo cp /proc/12345/fd/4 /var/log/nginx/access.log.backup
sudo mv /var/log/nginx/access.log.backup /var/log/nginx/access.log
sudo chown www-data:www-data /var/log/nginx/access.log
sudo kill -HUP 12345   # reload nginx

Database file deletion (MySQL)

When a data file is removed but the server is still running, use lsof +L1 to locate the open descriptor, copy the raw file from /proc, then restart MySQL after taking an emergency dump.

# Locate deleted ibd file
lsof +L1 | grep mysql
# Assume pid 1234, fd 5
sudo cp /proc/1234/fd/5 /var/lib/mysql/mysql/user.ibd
sudo chown mysql:mysql /var/lib/mysql/mysql/user.ibd
sudo systemctl restart mysql

Bulk accidental deletion

Immediately stop all write processes ( pkill -STOP), record the incident time, mount any recent LVM snapshots, and begin recovery from the snapshot or from a disk image.

Recovery evaluation and verification

File integrity check

#!/bin/bash
RECOVERED_DIR="/tmp/recovered"
ORIGINAL_MD5="e99a18c428cb38d5f260853678922e03"   # example hash
for file in $(find "$RECOVERED_DIR" -type f); do
  fname=$(basename "$file")
  md5=$(md5sum "$file" | cut -d' ' -f1)
  echo "File: $fname"
  echo "MD5:  $md5"
  if [ -n "$ORIGINAL_MD5" ] && [ "$md5" = "$ORIGINAL_MD5" ]; then
    echo "Status: ✓ Match"
  else
    echo "Status: ⚠ No match or unknown reference"
  fi
  echo "---"
done
# Directory structure comparison (example)
echo "=== Original vs Recovered ==="
ls -la /mnt/testmount/data/
echo ""
ls -la "$RECOVERED_DIR"/data/ 2>/dev/null || ls -la "$RECOVERED_DIR"/

Recovery success‑rate statistics

#!/bin/bash
ORIGINAL_COUNT=$(find /mnt/testmount/data -type f | wc -l)
RECOVERED_COUNT=$(find ./RECOVERED_FILES -type f | wc -l)
echo "Original files: $ORIGINAL_COUNT"
echo "Recovered files: $RECOVERED_COUNT"
printf "Recovery rate: %.2f%%
" $(echo "scale=4; $RECOVERED_COUNT*100/$ORIGINAL_COUNT" | bc)
# File‑type distribution
echo "
=== File type distribution ==="
echo "Original:"; find /mnt/testmount/data -type f | sed 's/.*\.//' | sort | uniq -c
echo "Recovered:"; find ./RECOVERED_FILES -type f 2>/dev/null | sed 's/.*\.//' | sort | uniq -c

Best‑practice checklist

Deletion does not equal overwriting; data remains recoverable until overwritten.

Act immediately after accidental deletion – the earlier the intervention, the higher the success rate.

Create a raw disk image before any recovery attempt to preserve data fidelity.

Implement proactive protection: regular snapshots (LVM/Btrfs), automated backups, and interactive command aliases.

Quick recovery command reference

extundelete /dev/sda1 --restore-all
extundelete /dev/sda1 --restore-file path/to/file
testdisk /dev/sdb
photorec /dev/sdb1
debugfs -w /dev/sda1
lsof +L1
dd if=/dev/sda of=/backup/sda.img bs=4M status=progress
xfs_repair -L /dev/sda1