find vs locate: Mastering Million-File Search & Safe Cleanup in Linux Operations

Problem Background

File search difficulty varies drastically with scale. A simple find / -name 'app.log' works in seconds on tens of thousands of files, but fails catastrophically on million-file systems: traversing 50 mount points with millions of files can take 40 minutes, saturate I/O, and slow production. Common pitfalls include find crossing NFS mounts and deleting shared archives, -exec rm forking per file and stalling overnight, locate missing recently deleted files due to stale indexes, and inode exhaustion causing "No space left on device" despite free disk space.

Applicable Scenarios

A decision table maps scenarios to tools: use find -name -maxdepth for small directories; locate for frequent pattern searches; locate first then find fallback for million-file trees; find for attribute-based searches (mtime, size, perm, user); df -i plus find for inode exhaustion; find -mtime +N -delete with dry-run and depth limits for log cleanup; find -print0 | xargs -0 for bulk actions; find -mount or locate for cross-mount searches.

Core Concepts

3.1 find: Real-time Directory Traversal

find PATH EXPR

depth-first traverses the entire tree, calling stat on each entry. Complexity is O(total directory entries), making it I/O and syscall intensive. Larger trees are slower; NFS or slow disks amplify latency; crossing many mount points explodes the search space.

3.2 Expression Evaluation & Short-circuit Order

Expressions evaluate left-to-right with -a (AND) short-circuiting. Placing -type f before -name skips string matching on non-regular files, saving CPU but not I/O. Real speedups come from pruning ( -prune), depth limits ( -maxdepth), and filesystem boundaries ( -mount).

3.3 print0 / xargs -0: Safe Pipeline

Filenames with spaces, newlines, or quotes break find -print | xargs. The safe pattern: find /data -name '*.log' -print0 | xargs -0 -r rm -f. -print0 uses NUL delimiter; xargs -0 splits on NUL; -r prevents execution on empty input.

3.4 locate: Index Lookup

locate

queries a prebuilt database ( /var/lib/mlocate/mlocate.db), achieving O(1) speed. Trade-offs: database updated daily via updatedb (cron at 4 AM), so new files are invisible until next update; deleted files linger in index until refresh.

3.5 Inodes & Directory Entries

Each file/directory consumes one inode (fixed at filesystem creation). df -h shows space; df -i shows inode usage. 100% inodes = cannot create files. Directory entries grow with file count; a directory with millions of entries makes ls, find, readdir extremely slow due to sequential reads.

3.6 High-Risk Actions Checklist

find ... -delete

: immediate, no confirmation. find ... -exec rm -f {} \;: per-file fork, slow and dangerous. find / -delete: catastrophic from root. find without -maxdepth on mount-heavy roots: crosses filesystems, may enter NFS. find deleting data under mount points: e.g., cleaning /data/log while NFS archive is mounted there.

Overall Troubleshooting Approach

Six-step workflow: define goal → estimate target scale → choose find/locate → restrict path/depth → prune irrelevant mounts → safe pipeline (print0/xargs) → dry-run (print first) → execute → verify → rollback ready. For search: check inodes, use locate for speed, find as fallback, limit scope, review results before action. For cleanup: dry-run with -print, confirm list, then -delete or xargs -0 rm; better yet, tar or mv to trash before removal.

Practical Steps

5.1 Pre-check: Estimate Target Scale

Quantify before running: df -i for inode headroom; df -h for space; estimate file counts with find /data -maxdepth 1 -type d -printf '%p\n' | head then find /data/app -maxdepth 3 -type f -printf '.' | wc -c. -printf '.' outputs a dot per file, counted by wc -c, faster than printing names. If df -i > 80%, cleanup priority high; if single-directory count takes >30 seconds, consider locate or sharding.

5.2 find Basics: Name, Type, Depth

Examples: exact name -name 'app.log'; case-insensitive -iname; wildcards -name '*.log'; full path -path '/data/app/*/tmp/*'; regex -regextype posix-extended -regex '/data/(app|web)/.*\.log'; types -type f/d/l/b/c/p/s; depth limits -maxdepth 2, -mindepth 2 -maxdepth 4. -maxdepth 1 is a performance fuse for large directories; -path matches full path vs -name matching only basename.

5.3 find by Time

-mtime +30

(modified >30 days ago), -mtime -1 (within 24h), -mmin -60 (within 60 minutes). +N /-N semantics: -mtime +0 means 1+ days ago; -mtime -1 means last 24h. ctime (inode change), atime (access, unreliable with noatime mount). -newer file and -newermt 'YYYY-MM-DD HH:MM:SS' (GNU extension) for absolute time ranges.

5.4 find by Size, Permissions, Owner

Size: -size +100M (>100MiB), -size -1k (<1KiB). Permissions: -perm 644 (exact), -perm -u+w (at least user write), -perm /u+w,g+w (any of these bits). SUID/SGID audit: find / -perm -4000 -type f (SUID), -perm -2000 (SGID), combine with -executable for high-risk. Owner: -user appuser, -group appgrp, -nouser, -nogroup for orphaned files.

5.5 Pruning Irrelevant Directories & Mount Points

Critical for performance and safety. find / -mount -name '*.conf' (or -xdev) stays on starting filesystem. -prune skips subtrees:

find /data -path '/data/log/trash' -prune -o -name '*.log' -print

. Multiple prunes chained with -o. Prune must be left of -o; right side needs explicit -print. More granular than -maxdepth.

5.6 Executing Commands: -exec vs xargs

-exec cmd {} \;

forks per file (slow). -exec cmd {} + batches arguments (faster). xargs -0 -r -n 100 controls batch size, avoids ARG_MAX. Use xargs or + for multi-arg commands ( cp, rm, grep); use \; for single-arg commands. Always -print0 | xargs -0 for special filenames. Add -n N for huge batches.

5.7 find -delete & Safe Cleanup

-delete

is built-in, faster than -exec rm, implies -depth (bottom-up). Dangerous: no confirmation, hidden -depth deletes children before parents. Safe patterns: 1) dry-run find ... -print | tee /tmp/will_delete.list; 2) tar archive then delete: find ... -print0 | tar --null -czf backup.tar.gz -T -; 3) mv to dated trash directory, verify, then rm -rf trash. Always dry-run; never run -delete without -maxdepth and mount pruning.

5.8 locate Usage & Index

Install mlocate (CentOS) or plocate (Ubuntu, faster). sudo updatedb builds index. Queries: locate app.log, locate '*.log' (quote wildcards), locate -i (case-insensitive), locate -b 'app.log' (basename only), locate -r '/data/.*\.log$' (regex), locate -c '*.log' (count), locate -S (index stats). Default matches anywhere in path; use -b for basename.

5.9 updatedb Config & Scheduling

Config in /etc/updatedb.conf (mlocate) or /etc/plocate.conf. PRUNEPATHS excludes directories (tmp, proc, sys, docker, cache). PRUNEFS skips filesystem types (nfs, tmpfs, overlay). PRUNE_BIND_MOUNTS=yes avoids duplicate indexing. Default daily cron ( /etc/cron.daily/mlocate). For fresher index, add cron every 4 hours: 3 */4 * * * /usr/bin/updatedb, but monitor I/O impact. Index size: MB to hundreds of MB. Index readable by all users (paths visible); sensitive paths must be pruned.

5.10 Million-File Inode Exhaustion: Closed-loop Troubleshooting

Symptom: write fails with "No space left on device", df -h shows space, df -i shows 100% inodes. Diagnosis: df -i identifies mount; then find top subdirectories by file count:

sudo find /data -maxdepth 1 -type d -printf '%p\t' -exec sh -c 'find "$1" -xdev -type f -printf "." | wc -c' _ {} \;

. Drill down to level 2/3. Culprit: >1M files in a directory (session files, cache fragments, unrolled logs). Fix: dry-run

find /data/cache -maxdepth 4 -type f -mtime +7 -print | wc -l

, then mv to trash with xargs -0 -r -n 5000 mv -t $TRASH. Verify df -i drops. Root cause fix: reduce file count (Redis for sessions, sharded cache directories, log rotation, temp file cleanup). Last resort: rebuild filesystem with higher inode density ( mkfs.ext4 -i 4096), requiring full backup, maintenance window, dual approval.

Case Studies

Case 1: Log Cleanup Deletes Shared Archive

Running find /data/log -name '*.log' -mtime +30 -delete removed NFS-mounted /data/log/archive (team-shared NAS). Root cause: find crosses filesystems by default; no -mount, -maxdepth, or -prune for archive; no dry-run. Fix:

find /data/log -mount -path '/data/log/archive' -prune -o -name '*.log' -mtime +30 -delete

. Policy: all cleanup scripts code-reviewed, mount points explicitly pruned, dry-run mandatory.

Case 2: find Runs 30 Minutes, No Output

find /data -name 'app.log'

hangs, iostat shows 95% utilization. Process state D (uninterruptible sleep), wchan shows nfs or getdents. Root cause: /data contains slow NFS mounts; find serializes across them. Fix: kill find; use locate app.log for instant result; if missing, find /data/app -mount -maxdepth 4 -name 'app.log' (restrict to local subtree, limit depth). No rollback needed (read-only).

Case 3: Inode Exhaustion Root Cause

df -i

shows 99% on /data. Use du -x --inodes /data | sort -rn | head (GNU coreutils 8.x) or nested find to pinpoint directory with millions of files. Common sources: PHP session files, cache shards, per-request logs, abandoned uploads. Cleanup: dry-run, mv to trash, verify inode recovery. Rollback: move back from trash. Prevention: scheduled cleanup, application-level expiration (Redis), sharded directories.

Risk Reminders

Operation: find ... -delete | Risk: Irreversible | Must Do: Dry-run -print, save list

Operation: find ... -exec rm | Risk: Slow, dangerous | Must Do: Use -delete or xargs -0 rm, dry-run first

Operation: find / unrestricted | Risk: Crosses all mounts, scans everything | Must Do: -mount, -maxdepth, -prune Operation: find without -maxdepth on large tree | Risk: Runs long, saturates I/O | Must Do: Add -maxdepth N Operation: find crosses NFS | Risk: Hangs, impacts business | Must Do: -mount / -xdev, prune mounts

Operation: Spaces in names + plain pipe | Risk: Word splitting, wrong deletions | Must Do: -print0 + xargs -0 Operation: Frequent updatedb | Risk: I/O jitter | Must Do: Every 4 hours, monitor I/O

Operation: rm -rf "$TRASH" | Risk: Wrong path = disaster | Must Do: Hardcode path, no globs, ls confirm

Operation: Rebuild filesystem ( mkfs) | Risk: Wipes data | Must Do: Full backup + maintenance window + dual approval

Operation: find -newermt time ranges | Risk: Timezone/DST errors | Must Do: Use UTC or explicit TZ

Special filenames: spaces, newlines, quotes, leading dash, Unicode → always -print0 / -exec {} + / -delete. Leading dash: use cmd -- {} or cmd .//{}. Locale: set LANG=zh_CN.UTF-8 for Chinese names. Permissions: Permission denied → 2>/dev/null or sudo; but sudo find deleting others' files silently → verify ownership. locate index world-readable → sensitive paths in PRUNEPATHS.

Monitoring & Observability

8.1 find Process State

ps -eo pid,ppid,etime,pcpu,pmem,comm,args | grep '[f]ind'

. etime >30 min → likely stuck. strace -p $PID -f -e trace=openat,stat,getdents shows current syscall. ls -l /proc/$PID/cwd reveals current directory. cat /proc/$PID/wchan shows kernel wait (e.g., getdents, cached_lookup).

8.2 Disk I/O Impact

iostat -x 1 5

: watch %util, await, r/s, w/s. %util > 80% sustained → business impacted. Mitigate: ionice -c 3 -p $PID (idle I/O class). If still heavy, kill and re-run with stricter -maxdepth / -prune. pidstat -d 1 5 for per-process I/O.

8.3 Inode Usage Monitoring (Prometheus/node_exporter)

Metrics: node_filesystem_files (total inodes), node_filesystem_files_free (free). Usage = 1 - free/total. Alert thresholds: >80% warning, >90% critical; growth rate >5%/day → possible small-file leak.

Verification Methods

13.1 Verify find Expression Semantics

Dry-run complex expressions:

find /data -path '/data/log/trash' -prune -o -name '*.log' -mtime +30 -print

. Confirm trash excluded, others included.

13.2 Verify Cleanup List

find /data/log -mount -maxdepth 3 -name '*.log' -mtime +30 -print > /tmp/clean.list; wc -l /tmp/clean.list; head/tail /tmp/clean.list

. Manual spot-check paths.

13.3 Verify locate Index Freshness

sudo touch /data/test_marker_$(date +%s); locate test_marker_... || echo 'index stale'; sudo updatedb; locate test_marker_...

. Confirms index lag and update behavior.

Rollback Strategies

14.1 Cleanup Rollback: Trash Method

Always mv to dated trash ( /data/trash/YYYY-MM-DD_HHMM), observe, then rm -rf trash. Recovery:

find "$TRASH" -print0 | xargs -0 -r -n 5000 mv -t /original/path/

.

14.2 find -delete Rollback

Once -delete executes, filesystem-level rollback impossible without snapshots (LVM, ZFS, NAS), offsite backup, or NAS recycle bin. Hence trash-first policy.

14.3 Index Config Rollback

sudo cp /etc/updatedb.conf.bak /etc/updatedb.conf; sudo updatedb

.

Production Guidelines

All cleanup scripts code-reviewed; mount points explicitly pruned; dry-run saves list.

Root searches default -mount.

Filenames always -print0 / -exec {} + / -delete; no plain pipes.

Large trees: prefer locate; find fallback with -maxdepth.

Monitor inode usage; alert >80%; regularly hunt small-file leaks. updatedb every 4 hours, off-peak, monitor I/O.

NFS mount points in cleanup scripts explicitly -prune; never let find cross NFS for deletion.

High-risk rm via trash, observe, then remove.

Filesystem rebuild: full backup + window + dual approval.

Cleanup scripts log timestamp, path, count, operator for audit.

Batch Cleanup Script (Production-Ready)

Script /usr/local/bin/clean_old_logs.sh with: set -uo pipefail; configurable target dir, depth, pattern, days, trash base, log file, prune paths array; pre-check filesystem type (reject NFS/CIFS); builds -prune expressions; dry-run mode ( --dry-run) prints list and exits; real run moves files to timestamped trash via

while read -d '' f; do mv -- "$f" "$TRASH/"; done < <(find ... -print0)

; logs every step; verifies df -i after; prompts manual rm -rf on trash. Cron: 13 3 * * * /usr/local/bin/clean_old_logs.sh after successful dry-run.

Parallel Search for Million-File Speedup

Single-threaded find serializes stat calls. Shard top-level directories:

for sub in /data/app*; do find "$sub" -mount -maxdepth 5 -name '*.log' -mtime +30 -print & done; wait

. Or

find /data -maxdepth 1 -mindepth 1 -type d -print0 | xargs -0 -P 8 -I{} sh -c 'find "$1" -mount -type f -name "*.log" -mtime +30 -print' _ {} > result.list

. Parallelism = CPU cores; balance shard sizes; merge with sort -u. Warning: parallel I/O may saturate disks; use ionice -c 3 and monitor.

Performance Comparison & Selection

Benchmark estimates (million files, ext4, SSD): full find /data -name x → tens of minutes to hours, high I/O, not recommended; scoped find /data/app -mount -maxdepth 5 → seconds to minutes, medium I/O, known area; locate x → <1 sec, negligible I/O, name search with fresh index; parallel sharded find → faster but high I/O, urgent with I/O headroom; du --inodes → minutes, medium I/O, inode hunting. Selection heuristic: name search → locate (if index fresh); attribute search with known scope → find + -mount -maxdepth -prune; unknown location → du --inodes / df -i to narrow, then find; million-file attribute filter → root cause is too many files; temporary parallel + ionice.

Tool Comparison: find vs locate vs fd

fd

(modern find alternative): parallel, ignores .git /hidden by default, faster, but weaker expression power; not drop-in for scripts relying on GNU find semantics. ag / rg (content search) solve different problem. Production scripts: find for portability and full semantics; locate for interactive quick lookups; fd for interactive use.

Common Errors & Fixes

Symptom: missing argument to -name | Cause: Unquoted wildcard expanded by shell | Fix: find . -name '*.log' Symptom: paths must precede expression | Cause: Expression before path | Fix: Path first: find /data -name x Symptom: Many Permission denied | Cause: No read permission | Fix: 2>/dev/null or sudo Symptom: -delete: Directory not empty | Cause: Directory non-empty, no -type f | Fix: Add -type f or rely on -delete 's implicit -depth Symptom: xargs: argument list too long | Cause: Too many files exceed ARG_MAX | Fix: xargs -n N batch

Symptom: locate: can not stat mlocate.db | Cause: Index not built | Fix: sudo updatedb Symptom: locate shows deleted file | Cause: Index stale | Fix: sudo updatedb then query

Symptom: find runs long, state D | Cause: Stuck on I/O/NFS | Fix: Kill, add -mount, use locate

Symptom: -printf unsupported | Cause: Non-GNU find (BSD/macOS) | Fix: Use -ls or install findutils ( gfind)

Symptom: du --inodes invalid option | Cause: Old coreutils | Fix: Upgrade or use nested find counting

Troubleshooting mindset: syntax errors → quotes/argument order; permission → sudo/suppress; version gaps → GNU tools or upgrade; performance → ps / strace / iostat to locate bottleneck (CPU/I/O/NFS), then kill/throttle/switch tool.

Inode Root Cause Fixes

21.1 Reduce File Count (Preferred)

Replace file-based sessions with Redis.

Shard cache directories (e.g., ab/abcdef two-level).

Aggregate logs by time (daily/hourly), not per request.

Add TTL cleanup for upload temp files.

Single directory >100k files degrades ls / find; >1M severe; must shard.

21.2 Rebuild Filesystem with Higher Inode Density (High Risk)

mkfs.ext4 -i 4096 -b 4096 /dev/sdb1

(one inode per 4KB instead of default 16KB). Steps: full backup → unmount → mkfs → restore → remount → verify df -i → business validation. Requires maintenance window, dual approval, tested rollback (backup exists).

Mount Point & Filesystem Boundary Handling

mount | column -t

, findmnt list mounts. findmnt -n /path checks if path is mount point. find /data -mount -name '*.log' stays on local FS. find / -fstype ext4 -name '*.conf' filters by FS type. Rules: any find from root or large parent defaults to -mount unless cross-mount intended; cleanup scripts blacklist all mount subpaths in PRUNE_PATHS; use findmnt -n in pre-checks.

Security Auditing with find

Baseline SUID:

sudo find / -xdev -type f -perm -4000 2>/dev/null > /var/baseline/suid_$(date +%F).list

. Periodic diff: diff baseline current → alert on new entries (possible backdoor). World-writable files in /etc, /usr/bin, /sbin: sudo find /etc /usr/bin /usr/sbin -xdev -type f -perm -0002. Orphaned files: sudo find / -xdev -nouser -o -nogroup → chown or remove.

Automated Patrol Script (inode + SUID)

Script /usr/local/bin/fs_patrol.sh runs every 6 hours ( 33 */6 * * *): checks df -i for >80% inode usage; diffs SUID list against baseline; scans /etc, /usr/bin, /usr/sbin for world-writable files. Logs to /var/log/fs_patrol.log; alerts via monitoring on keywords.

Cleanup Verification Script

/usr/local/bin/clean_verify.sh <dir>

captures before/after: df -i, file count ( find -xdev -type f -printf . | wc -c), directory size ( du -shx). Comparison validates cleanup effectiveness: inode drop ≈ small files removed; size drop ≈ total bytes freed; no change → expression missed.

Appendices

A. find Recipes (High-Frequency Scenarios)

Recent config changes: find /etc /data/app -mount -mmin -10 -type f with -printf '%TY-%Tm-%Td %TH:%TM %p\n' | sort.

Top 20 largest files:

find /data -mount -type f -printf '%s\t%p
' | sort -rn | head -20

.

Empty files/dirs: find /data -mount -empty (or -type d -empty, -type f -empty).

Change ownership: dry-run find /data -mount -user olduser -print, then -exec chown newuser:newgroup {} +.

Tighten 777 files: find /data -mount -type f -perm 0777 -exec chmod 644 {} + (preserve executables separately).

Broken symlinks: find /data -mount -xtype l (GNU) or ! -exec test -e {} \; -print.

Time-range archive:

find /data/log -mount -maxdepth 2 -name '*.log' -newermt '2026-06-01' ! -newermt '2026-07-01' -print0 | tar -czf /backup/log_202606.tar.gz --null -T -

.

Recent access (atime):

find /data -mount -atime -1 -type f -printf '%AT %p
' | sort | tail -20

(note noatime mount).

FS-type specific: find / -xdev -fstype ext4 -type f -name '*.conf'.

Exclude multiple dirs:

find /data \( -path '/data/cache' -o -path '/data/tmp' -o -path '/data/trash' \) -prune -o -type f -name '*.log' -print

.

B. locate Deep Dive & Pitfalls

Index at /var/lib/mlocate/mlocate.db or /var/lib/plocate/plocate.db. locate -S shows file/dir counts. time sudo updatedb measures update cost (1-5 min on million files, I/O jitter). Multiple indexes:

sudo updatedb -o /data/var/app.db -U /data/app --database-root /data/app

; query with locate -d /data/var/app.db -i config. Security: mlocate index world-readable (paths exposed); plocate uses setuid binary for permission filtering, safer. Sensitive paths → PRUNEPATHS.

C. GNU find vs BSD find (macOS) Differences

GNU supports -printf, -newermt, -regextype, -delete; BSD lacks -printf / -newermt, uses -E for extended regex. Both support -maxdepth, -mount / -xdev. Cross-platform scripts: stick to POSIX subset ( -name, -type, -mtime, -maxdepth, -print, -exec); detect GNU with find --version or install findutils ( gfind) on macOS.

D. find Expression Precedence & Parentheses

-a

(AND) binds tighter than -o (OR). find /data -name '*.log' -o -name '*.txt' -mtime +30 -print means (-name '*.log') -o (-name '*.txt' -a -mtime +30 -a -print) → all .log printed, .txt filtered. Correct grouping:

find /data \( -name '*.log' -o -name '*.txt' \) -mtime +30 -print

. Parentheses escaped \( \) for shell. Always dry-run complex expressions.

E. Million-File Decision Flowchart (Text)

Need to find file?
├─ By name?
│  ├─ Index fresh (<4h) → locate (seconds)
│  └─ Index stale/missing → updatedb then locate, or find scoped subdir
├─ By attribute (time/size/perm/owner)?
│  └─ find, MUST add -mount -maxdepth -prune
└─ Unknown, diagnose first?
   ├─ df -i → inode full → du --inodes find big dir → clean/fix root
   ├─ df -h → space full → find -size +1G find large files
   └─ Business slow → ps/strace check for find hogging I/O

F. find & xargs Advanced

Control concurrency & batch: find ... -print0 | xargs -0 -r -n 1000 -P 4 process_cmd (1000 files per batch, 4 parallel). -P suits light commands; heavy I/O commands need caution. Handle leading-dash filenames: find ... -print0 | xargs -0 rm --. -exec ... {} + batches like xargs but within find; \; forks per file. For content search, grep -r better than find -exec grep.

G. Cleanup with Metrics (Prometheus)

Script emits textfile metrics: log_cleanup_files_removed, log_cleanup_inode_pct_before, log_cleanup_inode_pct_after to /var/log/cleanup_metric.prom for node_exporter textfile collector.

H. locate Ghost Files (Stale Index)

Symptom: locate config.yaml shows /data/app/old/config.yaml but cat says missing. Cause: index lag (file deleted after last updatedb). Fix: sudo updatedb; ghost disappears. Prevention: don't trust locate for just-changed files; use find; increase updatedb frequency (2-4h) with I/O trade-off; script verification:

for f in $(locate ...); do [[ -e "$f" ]] && echo "exists: $f"; done

(use -print0 / while read for spaces).

I. find Stuck on NFS: Full Remediation Loop

Symptom: find /data -name 'report.xlsx' 20 min, state D, NFS slowness reported. Diagnose: ps -eo pid,etime,stat,wchan:25,args | grep '[f]ind /data' (stat D, wchan nfs); mount | grep nfs; nfsstat -c. Root cause: multiple NFS mounts under /data, find serializes across them, one slow NFS blocks. Fix: kill -9 (may need lazy umount if stuck); re-run find /data -mount -name 'report.xlsx' (local only); if file might be on NFS, search that mount alone in background with timeout. Prevention: default -mount on all top-level finds; separate NFS searches with timeout; monitor NFS health, block cross-NFS finds when degraded.

J. Gradual Rollout & Monitoring for Cleanup Jobs

Phased approach: 1) dry-run; 2) gray release with relaxed retention (e.g., 90 days) on subset; 3) tighten stepwise (60→45→30 days); 4) enable cron. Metrics: files removed, space freed, inode drop %, duration, success/fail, post-cleanup inode trend. Alert on consecutive failures; if business reports missing files, restore from trash, audit expression.

K. Disk Space Troubleshooting Closed Loop

df -h

shows 95% on /data. Steps: 1) sudo du -hx --max-depth=1 /data | sort -rh | head (top dirs, -x no cross-mount); 2) drill into largest: du -hx --max-depth=1 /data/app | sort -rh | head; 3) largest files:

find /data/app -mount -type f -printf '%s\t%p
' | sort -rn | head -20

; 4) recent large files:

find /data/app -mount -type f -size +500M -mtime -7 -printf '%s\t%TY-%Tm-%Td\t%p
' | sort -rn

. If du vs df mismatch → deleted but open files: sudo lsof +L1 or sudo find /proc/*/fd -lname '*(deleted)'. Fix: restart holding process (e.g., log service). Verify df -h /data.

L. Filename Encoding

Chinese/Unicode names behave differently under locales. locale shows current; C locale shows ? and breaks matching. Fix: script header export LANG=zh_CN.UTF-8 (or en_US.UTF-8); ensures cron environment matches. Downstream commands (grep/sort) need same locale.

M. Cleanup Script Permissions & Audit

Script owned by root, mode 700. Sudoers restricted:

opsuser ALL=(root) NOPASSWD: /usr/local/bin/clean_old_logs.sh

in /etc/sudoers.d/clean_logs (440). Log file append-only: sudo chattr +a /var/log/clean_old_logs.log. Auditd rule:

-a always,exit -F path=/usr/local/bin/clean_old_logs.sh -F perm=x -k clean_logs

; query ausearch -k clean_logs for who/when.

N. Operations Self-Checklist

[ ] All cleanup scripts use trash + dry-run, no direct -delete [ ] Scripts have -mount; NFS/CIFS mounts in prune blacklist

[ ] Filenames via -print0 / -exec {} +, no plain pipes

[ ] Root/large-tree finds default -mount -maxdepth [ ] locate update frequency tuned, I/O monitored

[ ] Inode usage monitored, >80% alert

[ ] SUID/world-writable baselines established, periodic diff

[ ] Script perms 700, sudoers scoped, logs append-only

[ ] Cleanup jobs gradual (relaxed → tight), metrics collected

[ ] Script locale fixed UTF-8

[ ] FS rebuild has full backup + dual approval

[ ] Cleanup runbook written & rehearsed

[ ] Disaster recovery: snapshots/offsite backup/NAS recycle bin

[ ] New hires complete find cleanup drill

O. One-Liner Cheat Sheet

# Recent config changes (10 min)
find /etc /data/app -mount -mmin -10 -type f
# Top 20 largest files
find /data -mount -type f -printf '%s\t%p
' | sort -rn | head -20
# 30-day old logs (dry-run)
find /data/log -mount -maxdepth 3 -name '*.log' -mtime +30 -print
# SUID files
sudo find / -xdev -type f -perm -4000 2>/dev/null
# Broken symlinks
find /data -mount -xtype l
# Inode hunt
df -i; sudo du -x --inodes /data 2>/dev/null | sort -rn | head
# locate quick search
locate -i config.yaml
# Deleted but held files
sudo lsof +L1 2>/dev/null | head
# No-cross-fs search
find /data -mount -name 'app.log'
# Safe bulk delete (mv to trash)
find /data/log -mount -maxdepth 3 -name '*.log' -mtime +30 -print0 | xargs -0 -r mv -t /data/trash

Summary

find

and locate represent two retrieval paradigms: find real-time traversal with rich attribute filtering but slow on large trees and I/O heavy; locate index-based, instant name search but stale and attribute-blind. Million-file reality: the answer isn't "faster find" but: 1) prefer locate for name lookups; 2) find must have -mount, -maxdepth, -prune to limit scope; 3) high-risk cleanup uses trash + dry-run, never direct -delete; 4) inode exhaustion is the root problem — find culprit directory, clean expired, add expiration logic, rebuild FS with higher inode density if needed. Discipline: restrict path, restrict depth, prune mounts, safe pipeline, print before action, archive list, rollback ready. Seven rules followed, find won't page you at 3 AM. One-liner principle: if locate works, don't find full disk; if -mount works, don't cross mounts; if trash works, don't -delete; if dry-run works, don't execute blindly. Million-scale isn't find's home turf — it's "reduce file count + index fast lookup + controlled cleanup" territory. Post this principle atop every cleanup script; it beats any documentation. Final litmus test before hitting Enter on a production find: will it cross unexpected mounts? Can I recover deletions in 5 minutes? Will it slow the business? Three yeses → proceed.