Which Linux Command Deletes Hundreds of Thousands of Files the Fastest?
This article benchmarks several Linux techniques—including rm, find, rsync, Python, and Perl—for deleting 500,000 small files, measuring execution times and revealing that rsync with the --delete option completes the task in under 20 seconds, far outpacing traditional commands.
Today we test the efficiency of deleting a large number of files on Linux.
Setup: Create 500,000 files
$ for i in $(seq 1 500000); do echo text >> $i.txt; doneMethod 1: rm
$ time rm -f *Result: rm fails with “argument list too long”; the command reports 3.985 s user, 0.29 s system but cannot delete all files.
Method 2: find with -exec rm
$ time find ./ -type f -exec rm {} \;Result: Approximately 43 minutes (49.86 s user, 1032.13 s system) to delete 500 k files.
Method 3: find with -delete
$ time find ./ -type f -deleteResult: About 9 minutes (0.43 s user, 11.21 s system), total 9:13.38.
Method 4: rsync --delete
# First create an empty directory "blanktest"
$ time rsync -a --delete blanktest/ test/Result: 0.59 s user, 7.86 s system, total 16.418 s – the fastest among tested methods.
Method 5: Python script
import os, timeit
def main():
for pathname, dirnames, filenames in os.walk('/home/username/test'):
for filename in filenames:
file = os.path.join(pathname, filename)
os.remove(file)
if __name__ == '__main__':
t = timeit.Timer('main()', 'from __main__ import main')
print(t.timeit(1))Result: Approximately 529 seconds (about 9 minutes) to delete all files.
Method 6: Perl one‑liner
$ time perl -e 'for(<*>) { ((stat)[9] < (unlink)) }'Result: 1.28 s user, 7.23 s system, total around 16.8 s.
Results Summary
rm: not usable for huge file counts.
find -exec rm: ~43 minutes.
find -delete: ~9 minutes.
rsync --delete: ~16 seconds (fastest).
Perl: ~16 seconds.
Python: ~9 minutes.
Conclusion: Using rsync with the --delete option is the quickest and most convenient way to delete massive numbers of small files on Linux.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
ITPUB
Official ITPUB account sharing technical insights, community news, and exciting events.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
