Operations 4 min read

Which Linux Command Deletes Hundreds of Thousands of Files the Fastest?

This article benchmarks several Linux techniques—including rm, find, rsync, Python, and Perl—for deleting 500,000 small files, measuring execution times and revealing that rsync with the --delete option completes the task in under 20 seconds, far outpacing traditional commands.

ITPUB
ITPUB
ITPUB
Which Linux Command Deletes Hundreds of Thousands of Files the Fastest?

Today we test the efficiency of deleting a large number of files on Linux.

Setup: Create 500,000 files

$ for i in $(seq 1 500000); do echo text >> $i.txt; done

Method 1: rm

$ time rm -f *

Result: rm fails with “argument list too long”; the command reports 3.985 s user, 0.29 s system but cannot delete all files.

Method 2: find with -exec rm

$ time find ./ -type f -exec rm {} \;

Result: Approximately 43 minutes (49.86 s user, 1032.13 s system) to delete 500 k files.

Method 3: find with -delete

$ time find ./ -type f -delete

Result: About 9 minutes (0.43 s user, 11.21 s system), total 9:13.38.

Method 4: rsync --delete

# First create an empty directory "blanktest"
$ time rsync -a --delete blanktest/ test/

Result: 0.59 s user, 7.86 s system, total 16.418 s – the fastest among tested methods.

Method 5: Python script

import os, timeit

def main():
    for pathname, dirnames, filenames in os.walk('/home/username/test'):
        for filename in filenames:
            file = os.path.join(pathname, filename)
            os.remove(file)

if __name__ == '__main__':
    t = timeit.Timer('main()', 'from __main__ import main')
    print(t.timeit(1))

Result: Approximately 529 seconds (about 9 minutes) to delete all files.

Method 6: Perl one‑liner

$ time perl -e 'for(<*>) { ((stat)[9] < (unlink)) }'

Result: 1.28 s user, 7.23 s system, total around 16.8 s.

Results Summary

rm: not usable for huge file counts.

find -exec rm: ~43 minutes.

find -delete: ~9 minutes.

rsync --delete: ~16 seconds (fastest).

Perl: ~16 seconds.

Python: ~9 minutes.

Conclusion: Using rsync with the --delete option is the quickest and most convenient way to delete massive numbers of small files on Linux.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

Pythonrsyncperlrmfindfile deletion
ITPUB
Written by

ITPUB

Official ITPUB account sharing technical insights, community news, and exciting events.

0 followers
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.