Which Linux Command Deletes Massive Files Fastest? Benchmark Results

We test the efficiency of deleting a large number of files on Linux.

First, we create 500,000 files:

for i in $(seq 1 500000); do echo text >> $i.txt; done

1. rm time rm -f * Result: rm fails due to "argument list too long". Execution time 3.63s user, 0.29s system, 98% CPU, total 3.985s, but not usable for many files.

2. find with -exec rm time find ./ -type f -exec rm {} \; Result: 49.86s user, 1032.13s system, total 43:19.17 (about 43 minutes).

3. find with -delete time find ./ -type f -delete Result: 0.43s user, 11.21s system, total 9:13.38 (about 9 minutes).

4. rsync --delete time rsync -a --delete blanktest/ test/ Result: 0.59s user, 7.86s system, total 16.418s (about 16 seconds).

5. Python script

import os
import timeit

def main():
    for pathname, dirnames, filenames in os.walk('/home/username/test'):
        for filename in filenames:
            file = os.path.join(pathname, filename)
            os.remove(file)

if __name__ == '__main__':
    t = timeit.Timer('main()', 'from __main__ import main')
    print(t.timeit(1))

Result: approximately 9 minutes.

6. Perl one-liner

time perl -e 'for(<*>) { ((stat)[9] < (unlink)) }'

Result: 1.28s user, 7.23s system, total 16.784s (about 16 seconds), fastest among scripts.

Result summary:

rm: not usable for many files
find -exec: 43 minutes
find -delete: 9 minutes
Perl: 16 seconds
Python: 9 minutes
rsync --delete: 16 seconds

Conclusion: rsync with --delete is the fastest and most convenient way to delete a large number of small files.