Mastering rsync: How to Exclude Files and Directories Efficiently

Introduction

Rsync is a fast and versatile command-line utility that synchronizes files and directories between two locations via a remote shell.

It can mirror data, create incremental backups, and copy files, and you may need to exclude certain files or directories based on name or location.

Preparation

Brief overview of rsync workflow and usage. The -a option recursively syncs directories, preserving links, timestamps, groups, ownership, and permissions.

When excluding, use paths relative to the source directory. Two options exist to specify exclusions:

Use --exclude on the command line.

Use --exclude-from with a file containing patterns.

Excluding specific files

Pass the relative path to --exclude. Example:

rsync -a --exclude 'file.txt' src_directory/ dst_directory/

Excluding specific directories

Pass the directory’s relative path to --exclude. Example:

rsync -a --exclude 'dir1' src_directory/ dst_directory/

To exclude the contents but keep the directory itself, use dir1/* instead of dir1:

rsync -a --exclude 'dir1/*' src_directory/ dst_directory/

Excluding multiple files or directories

Specify multiple --exclude options, or combine them with braces:

rsync -a --exclude 'file1.txt' --exclude 'dir1/*' --exclude 'dir2' src_directory/ dst_directory/

Or use a single --exclude with a brace list:

rsync -a --exclude={'file1.txt','dir1/*','dir2'} src_directory/ dst_directory/

When many patterns are needed, use --exclude-from with a file listing each pattern, one per line:

rsync -a --exclude-from='exclude-file.txt' src_directory/ dst_directory/

Contents of exclude-file.txt:

file1.txt
dir1/*
dir2

Excluding with regular‑expression patterns

Use pattern matching to exclude groups of files, e.g., all .jpg files:

rsync -a --exclude '*.jpg*' src_directory/ dst_directory/

To copy only .jpg files while excluding everything else:

rsync -a -m --include='*.jpg' --include='*/' --exclude='*' src_directory/ dst_directory/

When using multiple include/exclude rules, the first matching rule applies. The -m flag removes empty directories.

Alternatively, pipe the output of find to rsync:

find src_directory/ -name "*.jpg" -printf '%P\0
' | rsync -a --files-from=- src_directory/ dst_directory/

Conclusion

After mastering these options, rsync can replace scp as the standard tool for synchronizing files and directories locally or between servers.