Mastering rsync: Uncover the Secrets of Efficient File Synchronization

If you are a sysadmin handling dozens to thousands of servers, bulk operations, environment sync, and data sync are essential skills.

The indispensable tool for synchronization is rsync , and this article explores the art of using it effectively.

Basic usage (no options): rsync main.c machineB:/home/userB 1. Synchronization occurs when source and destination file contents differ; rsync ensures both sides match.

2. By default, rsync does not preserve the file’s modify time; the destination file’s timestamp is updated to the current time after transfer.

3. Permissions are copied only if the destination file does not already exist; existing files keep their original permissions.

4. As long as rsync can read the source and write to the target path, it will synchronize the file.

5. rsync creates files using the login account on the destination; preserving owner and group requires root privileges.

Using the -t option: rsync -t main.c machineB:/home/userB 1. The -t flag forces the destination file’s modify time to match the source.

2. rsync first compares timestamps and file sizes; if they match, the file is considered identical and is not transferred.

3. However, if timestamps and sizes match but the content differs, rsync will miss the change—a known pitfall.

4. To avoid this, use the -I option.

Using the -I option: rsync -I main.c machineB:/home/userB 1. -I forces a full content check for each file, ensuring data consistency at the cost of speed.

2. The destination file’s modify time is still updated to the current time.

Verbose output with -v : rsync -vI main.c machineB:/home/userB Increasing the number of -v flags provides more detailed logs.

Compression with -z :

This option compresses data before transmission, useful on slow networks; it uses the same algorithm as gzip.

Recursive sync with -r :

Without -r, rsync skips directories: rsync superman machineB:/home/userB → skipping directory superman Use -r to synchronize directories recursively: rsync -r superman machineB:/home/userB Handling symbolic links with -l :

Without -l, rsync skips symlinks: rsync softlink machineB:/home/userB → skipping non-regular file "softlink" Adding -l copies the symlink itself; -L follows the link.

Preserving permissions with -p :

Without -p, existing files keep their original permissions; new files inherit source permissions.

With -p, permissions are always synchronized.

Preserving group and owner with -g and -o :

These options keep group and owner information, requiring appropriate privileges.

Device files with -D :

Preserves device files (root only).

Archive mode with -a : -a is equivalent to -rlptgoD, enabling recursive sync and preserving most attributes.

It cannot sync hard links; use -H for that.

Deletion options: --delete removes files on the destination that no longer exist on the source (requires -r). --delete-excluded deletes files matching exclude patterns. --delete-after performs deletions after the transfer instead of before.

Use the dry‑run option -n first to preview deletions.

Excluding files:

Use --exclude to omit specific paths, or --exclude-from to read patterns from a file.

Partial transfers and progress: --partial enables resume of interrupted transfers; -P is a shortcut for --partial --progress. --progress shows transfer progress, providing visual feedback.

Overall, mastering these options lets you harness rsync’s full power for reliable, efficient synchronization.