Mastering Rsync Incremental Backups: Configuration, Optimization, and Practical Implementation

Overview

Data loss caused by a single hard‑disk failure can cripple a business. Regular, reliable backups are essential. For Linux environments the rsync utility (Remote Sync) provides a lightweight, agent‑less solution that transfers only the differences between files.

Delta Transfer Algorithm

Compute block checksums of the destination (old) file.

Send those checksums to the source.

The source scans the new file with a rolling checksum to find matching blocks.

Only the non‑matching (changed) blocks are transmitted.

The destination reconstructs the new file by applying the received blocks to the old file.

Result: a 10 GB file with only 100 MB changed transfers roughly 100 MB instead of the full 10 GB.

File Comparison

By default rsync compares size and mtime . If either differs, the file is transferred.

Use --checksum for a full content hash comparison (slower but safest).

Performance Comparison (real world)

+----------------------+----------------+----------+-------------------+
| Method               | Data transferred| Time     | Network usage    |
+----------------------+----------------+----------+-------------------+
| Full copy (scp/tar)  | 15 GB          | ~12 min  | Full bandwidth    |
| rsync first sync     | 15 GB          | ~13 min  | Full bandwidth    |
| rsync incremental*   | ~200 MB        | ~15 s    | Very low          |
| rsync incremental**  | ~0 MB          | ~3 s     | Negligible        |
+----------------------+----------------+----------+-------------------+
*≈ daily change of 200 MB
**no change

Typical Use Cases

Local backup (different disks/partitions on the same host).

Remote backup over SSH or rsync daemon.

Near‑real‑time sync with inotify.

Disaster‑recovery across data‑centers.

Configuration distribution.

Website deployment (sync build artefacts to production).

Environment Requirements

OS: Ubuntu 24.04 LTS or Rocky Linux 9.x (both must have rsync installed).

rsync ≥ 3.3.0 (Ubuntu 24.04 ships 3.3.0; Rocky 9 ships 3.2.3 – upgrade if needed).

OpenSSH ≥ 9.x for encrypted transport.

Optional: inotify-tools ≥ 4.x for real‑time sync.

Optional: flock for script locking.

Basic Usage

Installation

# Ubuntu/Debian
sudo apt install -y rsync

# Rocky Linux / CentOS / RHEL
sudo dnf install -y rsync

# Alpine Linux
sudo apk add rsync

# Verify version
rsync --version | head -1

Common Options (quick reference)

# Archive mode (preserves permissions, timestamps, links, …)
-a   # equivalent to -rlptgoD

# Verbose output
-v   # -vv for even more detail

# Dry‑run (simulation)
-n   # or --dry-run

# Compression (use only for text data or low‑bandwidth links)
-z   # --compress-level=1..9 to tune speed/compression

# Bandwidth limit (KB/s)
--bwlimit=10240   # 10 MB/s

# Delete files on the destination that no longer exist on the source
--delete   # combine with --dry-run first!

# Safer delete – stop if more than N files would be removed
--max-delete=100

# Exclude patterns
--exclude='*.log' --exclude='node_modules'
--exclude-from=/etc/rsync_exclude.list

# Include only specific files
--include='*.conf' --exclude='*'

# Compare only size (fast, but may miss content changes)
--size-only

# Full content checksum comparison (slow, most reliable)
--checksum

# Show transfer statistics
--stats

# Show progress per file
--progress   # or -P (implies --partial)

# Preserve hard links (useful with --link-dest)
--hard-links

# Limit I/O timeout (seconds)
--timeout=300

Trailing‑Slash Gotcha

The presence or absence of a trailing / on the source path changes what is copied:

# Copy contents of /data/ into /backup/ (no extra directory level)
rsync -av /data/ /backup/

# Copy the directory itself, creating /backup/data/
rsync -av /data /backup/

Always double‑check the slash before running a command that contains --delete or -n for a dry‑run.

Advanced Scenarios

SSH Mode vs Daemon Mode

+-------------------+----------------------+----------------------+-------------------+
| Aspect            | SSH mode             | Daemon mode          | Typical use case  |
+-------------------+----------------------+----------------------+-------------------+
| Config complexity | Low (just SSH)       | Medium (rsyncd.conf) | Most deployments  |
| Data encryption   | Yes (SSH)            | No (plain)           | Internet‑facing |
| Authentication    | SSH keys / passwords | rsync built‑in auth  | Same as above    |
| Performance      | Slightly lower (CPU) | Slightly higher      | Intra‑datacenter |
| Port              | 22 (SSH)             | 873 (rsync)          | Same as above    |
| Security          | High                 | Medium (add stunnel) | Same as above    |
+-------------------+----------------------+----------------------+-------------------+

In practice about 90 % of backup jobs use SSH mode; daemon mode is only considered for high‑throughput internal networks where encryption overhead is a concern.

Incremental Backups with --link-dest

--link-dest=DIR

tells rsync to look for identical files in DIR. When a match is found a hard link is created in the destination instead of copying the file. This yields a full‑snapshot appearance while storing only the changed data.

# Weekly full backup (Sunday)
rsync -a /data/ /backup/weekly/$(date +%Y-%m-%d)/

# Daily incremental (Mon‑Sat) – keep a "latest" symlink to the previous snapshot
rsync -a --link-dest=$(readlink -f /backup/weekly/latest) \
      /data/ /backup/daily/$(date +%Y-%m-%d)/

# Update the "latest" symlink
ln -snf /backup/daily/$(date +%Y-%m-%d) /backup/daily/latest

Space‑saving example (10 GB source, 500 MB daily change):

+----------------------+-------------------+-------------------+
| Backup scheme        | 7‑day total space | Comment           |
+----------------------+-------------------+-------------------+
| Full copy each day   | 70 GB (10 GB×7)   | Very wasteful     |
| rsync + --link-dest  | ~13.5 GB           | 10 GB base + 0.5 GB×7 |
+----------------------+-------------------+-------------------+

Keeping 30 days with the same method would require only ~25 GB instead of 300 GB.

Large‑File Optimisation

# Update the file in place (no temporary copy)
rsync -av --inplace large_db_dump.sql remote:/backup/

# Append‑only logs (only new lines are sent)
rsync -av --append /var/log/app.log remote:/backup/logs/

# Resume‑capable transfer for very big files
rsync -avP --timeout=300 --bwlimit=51200 huge_backup.tar.gz remote:/backup/

Massive Small‑File Optimisation

# Reduce memory usage and speed up the file‑list phase
rsync -av --delete-during --size-only src/ dst/

# Force delta algorithm even for local copies
rsync -av --no-whole-file src/ dst/

# Show overall progress instead of per‑file output
rsync -a --info=progress2 src/ dst/

# Split a directory with millions of files into sub‑directories
for sub in a b c d e f g h i j k l m n o p q r s t u v w x y z; do
    rsync -a --info=progress2 src/${sub} dst/${sub}
done

Cron Scheduling with flock

# Edit crontab (crontab -e)
0 2 * * * /usr/local/bin/daily_backup.sh >> /var/log/cron_backup.log 2>&1
0 */6 * * * /usr/local/bin/daily_backup.sh >> /var/log/cron_backup.log 2>&1
0 1 * * 0 /usr/local/bin/weekly_full_backup.sh >> /var/log/cron_backup.log 2>&1

Wrap the script with flock to avoid overlapping runs:

#!/bin/bash
LOCK_FILE="/var/run/rsync_backup.lock"
exec 200>"$LOCK_FILE"
if ! flock -n 200; then
    echo "[$(date)] Another backup is running – exiting" >> /var/log/rsync_backup.log
    exit 0
fi
# ...rest of the backup logic...

Error Handling & Monitoring

Common Exit Codes (selected)

0   Success
1   Syntax or usage error
2   Protocol incompatibility
3   File‑selection error
5   Daemon start failure
10  Socket I/O error (network)
11  File I/O error (disk, permissions)
12  Data‑stream error (connection lost)
24  Partial transfer – source file changed during copy (usually harmless)
30  Timeout (increase --timeout)

Typical Problems & Fixes

Permission denied : ensure the target directory is owned by the rsync user or use --chmod=Du+rwx.

Disk full : clean old snapshots, verify df -h, and check inode usage with df -i.

Connection reset : add -e "ssh -o ServerAliveInterval=30" or increase --timeout.

Timeout : use --timeout=600 and --progress to keep the connection alive.

Resuming Interrupted Transfers

Rsync automatically resumes partial files when --partial or -P is used. Re‑run the same command and only the missing parts will be sent.

Prometheus Exporter (example)

# /usr/local/bin/backup_monitor.sh
METRICS="/var/lib/prometheus/node-exporter/backup_status.prom"
TMP=$(mktemp)

echo "# HELP backup_last_success_timestamp Timestamp of last successful backup" > $TMP
echo "# TYPE backup_last_success_timestamp gauge" >> $TMP
echo "# HELP backup_size_bytes Size of the latest backup (bytes)" >> $TMP
echo "# TYPE backup_size_bytes gauge" >> $TMP

echo "# HELP backup_age_seconds Age of the latest backup" >> $TMP
echo "# TYPE backup_age_seconds gauge" >> $TMP

for d in /backup/*/; do
    name=$(basename "$d")
    latest=$(readlink -f "$d/latest" 2>/dev/null || true)
    if [ -d "$latest" ]; then
        mtime=$(stat -c %Y "$latest")
        age=$(( $(date +%s) - mtime ))
        size=$(du -sb "$latest" | cut -f1)
        echo "backup_last_success_timestamp{task=\"$name\"} $mtime" >> $TMP
        echo "backup_size_bytes{task=\"$name\"} $size" >> $TMP
        echo "backup_age_seconds{task=\"$name\"} $age" >> $TMP
    else
        echo "backup_last_success_timestamp{task=\"$name\"} 0" >> $TMP
        echo "backup_size_bytes{task=\"$name\"} 0" >> $TMP
        echo "backup_age_seconds{task=\"$name\"} 999999" >> $TMP
    fi
done

mv $TMP $METRICS

Prometheus alert rules (YAML snippet) can be added to fire when backup_age_seconds exceeds 26 hours or when backup_size_bytes drops more than 50 % compared to the previous day.

Best‑Practice Checklist

Always run a dry‑run ( -n) before using --delete.

Use a trailing slash on the source to copy contents, not the directory itself.

Prefer SSH mode (simple, encrypted, works everywhere).

Implement incremental snapshots with --link-dest for space efficiency.

Wrap backup scripts with flock, logging, and optional alerting.

Validate backups: checksum comparison, random file sampling, and periodic restore drills.

Limit bandwidth during business hours ( --bwlimit) and monitor disk space before each run.

For real‑time needs, combine inotifywait with rsync, but remember it is not truly instantaneous – use a distributed FS for sub‑second sync.

Export metrics (size, age, timestamp) for observability and set alerts for missed runs or size anomalies.

Following these guidelines yields a robust, low‑overhead backup system that can be scaled from a single server to hundreds of machines while keeping data safe and recoverable.