Mastering Rsync: A Complete Guide to Linux Remote Data Synchronization

Overview

Rsync (remote synchronize) synchronizes files between hosts over LAN/WAN using the rsync algorithm, which transfers only the differing parts of files. It runs on most Unix‑like systems (Linux, Solaris, BSD) and has Windows ports such as cwRsync and Sync2NAS.

Principle

Originally created to replace rcp, rsync is maintained at rsync.samba.org and its configuration file rsync.conf follows the Samba format. Rsync can operate via rsh, ssh, or as a daemon. In daemon mode the server listens on TCP port 873, authenticates passwords, transfers the full file set on the first connection and performs incremental backups thereafter.

Key Features

Mirrors entire directory trees and file systems.

Preserves permissions, timestamps, symbolic and hard links.

Installs without special privileges.

Optimized workflow for high transfer efficiency.

Supports transfer over rsh, ssh, or direct socket connections.

Allows anonymous transfers.

SSH Mode

1. Local synchronization

# mkdir src
# touch src/{1,2,3,4}
# mkdir dest
# rsync -av src/ dest/          # sync contents of src to dest (exclude src itself)
# rsync -av src dest/           # sync src directory itself to dest
# rsync -avR src/ dest/         # same effect even with trailing slash

2. LAN synchronization

# mkdir src
# touch src/{a,b,c,d}
# mkdir dest
# rsync -av 172.16.22.12:/data/test src/ dest/   # pull remote to local (enter root password)
# rsync -av src/ 172.16.22.12:/data/test dest/   # push local to remote
# rsync -av src 172.16.22.12:/data/test dest/   # sync whole directory
# rm -rf src/d                                 # delete a file locally
# rsync -av --delete src/ 172.16.22.12:/data/test dest/   # delete extraneous files on target

3. User‑specific LAN synchronization

# useradd george
# passwd george
# mkdir /home/george
# touch /home/george/g{1,2,3,4}
# rsync -av src -e ssh -l george 172.16.22.12:/home/george   # local to remote as george
# rsync -av 172.16.22.12:/home/george/ g* -e ssh -l george -p 22 dest/   # remote to local

Daemon Mode

1. Service startup

For heavily loaded servers run rsync as an independent daemon:

# yum install rsync xinetd
# /usr/bin/rsync --daemon

For lightly loaded servers run it via xinetd:

# yum install rsync xinetd
# vim /etc/xinetd.d/rsync   # set "disable = no"
# /etc/init.d/xinetd start
# chkconfig rsync on
# netstat -ntpl | grep 873   # verify listening on port 873

2. Configuration parameters (rsyncd.conf)

Global parameters (apply to all modules unless overridden): address: IP address for standalone mode (ignored by xinetd). port: Listening port, default 873. motd file: Message displayed to clients on connection. pid file: File where the daemon writes its PID. log file: Path for daemon logs (instead of syslog). syslog facility: Syslog priority. socket options: Custom TCP options. lockfile: Path for lock file controlling max connections. timeout = 600: Connection timeout in seconds.

Module parameters (define exported directories): path (required): Filesystem path exported by the module. comment: Description shown to clients.

Module control parameters (security and behavior): use chroot = yes: Chroot to path before transfer (requires root). uid / gid: UID/GID under which files are transferred (default nobody). max connections: Maximum concurrent connections (0 = unlimited). list: Whether the module appears in the module list (default true). read only: Disallow client uploads (default true). write only: Disallow client downloads (default false). ignore errors: Ignore I/O errors (default true). ignore nonreadable: Skip files without read permission (default false). dont compress: File patterns excluded from compression.

File‑filter parameters : exclude: Space‑separated patterns to exclude (same as --exclude on client). exclude from: File containing exclude rules. include: Space‑separated patterns to include. include from: File containing include rules.

Authentication parameters : auth users: Space‑ or comma‑separated list of allowed usernames (independent of system users). secrets file: File storing "user:password" pairs (used only when auth users is defined, mode 600). strict modes: Enforce strict permissions on the secrets file (default true).

Access control parameters : hosts allow: List of client hosts permitted to connect (default *). hosts deny: List of client hosts denied access.

Logging parameters : transfer logging: Enable logging of transfer operations (default false). log format: Custom log format, e.g. "%o %h [%a] %m (%u) %f %l".

3. Example server configuration

# vim /etc/rsyncd.conf
uid = root
gid = root
use chroot = no
max connections = 5
timeout = 600
pid file = /var/run/rsyncd.pid
lockfile = /var/run/rsyncd.lock
log file = /var/log/rsyncd.log
[web1]
  path = /data/test/src
  ignore errors = yes
  read only = no
  write only = no
  hosts allow = 192.168.22.12
  hosts deny =
  list = yes
  auth users = web
  secrets file = /etc/web.passwd

Testing

1. Client setup

# yum -y install rsync
# mkdir /data/

2. Common rsync options

-a

: Archive mode (equivalent to -rlptgoD). -r: Recursive. -l: Preserve symlinks. -i: List files on the server. -p: Preserve permissions. -t: Preserve modification times. -g: Preserve group. -o: Preserve owner. -D: Preserve device files. -z: Compress during transfer. -P: Show progress. -v: Verbose output.

3. Using a password file

# echo "123" > /tmp/rsync.password
# chmod 600 /tmp/rsync.password
# rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/

4. Automated client sync (cron)

# crontab -e
10 0 * * * rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

Real‑time synchronization with inotify‑tools

1. Install inotify‑tools

# yum install -y gcc   # build dependencies
# mkdir /usr/local/inotify
# tar -xf inotify-tools-3.14.tar.gz
# cd inotify-tools-3.14
# ./configure --prefix=/usr/local/inotify
# make && make install

2. Set environment variables

# vim /root/.bash_profile
export PATH=/usr/local/inotify/bin/:$PATH
# source /root/.bash_profile
# echo '/usr/local/inotify/lib' >> /etc/ld.so.conf
# ldconfig
# ln -s /usr/local/inotify/include /usr/include/inotify

3. Common inotify‑tools options

-m

: Keep listening (default exits after an event). -r: Recursively watch directories. -q: Print only events. -e: Specify events (e.g., modify,create,move,delete,attrib). --fromfile: Read watch list from a file. --timefmt: Set time format. --format: Set output format. --exclude / --excludei: Exclude patterns (case‑sensitive/insensitive).

4. Real‑time sync test

Initial sync, then monitor source directory and push changes to the destination via ssh.

# mkdir /data/test/dest/   # on destination
# mkdir /data/test/src/    # on source
# rsync -av --delete /data/test/src/ 192.168.22.12:/data/test/dest/   # initial sync
# vim /data/test/test.sh
#!/bin/bash
inotifywait -mrq -e modify,create,move,delete,attrib /data/ |
while read events; do
  rsync -a --delete /data/ 192.168.22.12:/data/ >> /tmp/rsync.log 2>&1
  echo "$(date +'%F %T') event: $events" >> /tmp/rsync.log
done
# chmod 755 /data/test/test.sh
# /data/test/test.sh &
# echo '/data/test/test.sh &' >> /etc/rc.local   # enable at boot

A similar script on the target can pull changes, achieving bidirectional synchronization.