Mastering Rsync: From Basic Sync to Daemon and Real‑Time Inotify Integration

Introduction

Rsync (remote synchronize) is a fast data‑synchronization utility for Unix‑like systems (Linux, Solaris, BSD) and Windows ports (e.g., cwRsync). It transfers only the differing parts of files using the rsync algorithm, which makes incremental updates efficient.

Principles

Rsync was created to replace rcp and is maintained at rsync.samba.org. Its configuration file rsync.conf follows a Samba‑style syntax. Rsync can run over rsh / ssh or as a daemon listening on TCP port 873.

Key Features

Mirrors entire directory trees.

Preserves permissions, timestamps, symbolic and hard links.

Does not require special privileges to install.

Optimized for high transfer efficiency.

Supports rsh, ssh, or direct socket connections.

Allows anonymous transfers.

SSH Mode

1. Local Synchronization

# mkdir src
# touch src/{1,2,3,4}
# mkdir dest
# rsync -av src/ dest/          # sync contents of src to dest (exclude src itself)
# rsync -av src dest/           # sync src directory including the directory itself
# rsync -avR src/ dest/         # same as above even with trailing slash

2. LAN Synchronization

# mkdir src
# touch src/{a,b,c,d}
# mkdir dest
# rsync -av 172.16.22.12:/data/test/src/ dest/   # pull from remote to local (requires password)
# rsync -av src/ 172.16.22.12:/data/test/dest/   # push local files to remote
# rsync -av src 172.16.22.12:/data/test/dest/    # sync whole directory
# rm -rf src/d                                 # delete a file locally
# rsync -av --delete src/ 172.16.22.12:/data/test/dest/   # delete extraneous files on remote

3. Specified User Synchronization

# useradd george
# passwd george
# mkdir /home/george/test
# touch /home/george/test/g{1,2,3,4}
# rsync -av src -e "ssh -l george" 172.16.22.12:/home/george   # local → remote as user george
# rsync -av 172.16.22.12:/home/george/test/g* -e "ssh -l george -p 22" dest/   # pull specific files

Daemon Mode

1. Service Startup

Standalone (heavy‑load) :

# yum install rsync xinetd
# /usr/bin/rsync --daemon   # start daemon directly

Xinetd (light‑load) :

# yum install rsync xinetd
# vim /etc/xinetd.d/rsync   # set "disable = no"
# /etc/init.d/xinetd start   # start xinetd
# chkconfig rsync on
# netstat -ntpl | grep 873   # verify listening on port 873

2. Configuration (rsyncd.conf)

Global parameters (apply to all modules unless overridden):

address – IP address for standalone mode (ignored by xinetd).

port – listening port (default 873, ignored by xinetd).

motd file – message displayed to clients on connection.

pid file – file where the daemon writes its PID.

log file – optional transfer log.

syslog facility – syslog priority for daemon messages.

socket options – custom TCP options.

lockfile – path for the lock file.

timeout = 600 – inactivity timeout (seconds).

Module parameters (define each shared directory):

path – mandatory absolute path of the directory to expose.

comment – description shown to clients.

use chroot – default yes; chroots to path for security (requires root).

uid / gid – UID/GID under which files are transferred (default nobody).

max connections – maximum concurrent connections (0 = unlimited).

read only / write only – control upload/download permissions.

ignore errors – ignore I/O errors.

hosts allow / hosts deny – whitelist/blacklist of client IPs.

auth users – list of usernames allowed to connect (independent of system users).

secrets file – file containing "user:password" pairs (mode 600).

strict modes – enforce permissions on the secrets file.

transfer logging – enable per‑transfer logging.

log format – custom log entry format (e.g., "%o %h [%a] %m (%u) %f %l").

3. Example rsyncd.conf

# vim /etc/rsyncd.conf
uid = root
gid = root
use chroot = no
max connections = 5
timeout = 600
pid file = /var/run/rsyncd.pid
lockfile = /var/run/rsyncd.lock
log file = /var/log/rsyncd.log

[web1]
  path = /data/test/src
  comment = Test module
  ignore errors = yes
  read only = no
  write only = no
  hosts allow = 192.168.22.12
  hosts deny =
  list = yes
  auth users = web
  secrets file = /etc/web.passwd

# mkdir -p /data/test/src/george
# touch /data/test/src/{1,2,3}
# echo "web:123" > /etc/web.passwd
# chmod 600 /etc/web.passwd
# service xinetd restart   # if using xinetd

Testing

1. Client Setup

# yum -y install rsync
# mkdir /data/test

2. Common Options

-a

– archive mode (equivalent to -rlptgoD). -v – verbose. -z – compress data during transfer. -P – show progress and keep partially transferred files. --delete – delete files on the destination that no longer exist on the source. --exclude=PATTERN – exclude files matching PATTERN. --password-file=FILE – read password from FILE (mode 600).

3. Example Commands

# rsync -avzP [email protected]::web1 /data/test/                     # pull module "web1"
# rsync -avzP --delete [email protected]::web1 /data/test/          # make local mirror identical
# rsync -avzP --delete /data/test/ [email protected]::web1          # push local changes
# rsync -ir --password-file=/tmp/rsync.password [email protected]::web1   # list files recursively
# rsync -avzP --exclude="*3*" --password-file=/tmp/rsync.password [email protected]::web1 /data/test/   # sync all but files containing "3"

4. Synchronization via Password File

# echo "123" > /tmp/rsync.password
# chmod 600 /tmp/rsync.password
# rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

5. Automated Sync with Cron

# crontab -e
# Add line (run daily at 00:10)
10 0 * * * rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

Real‑Time Data Synchronization with inotify‑tools

1. About inotify‑tools

Provides C libraries and command‑line utilities ( inotifywait, inotifywatch) for monitoring filesystem events on Linux.

2. Installation

# yum install -y gcc
# mkdir /usr/local/inotify
# tar -xf inotify-tools-3.14.tar.gz -C /usr/local/inotify --strip-components=1
# cd /usr/local/inotify
# ./configure --prefix=/usr/local/inotify
# make && make install

3. Environment Setup

# vim /root/.bash_profile
export PATH=/usr/local/inotify/bin:$PATH
# source /root/.bash_profile
# echo '/usr/local/inotify/lib' >> /etc/ld.so.conf
# ldconfig
# ln -s /usr/local/inotify/include /usr/include/inotify

4. Common inotifywait Options

-m

– monitor mode (keep listening). -r – recursive. -q – quiet, output only events. -e EVENT – specify events (e.g., modify,create,move,delete,attrib). --exclude=REGEX – exclude paths matching REGEX. --format=FMT – custom output format.

5. Real‑Time Sync Example

Monitor /data/test/src on the source host and push changes to the destination host via rsync.

# mkdir -p /data/test/dest   # on destination
# mkdir -p /data/test/src    # on source
# rsync -av --delete /data/test/src/ 192.168.22.12:/data/test/dest   # initial sync

# vim /data/test/sync.sh
#!/bin/bash
/usr/local/inotify/bin/inotifywait -mrq -e modify,create,move,delete,attrib /data/test/src |
while read events; do
  rsync -a --delete /data/test/src/ 192.168.22.12:/data/test/dest
  echo "$(date +'%F %T')  Event: $events" >> /tmp/rsync.log 2>&1
done

# chmod 755 /data/test/sync.sh
# /data/test/sync.sh &   # run in background
# echo '/data/test/sync.sh &' >> /etc/rc.local   # start on boot

A complementary script on the destination can pull changes back, achieving bidirectional synchronization:

# rsync -a --delete /data/test/dest/ 192.168.22.11:/data/test/src