Master Rsync: Fast File Sync Over LAN/WAN with SSH and Daemon Modes

1. Introduction

1.1 Overview

Rsync (remote synchronize) is a remote data synchronization tool that can quickly sync files between multiple hosts over LAN/WAN. It uses the rsync algorithm to transfer only the differences between files, making synchronization fast.

Rsync runs on most Unix-like systems (Linux, Solaris, BSD) and also has Windows ports such as cwRsync and Sync2NAS.

1.2 Principles

Rsync was originally created to replace rcp and is now maintained at rsync.samba.org; its configuration file rsync.conf follows the Samba format. It can operate via rsh or ssh, or run as a daemon listening on port 873.

When a client connects, the server authenticates the password and performs an initial full transfer; subsequent runs transfer only incremental changes.

1.3 Features

Can mirror entire directory trees and file systems.

Preserves permissions, timestamps, and symbolic/hard links.

Installs without special privileges.

Optimized transfer process for high efficiency.

Supports rsh, ssh, or direct socket connections.

Allows anonymous transfers.

2. SSH Mode

2.1 Local-to-Local Sync

# mkdir src
# touch src/{1,2,3,4}
# mkdir dest
# rsync -av src/ dest/   # sync contents of src to dest (excluding src itself)
# rsync -av src dest/    # sync src directory itself to dest
# rsync -avR src/ dest/  # same effect even with trailing slash

2.2 LAN Sync

# mkdir src
# touch src/{a,b,c,d}
# mkdir dest
# rsync -av 172.16.22.12:/data/test/src/ dest/   # pull from remote to local
# rsync -av src/ 172.16.22.12:/data/test/dest/   # push local to remote
# rsync -av src 172.16.22.12:/data/test/dest/   # sync whole directory
# rm -rf src/d                                 # delete a file
# rsync -av --delete src/ 172.16.22.12:/data/test/dest/   # delete extraneous files on remote

2.3 Specified User Sync

# useradd george
# passwd george
# mkdir /home/george/test
# touch /home/george/test/g{1,2,3,4}
# rsync -av src -e ssh -l george 172.16.22.12:/home/george   # local to remote as george
# rsync -av 172.16.22.12:/home/george/test/g* -e ssh -l george -p 22 dest/

3. Daemon Mode

3.1 Starting the Service

Heavy‑load server (stand‑alone)

# yum install rsync xinetd
# /usr/bin/rsync --daemon

Light‑load server (xinetd)

# yum install rsync xinetd
# vim /etc/xinetd.d/rsync   # set "disable = no"
# /etc/init.d/xinetd start
# chkconfig rsync on
# netstat -ntpl | grep 873   # verify service is listening

3.2 Configuration Details

Both modes use rsyncd.conf, whose syntax resembles Samba’s main configuration file.

Global parameters (apply to all modules unless overridden):

address – IP address for standalone mode; ignored by xinetd.

port – listening port (default 873).

motd file – message displayed to clients on connection.

pid file – file storing the daemon PID.

log file – file for rsync logs.

syslog facility – syslog priority.

socket options – custom TCP options.

lockfile – path for lock file.

timeout – idle timeout (default 600 seconds).

Module parameters define which directories are shared.

path – required, the directory exported by the module.

comment – description shown to clients.

use chroot – defaults to true; improves security but requires root.

uid / gid – user and group under which files are transferred (default nobody).

max connections – limit of concurrent connections (0 = unlimited).

list – whether the module appears in the module list (default true).

read only / write only – control upload/download permissions.

ignore errors – skip I/O errors.

auth users – list of usernames allowed to connect.

secrets file – file containing "user:password" pairs (mode 600).

hosts allow / hosts deny – IP‑based access control.

transfer logging – enable logging of transfer operations.

log format – custom log line format (e.g., "%o %h [%a] %m (%u) %f %l").

exclude / include – patterns for excluding or including files.

3.3 Server Configuration Example

# vim /etc/rsyncd.conf   # create configuration file
uid = root
gid = root
use chroot = no
max connections = 5
timeout = 600
pid file = /var/run/rsyncd.pid
lockfile = /var/run/rsyncd.lock
log file = /var/log/rsyncd.log

[web1]
  path = /data/test/src
  ignore errors = yes
  read only = no
  write only = no
  hosts allow = 192.168.22.12
  hosts deny =
  list = yes
  auth users = web
  secrets file = /etc/web.passwd

After creating the directory structure and password file, restart xinetd:

# mkdir /data/test/src
# mkdir /data/test/src/george
# touch /data/test/src/{1,2,3}
# echo "web:123" > /etc/web.passwd
# chmod 600 /etc/web.passwd
# service xinetd restart

4. Testing

4.1 Client Setup

# yum -y install rsync
# mkdir /data/test

4.2 Basic Transfer

# rsync -avzP [email protected]::web1 /data/test/   # sync module to local

Key options:

-a – archive mode (equivalent to -rlptgoD).

-r – recursive.

-l – copy symlinks.

-i – list files on server.

-p – preserve permissions.

-t – preserve timestamps.

-g – preserve group.

-o – preserve owner.

-D – preserve device files.

-z – compress during transfer.

-P – show progress and keep partially transferred files.

-v – verbose output.

4.3 Advanced Options

# rsync -avzP --delete [email protected]::web1 /data/test/
# rsync -avzP --delete /data/test/ [email protected]::web1
# rsync -avzP --delete /data/test/ [email protected]::web1/george
# rsync -ir --password-file=/tmp/rsync.password [email protected]::web1
# rsync -avzP --exclude="*3*" --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

4.4 Password File Sync

# echo "123" > /tmp/rsync.password
# chmod 600 /tmp/rsync.password
# rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

4.5 Automated Sync via Cron

# crontab -e
10 0 * * * rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

5. Real‑Time Sync with Inotify‑tools

5.1 About inotify‑tools

inotify‑tools provides C libraries and command‑line utilities (inotifywait, inotifywatch) for monitoring filesystem events on Linux.

5.2 Installation

# yum install -y gcc
# mkdir /usr/local/inotify
# tar -xf inotify-tools-3.14.tar.gz
# cd inotify-tools-3.14
# ./configure --prefix=/usr/local/inotify/
# make && make install

5.3 Environment Setup

# vim /root/.bash_profile   # add PATH
export PATH=/usr/local/inotify/bin/:$PATH
# source /root/.bash_profile
# echo '/usr/local/inotify/lib' >> /etc/ld.so.conf
# ldconfig
# ln -s /usr/local/inotify/include /usr/include/inotify

5.4 Common Options

-m – keep listening.

-r – recursive.

-q – quiet output.

-e – specify events (access, modify, attrib, open, delete, create, move, etc.).

--exclude – regex to exclude files.

--excludei – case‑insensitive exclude.

--timefmt – time format.

--format – output format.

5.5 Example Real‑Time Sync Script

# mkdir /data/test/dest
# mkdir /data/test/src
# rsync -av --delete /data/test/src/ 192.168.22.12:/data/test/dest
# vim /data/test/test.sh
#!/bin/bash
/usr/local/inotify/bin/inotifywait -mrq -e modify,create,move,delete,attrib /data/test/src |
while read events; do
  rsync -a --delete /data/test/src/ 192.168.22.12:/data/test/dest
  echo "`date +'%F %T'` Event: $events" >> /tmp/rsync.log 2>&1
done
# chmod 755 /data/test/test.sh
# /data/test/test.sh &
# echo '/data/test/test.sh &' >> /etc/rc.local   # run at boot

A similar script on the destination host can push changes back, achieving bidirectional synchronization.