Master Rsync: Fast File Sync, Daemon Setup, and Real‑Time Inotify Integration

1. Introduction

Rsync (remote synchronize) is a remote data‑synchronization tool that can quickly sync files between multiple hosts over LAN or WAN. It uses the Rsync algorithm to transfer only the differing parts of files, making synchronization very fast.

Rsync runs on most Unix‑like systems (Linux, Solaris, BSD) and also has Windows ports such as cwRsync and Sync2NAS.

2. Principle

Rsync was originally created to replace rcp and is now maintained at rsync.samba.org; its configuration file format resembles Samba’s.

It can operate via rsh or ssh, or run as a daemon. In daemon mode the Rsync server listens on port 873 for client connections.

When a client connects, the server authenticates the password and performs an initial full transfer; subsequent runs transfer only incremental changes.

3. Features

Can mirror entire directory trees and file systems.

Preserves permissions, timestamps, and symbolic/hard links.

Installs without special privileges.

Optimized workflow for high transfer efficiency.

Supports rsh, ssh, or direct socket connections.

Allows anonymous transfers.

4. SSH Mode

4.1 Local sync

# mkdir src</code>
<code># touch src/{1,2,3,4}</code>
<code># mkdir dest</code>
<code># rsync -av src/ dest/   # sync contents of src to dest (excluding src itself)</code>
<code># rsync -av src dest/    # sync src directory itself to dest</code>
<code># rsync -avR src/ dest/  # same as above even with trailing slash

4.2 LAN sync

# mkdir src</code>
<code># touch src/{a,b,c,d}</code>
<code># mkdir dest</code>
<code># rsync -av 172.16.22.12:/data/test/src/ dest/   # pull from remote to local (requires root password)</code>
<code># rsync -av src/ 172.16.22.12:/data/test/dest/   # push local files to remote</code>
<code># rsync -av src 172.16.22.12:/data/test/dest/   # sync whole directory</code>
<code># rm -rf src/d   # delete a file</code>
<code># rsync -av --delete src/ 172.16.22.12:/data/test/dest/   # delete extraneous files on target

4.3 LAN sync with specific user

# useradd george</code>
<code># passwd george</code>
<code># mkdir /home/george/test</code>
<code># touch /home/george/test/g{1,2,3,4}

# rsync -av src '-e ssh -l george' 172.16.22.12:/home/george   # local to remote as user george</code>
<code># rsync -av 172.16.22.12:/home/george/test/g* '-e ssh -l george -p 22' dest/   # pull specific files

5. Daemon Mode

5.1 Service start

For a heavily loaded server run Rsync as a standalone daemon:

# yum install rsync xinetd</code>
<code># /usr/bin/rsync --daemon

For a lightly loaded server run via xinetd:

# yum install rsync xinetd</code>
<code># vim /etc/xinetd.d/rsync   # set "disable = no"</code>
<code># /etc/init.d/xinetd start</code>
<code># chkconfig rsync on</code>
<code># netstat -ntpl | grep 873   # verify service is listening

5.2 Configuration details

Both modes require an rsyncd.conf file, similar in format to Samba’s main config.

Global parameters (apply to all modules unless overridden):

address – IP address when running standalone (ignored by xinetd).

port – listening port (default 873, ignored by xinetd).

motd file – message displayed to clients on connection.

pid file – file where the daemon writes its PID.

log file – path to the daemon’s log file.

syslog facility – syslog priority for messages.

socket options – custom TCP options.

lockfile – location of the lock file.

timeout – idle timeout (default 600 seconds).

Module parameters define which directories are exported:

path – required, the directory to sync.

comment – description shown to clients.

Module control parameters :

use chroot – defaults to true; improves security but requires root.

uid / gid – user and group IDs for file transfers (default nobody).

max connections – maximum concurrent connections (0 = unlimited).

lock file – lock file for max‑connections control.

list – whether the module appears in the module list (default true).

read only / write only – access control flags.

ignore errors – ignore I/O errors.

dont compress – file patterns to exclude from compression.

File‑filter parameters :

exclude – space‑separated list of files/dirs to exclude.

exclude from – file containing exclude rules.

include – space‑separated list of files/dirs to include.

include from – file containing include rules.

Authentication parameters :

auth users – list of usernames allowed to connect (independent of system users).

secrets file – file storing "user:password" pairs (mode 600).

strict modes – enforce permissions on the secrets file.

Access control parameters :

hosts allow – whitelist of client hosts (default *).

hosts deny – blacklist of client hosts.

Logging parameters :

transfer logging – enable per‑transfer logging (default false).

log format – format string for log entries (e.g., "%o %h [%a] %m (%u) %f %l").

5.3 Server configuration example

# vim /etc/rsyncd.conf   # create configuration file

uid = root

gid = root

use chroot = no

max connections = 5

timeout = 600

pid file = /var/run/rsyncd.pid

lockfile = /var/run/rsyncd.lock

log file = /var/log/rsyncd.log

[web1] # module name

# mkdir /data/test/src</code>
<code># mkdir /data/test/src/george</code>
<code># touch /data/test/src/{1,2,3}</code>
<code># echo "web:123" > /etc/web.passwd</code>
<code># chmod 600 /etc/web.passwd</code>
<code># service xinetd restart

6. Client usage

# yum -y install rsync</code>
<code># mkdir /data/test

Example command to pull the web1 module:

# rsync -avzP [email protected]::web1 /data/test/   # password prompt required

Common options:

-a – archive mode (equivalent to -rlptgoD).

-r – recursive.

-l – copy symlinks as symlinks.

-i – list files.

-p – preserve permissions.

-t – preserve timestamps.

-g – preserve group.

-o – preserve owner.

-D – preserve device files.

-z – compress during transfer.

-P – show progress and keep partially transferred files.

-v – verbose output.

# rsync -avzP --delete [email protected]::web1 /data/test/   # make client mirror server</code>
<code># rsync -avzP --delete /data/test/ [email protected]::web1   # upload client files</code>
<code># rsync -avzP --delete /data/test/ [email protected]::web1/george   # upload to subdirectory</code>
<code># rsync -ir --password-file=/tmp/rsync.password [email protected]::web1   # list remote files recursively</code>
<code># rsync -avzP --exclude="*3*" --password-file=/tmp/rsync.password [email protected]::web1 /data/test/   # exclude files containing "3"

6.1 Password‑file sync

# echo "123" > /tmp/rsync.password</code>
<code># chmod 600 /tmp/rsync.password</code>
<code># rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/

6.2 Automated sync with cron

# crontab -e</code>
<code>10 0 * * * rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/</code>
<code># crontab -l

7. Real‑time synchronization with inotify‑tools

Inotify‑tools provide C libraries and command‑line utilities ( inotifywait, inotifywatch) to monitor filesystem events.

7.1 Installation

# yum install -y gcc</code>
<code># mkdir /usr/local/inotify</code>
<code># tar -xf inotify-tools-3.14.tar.gz</code>
<code># cd inotify-tools-3.14</code>
<code># ./configure --prefix=/usr/local/inotify/</code>
<code># make && make install

7.2 Environment setup

# vim /root/.bash_profile</code>
<code>export PATH=/usr/local/inotify/bin/:$PATH</code>
<code># source /root/.bash_profile</code>
<code># echo '/usr/local/inotify/lib' >> /etc/ld.so.conf</code>
<code># ldconfig</code>
<code># ln -s /usr/local/inotify/include /usr/include/inotify

7.3 Common inotify options

-m – keep listening.

-r – recursive.

-q – quiet output.

-e – specify events (access, modify, attrib, open, delete, create, move, etc.).

--exclude – regex to exclude files.

--excludei – case‑insensitive exclude.

7.4 Real‑time sync script example

# mkdir /data/test/dest   # on destination machine</code>
<code># mkdir /data/test/src    # on source machine</code>
<code># rsync -av --delete /data/test/src/ 192.168.22.12:/data/test/dest   # test command</code>
<code># vim /data/test/test.sh</code>
<code>#!/bin/bash</code>
<code>/usr/local/inotify/bin/inotifywait -mrq -e modify,create,move,delete,attrib /data/test/src | while read events; do</code>
<code>  rsync -a --delete /data/test/src/ 192.168.22.12:/data/test/dest</code>
<code>  echo "$(date +'%F %T') event: $events" >> /tmp/rsync.log 2>&1</code>
<code>done</code>
<code># chmod 755 /data/test/test.sh</code>
<code># /data/test/test.sh &</code>
<code># echo '/data/test/test.sh &' >> /etc/rc.local   # run at boot

Running a similar script on the destination machine enables bidirectional synchronization.