scp vs rsync: Choosing the Right Tool for Server File Transfers

Background and Use Cases

System administrators often need to move files between a local machine and remote servers, or between two servers. Linux provides scp (secure copy) for simple, one‑off transfers and rsync for incremental sync, compression, and resume capabilities, which are better suited for large files and regular backups.

scp Details

Basic principle

scp

uses SSH to encrypt the transfer. It runs a local command that opens an SSH session to the remote host, copies the file, and then returns the result.

Syntax

# Copy from local to remote
scp [options] source_path destination_path

# Copy from remote to local
scp [options] user@remote_host:remote_path local_path

# Copy between two remote hosts
scp [options] user1@host1:path1 user2@host2:path2

Common options

-r

: recursive copy of directories -p: preserve timestamps and permissions -q: quiet mode (no progress display) -v: verbose/debug output -C: enable compression -P port: specify SSH port (uppercase P) -i identity_file: specify private key -l limit: limit bandwidth (Kbit/s) -o ssh_option: pass additional SSH options -F ssh_config: use a custom SSH config file

Practical examples

Upload a single file:

scp /path/to/local/file.txt [email protected]:/remote/path/

Upload with a custom SSH port:

scp -P 2222 /path/to/local/file.txt [email protected]:/remote/path/

Upload with a specific private key:

scp -i ~/.ssh/id_rsa /path/to/local/file.txt [email protected]:/remote/path/

Preserve file attributes:

scp -p /path/to/local/file.txt [email protected]:/remote/path/

Batch upload with a wildcard:

scp /local/path/*.txt [email protected]:/remote/path/

Limit bandwidth to 500 KB/s (500 KB/s = 4000 Kbit/s):

scp -l 4000 /largefile.tar.gz [email protected]:/remote/path/

Common errors and fixes

Connection refused – cause: non‑standard SSH port or SSH service not running.

# Verify SSH port
ssh -p 2222 [email protected]

# Use the same port with scp
scp -P 2222 file.txt [email protected]:/path/

Permission denied (publickey) – cause: missing or mis‑configured SSH key.

# Ensure private key permissions
chmod 600 ~/.ssh/id_rsa

# Specify the key explicitly
scp -i ~/.ssh/id_rsa file.txt [email protected]:/path/

Not a regular file – cause: copying a directory without -r or missing trailing slash for target directory.

# Copy a directory
scp -r /local/directory [email protected]:/remote/path/

No such file or directory – cause: target path does not exist.

# Create the remote directory first
ssh [email protected] "mkdir -p /remote/path"

# Then copy
scp file.txt [email protected]:/remote/path/

rsync Details

Basic principle

rsync

uses a delta‑transfer algorithm to send only the differences between source and destination, dramatically reducing network traffic for repeated synchronisations.

Syntax

# Local sync
rsync [options] source_path destination_path

# Remote sync via SSH
rsync [options] source_path user@host:dest_path

# Remote sync via rsync daemon
rsync [options] source_path user@host::module/dest_path

Key options

-r

: recursive (does not preserve attributes) -a: archive mode (equivalent to -rlptgoD) -z: compress data during transfer -P: same as --partial --progress (resume + progress) --delete: delete files in destination that are absent in source (mirror sync) --exclude=PATTERN / --exclude-from=FILE: exclude matching files --include=PATTERN / --include-from=FILE: include matching files (must appear before excludes) -n or --dry-run: preview changes without copying -e ssh_option: specify remote shell (e.g., ssh -p 2222) --bwlimit=RATE: limit bandwidth (KB/s) --partial: keep partially transferred files for resume

Practical examples

Local directory sync (trailing slash copies contents only):

rsync -av /source/directory/ /target/directory/

# Sync the directory itself
rsync -av /source/directory /target/

Remote sync via SSH (upload):

rsync -avz -e ssh /local/directory/ [email protected]:/remote/directory/

Remote sync via SSH (download):

rsync -avz -e ssh [email protected]:/remote/directory/ /local/directory/

Incremental sync (only changed files):

rsync -avz /source/directory/ [email protected]:/remote/directory/

Mirror sync (delete extraneous files):

rsync -avz --delete /source/directory/ [email protected]:/remote/directory/

Exclude specific files (e.g., log files):

rsync -avz --exclude='*.log' /source/directory/ [email protected]:/remote/directory/

Preview only (dry‑run):

rsync -avzn /source/directory/ [email protected]:/remote/directory/

Limit bandwidth to 1 MB/s:

rsync -avz --bwlimit=1024 /large/directory/ [email protected]:/remote/directory/

Resume interrupted transfer:

rsync -avzP /large/directory/ [email protected]:/remote/directory/

Common errors and fixes

Skipping directory – cause: missing trailing slash on source path.

# Correct usage
rsync -av /source/directory/ /target/directory/

Permission denied (publickey) – same troubleshooting as for scp.

# Verify SSH key permissions
chmod 600 ~/.ssh/id_rsa

# Use -e to debug SSH
rsync -avz -e "ssh -v" /source remote_user@host:/target/

Connection unexpectedly closed – cause: remote daemon not running or SSH timeout.

# Check daemon status
systemctl status rsync

# Increase timeout
rsync -avz --timeout=600 /source remote_user@host:/target/

Safe read failed – cause: unstable network.

# Use resume support
rsync -avzP /source remote_user@host:/target/

scp vs rsync Comparison

Feature comparison

Transfer mode : scp – full copy; rsync – incremental copy

Resume support : scp – none; rsync – --partial Compression : scp – -C; rsync – -z Exclude files : scp – not supported; rsync – --exclude Dry‑run preview : scp – not supported; rsync – -n Speed (full copy) : both fast for small data

Speed (incremental) : scp – copies everything; rsync – copies only changed parts

Syntax complexity : scp – simple; rsync – more complex

Typical use case : scp – small files, one‑off transfers; rsync – large files, periodic sync, backups

Selection guidance

Use scp when transferring small files (< 50 MB), a single operation, stable network, and a quick command is needed.

Use rsync when dealing with large or many files, requiring incremental updates, need to exclude patterns, want a dry‑run preview, or need resume capability.

Performance comparison

Assume a source directory with 1 000 files, only 10 changed:

scp transfers all 1 000 files.

rsync transfers only the 10 changed files, saving bandwidth and time.

For a single 10 GB file where only a few hundred MB differ, scp would copy the full 10 GB, whereas rsync might transfer only the delta.

Production Scenarios

Large log migration

Goal: move 500 GB of logs over a 100 Mbps link without disrupting services.

# migrate_logs.sh – example script
#!/bin/bash
SOURCE_USER=root
SOURCE_HOST=192.168.1.50
SOURCE_PATH=/var/log/myapp
TARGET_USER=root
TARGET_HOST=192.168.1.100
TARGET_PATH=/var/log/myapp
BW_LIMIT=10240   # 10 MB/s ≈ 80 Mbps
EXCLUDE_FILE=/tmp/rsync_exclude.txt
cat > "$EXCLUDE_FILE" <<'EOF'
*.current.log
*.tmp
EOF

# First (full) sync
rsync -avzP --bwlimit=$BW_LIMIT --exclude-from="$EXCLUDE_FILE" \
    -e "ssh -p 22" \
    $SOURCE_USER@$SOURCE_HOST:"$SOURCE_PATH/" \
    $TARGET_USER@$TARGET_HOST:"$TARGET_PATH/"

# Second (incremental) sync
rsync -avzP --bwlimit=$BW_LIMIT --exclude-from="$EXCLUDE_FILE" \
    -e "ssh -p 22" \
    $SOURCE_USER@$SOURCE_HOST:"$SOURCE_PATH/" \
    $TARGET_USER@$TARGET_HOST:"$TARGET_PATH/"

Zero‑downtime code deployment

Goal: deploy new code without service interruption, verify in a test directory first, then switch to production.

# deploy.sh – zero‑downtime deployment
#!/bin/bash
set -e
APP_USER=deploy
APP_HOST=192.168.1.100
APP_PATH=/var/www/myapp
TEST_PATH=/var/www/myapp_test
SOURCE_PATH=/home/deploy/releases/$(date +%Y%m%d_%H%M%S)

# Prepare release (placeholder for build steps)
mkdir -p "$SOURCE_PATH"
cp -r /home/deploy/app/* "$SOURCE_PATH/"

# Sync to test environment
rsync -avz --exclude='.env' --exclude='.git' --exclude='node_modules' \
    -e "ssh -p 22" "$SOURCE_PATH/" $APP_USER@$APP_HOST:"$TEST_PATH/"

# Manual or automated verification would occur here
read -p "Confirm switch to production? (yes/no): " CONFIRM
if [[ "$CONFIRM" != "yes" ]]; then
  echo "Deployment cancelled"
  exit 0
fi

# Sync to production
rsync -avz --exclude='.env' --exclude='.git' --exclude='node_modules' \
    -e "ssh -p 22" "$SOURCE_PATH/" $APP_USER@$APP_HOST:"$APP_PATH/"

# Restart application
ssh -p 22 $APP_USER@$APP_HOST "systemctl restart myapp"

# Simple health check
ssh -p 22 $APP_USER@$APP_HOST "curl -s http://localhost/health"

echo "=== Deployment completed ==="

Database backup synchronization

Goal: daily MySQL dump, compress, and sync to an off‑site backup server.

# backup_db.sh – database backup and sync
#!/bin/bash
set -e
DB_HOST=localhost
DB_USER=backup
DB_PASS='password123'
DB_NAME=myapp
BACKUP_DIR=/backup/db
REMOTE_USER=backup
REMOTE_HOST=192.168.1.200
REMOTE_PATH=/backup/myapp
RETENTION_DAYS=30

mkdir -p "$BACKUP_DIR"
BACKUP_FILE="$BACKUP_DIR/${DB_NAME}_$(date +%Y%m%d_%H%M%S).sql"

# Create DB dump
mysqldump -h "$DB_HOST" -u "$DB_USER" -p"$DB_PASS" \
    --single-transaction --routines --triggers --events "$DB_NAME" > "$BACKUP_FILE"

gzip "$BACKUP_FILE"
BACKUP_FILE="${BACKUP_FILE}.gz"

# Sync to remote backup server
rsync -avzP --bwlimit=5120 -e "ssh -p 22" "$BACKUP_FILE" $REMOTE_USER@$REMOTE_HOST:"$REMOTE_PATH/"

# Cleanup old local backups
find "$BACKUP_DIR" -name '*.sql.gz' -mtime +$RETENTION_DAYS -delete

# Backup statistics
echo "Local backup count: $(find "$BACKUP_DIR" -name '*.sql.gz' | wc -l)"
echo "Local backup size: $(du -sh "$BACKUP_DIR" | cut -f1)"

Best Practices

Use SSH key authentication to avoid password prompts.

Always run a dry‑run ( -n) before destructive operations like --delete.

Limit bandwidth with --bwlimit to protect production traffic.

Exclude sensitive files (e.g., .env, *.key, *.pem).

Log transfers for audit and troubleshooting.

Validate results: compare file counts, sizes, and optionally checksums.

Pre‑Transfer Checklist

Verify source file paths.

Verify destination paths and write permissions.

Test SSH connectivity ( ssh user@host).

Ensure sufficient disk space on target ( df -h).

Confirm bandwidth limits will not impact services.

Ensure no sensitive data is unintentionally transferred.

Post‑Transfer Validation Checklist

Compare file counts ( find | wc -l).

Compare total sizes ( du -sh).

Sample MD5 checksums ( md5sum).

Verify log file integrity.

Confirm applications can read the transferred files.