How to Harden SSH Securely Without Locking Yourself Out

Background and Problem

SSH (Secure Shell) is the primary protocol for remote management of Linux servers. The default configuration is insecure: port 22 receives brute‑force attacks, password authentication can be cracked or leaked, and root login is risky. Over‑aggressive hardening can also lock administrators out.

1. SSH Protocol Basics

1.1 Protocol Versions

SSH has two versions, SSHv1 (deprecated) and SSHv2. Modern systems use SSHv2, which provides stronger ciphers (AES, ChaCha20), a full Diffie‑Hellman key‑exchange, and public‑key authentication.

# Show client version
ssh -V
# Show server version
sshd -V
# Test protocol version (v1 should fail)
ssh -1 user@host
ssh -2 user@host

1.2 Connection Process

The connection consists of TCP handshake, protocol version exchange, key exchange (Diffie‑Hellman/ECDH), server authentication, user authentication (password, public key, etc.), and session establishment.

# Detailed debug output
ssh -vvv user@host
# Output sections: debug1 (connection & key exchange), debug2 (authentication), debug3 (session)

1.3 Key Types

Supported key types include RSA (commonly 4096‑bit), Ed25519 (modern, fast, secure), and ECDSA (curve‑dependent, may have compatibility issues).

# Generate RSA key (4096‑bit)
ssh-keygen -t rsa -b 4096 -C "[email protected]"
# Generate Ed25519 key (recommended)
ssh-keygen -t ed25519 -C "[email protected]"
# Generate ECDSA key (521‑bit)
ssh-keygen -t ecdsa -b 521 -C "[email protected]"
# Store key at a custom location
ssh-keygen -t ed25519 -f ~/.ssh/my_server_key -C "my_server"
# Add password protection (new OpenSSH format, -a sets KDF rounds)
ssh-keygen -t ed25519 -o -a 100 -C "[email protected]"

2. SSH Server Configuration

2.1 Configuration File Locations

The server configuration file is /etc/ssh/sshd_config; the client configuration is ~/.ssh/config. This guide focuses on the server side.

# View default configuration
man sshd_config
# Show effective configuration
sshd -T
# Check syntax without restarting
sshd -t
# Test configuration for a specific user/host
sshd -T -C user=root,host=localhost,addr=127.0.0.1

2.2 Core Security Settings

Key settings that directly affect security are listed below. Apply each change, verify, then restart the service.

# Edit server config
vim /etc/ssh/sshd_config
# Recommended settings:
Port 2222                     # Change default port
Protocol 2                    # Disable SSHv1
ListenAddress 0.0.0.0         # Optional: bind to all interfaces
PermitEmptyPasswords no
PasswordAuthentication no
ChallengeResponseAuthentication no
PermitRootLogin no
AllowUsers admin [email protected]/24
AllowGroups sshusers
IgnoreRhosts yes
HostbasedAuthentication no
GSSAPIAuthentication no
PubkeyAuthentication yes
ClientAliveInterval 300
ClientAliveCountMax 2
MaxAuthTries 3
MaxSessions 10
LoginGraceTime 30
PermitUserEnvironment no
IgnoreUserKnownHosts yes
PrintMotd no
DisableForwarding yes
X11Forwarding no
Subsystem sftp internal-sftp -l INFO
Match Group sftpusers
    ChrootDirectory /var/sftp
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

2.3 Verify Configuration Before Restart

# Check syntax
sshd -t
# If no output, syntax is OK
# Show selected effective options
sshd -T | grep -E "^passwordauthentication|^permitrootlogin|^pubkeyauthentication"
# Restart service
systemctl restart sshd
systemctl status sshd

3. Public‑Key Authentication

3.1 Principle

Clients hold a private key; the public key is placed on the server. During login the client signs random data with the private key, the server verifies the signature with the public key. This eliminates password‑based brute‑force attacks.

3.2 Server‑Side Configuration

# Enable public‑key auth
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
# Ensure correct permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

3.3 Client‑Side Setup and Management

# Generate a new key pair
ssh-keygen -t ed25519
# Copy public key to server (simplest)
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@host
# Manual copy
cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
# Verify permissions on server
ssh user@host "chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"
# List keys loaded in the agent
ssh-add -l

3.4 Managing authorized_keys

# View current keys
ssh user@host "cat ~/.ssh/authorized_keys"
# Append a new key
ssh user@host "cat >> ~/.ssh/authorized_keys" < ~/.ssh/new_key.pub
# Remove a specific key (example)
ssh user@host "grep -v 'ssh-rsa AAAAB3...' ~/.ssh/authorized_keys > /tmp/auth_keys && mv /tmp/auth_keys ~/.ssh/authorized_keys"
# Batch management script (example)
#!/bin/bash
AUTH_KEYS_FILE="$HOME/.ssh/authorized_keys"
USER_KEYS_DIR="$HOME/.ssh/user_keys"
mkdir -p "$USER_KEYS_DIR"
for user in alice bob charlie; do
    mkdir -p "$USER_KEYS_DIR/$user"
done
{
    echo "# Alice's keys"
    cat "$USER_KEYS_DIR/alice/"*.pub 2>/dev/null
    echo "# Bob's keys"
    cat "$USER_KEYS_DIR/bob/"*.pub 2>/dev/null
} > "$AUTH_KEYS_FILE"

4. Avoid Locking Yourself Out

4.1 Preparation Before Editing

Keep an active SSH session as an emergency recovery channel, back up the current configuration, and ensure console or out‑of‑band access.

# Backup configuration
cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup.$(date +%Y%m%d)
cp /etc/ssh/sshd_config /root/sshd_config.backup
# Verify syntax
sshd -t

4.2 Incremental Changes

Modify only one setting at a time, test the new configuration, then proceed to the next change.

# Step 1: Change port only
Port 2222
sshd -t && systemctl restart sshd
# Test new port while keeping the old session open
ssh -p 2222 user@host
# After confirming, move to the next setting

4.3 Bulk Changes with Ansible

# Change port on all hosts
ansible all -i inventory -m lineinfile \
    -a "path=/etc/ssh/sshd_config regexp='^Port' line='Port 2222'"
# Verify syntax on all hosts
ansible all -i inventory -m command -a "sshd -t"
# Restart SSH service
ansible all -i inventory -m systemd -a "name=sshd state=restarted"

4.4 Proper Order for Disabling Root Login

# 1. Create a sudo‑capable admin user
useradd -m -s /bin/bash admin
usermod -aG sudo admin
# 2. Set up public‑key login for admin
ssh-copy-id admin@host
# 3. Verify sudo works
ssh admin@host
sudo -i
# 4. After verification, disable root login
PermitRootLogin no
# 5. Keep at least one out‑of‑band access method (console, rescue mode)

4.5 Fail2Ban for Brute‑Force Protection

# Install Fail2Ban
apt-get install fail2ban   # Debian/Ubuntu
yum install fail2ban       # RHEL/CentOS
# Configure jail for SSH on the new port
cat > /etc/fail2ban/jail.local <<'EOF'
[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
findtime = 600
bantime = 3600
EOF
# Enable and start service
systemctl enable fail2ban
systemctl start fail2ban
# Check status
fail2ban-client status sshd

4.6 Emergency Recovery Methods

Cloud console login (web UI) to access the instance as root.

Cloud user‑data script executed on next boot to fix the config.

Boot into single‑user mode by editing GRUB (add single or init=/bin/bash), mount the root filesystem read‑write, and edit /etc/ssh/sshd_config.

Provider’s rescue mode: attach the disk to another instance, mount it, edit the config, then reboot.

5. Key Management Best Practices

5.1 Storage and Permissions

# Private key permissions
chmod 600 ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_rsa
# Public key can be 644
chmod 644 ~/.ssh/id_ed25519.pub
# .ssh directory must be 700
chmod 700 ~/.ssh
# Server‑side authorized_keys must be 600
chmod 600 ~/.ssh/authorized_keys
# Home directory should not be group‑writable
chmod go-w ~

5.2 Key Rotation

# Create a new key for 2024
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_2024 -C "rotation 2024"
# Add new public key to server
ssh user@host "cat >> ~/.ssh/authorized_keys" < ~/.ssh/id_ed25519_2024.pub
# Test the new key
ssh -i ~/.ssh/id_ed25519_2024 user@host
# After confirming, remove old key from authorized_keys
ssh user@host "grep -v 'old_key_comment' ~/.ssh/authorized_keys > /tmp/auth && mv /tmp/auth ~/.ssh/authorized_keys"
# Delete old private key locally
mv ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.old

5.3 Using SSH Agent Forwarding

# Start the agent
eval "$(ssh-agent -s)"
# Add key to the agent
ssh-add ~/.ssh/id_ed25519
# List loaded keys
ssh-add -l
# Enable forwarding in ~/.ssh/config
Host jump-server
    HostName jumphost.example.com
    User admin
    Port 2222
    ForwardAgent yes
# Or use -A on the command line
ssh -A [email protected]

5.4 Managing Multiple Hosts with Config File

# ~/.ssh/config example
Host *
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 300
    ServerAliveCountMax 2
    StrictHostKeyChecking ask

Host prod-*
    HostName %h.example.com
    User admin
    ForwardAgent no
    LogLevel INFO

Host jump
    HostName jumphost.example.com
    User admin
    Port 2222
    ForwardAgent yes

Host db-1
    HostName 192.168.1.100
    User dbadmin
    ProxyJump jump
    LocalForward 3306 127.0.0.1:3306

Host github
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_ed25519

6. Connection Troubleshooting

6.1 Common Errors

Connection refused

# Check if sshd is running
systemctl status sshd
# Verify listening ports
ss -tlnp | grep sshd
netstat -tlnp | grep sshd
# Check firewall rules
iptables -L -n | grep 22
ufw status
firewall-cmd --list-all

Permission denied

# Verify username
ssh user@host
# Debug authentication
ssh -vvv user@host
# Inspect server logs
tail -f /var/log/auth.log
journalctl -u sshd -f
# Specify key explicitly
ssh -i ~/.ssh/specific_key user@host

Connection timeout

# Test network reachability
ping -c 4 host
traceroute host   # Linux
tracert host      # Windows
# Test port accessibility
nc -zv host 22
telnet host 22
# Try binding to a specific source IP
ssh -b bind_ip user@host

6.2 Detailed Logging

# Client side verbose output
ssh -vvv user@host
# Server side real‑time logs
tail -f /var/log/auth.log
journalctl -u sshd -f
# Force a specific KEX algorithm for debugging
ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 user@host

6.3 Service Status Checks

# Service status
systemctl status sshd
# Listening sockets
ss -tlnp | grep sshd
# Test local connection
ssh localhost
# SELinux context (RHEL/CentOS)
getsebool -a | grep ssh
setsebool -P ssh_sysadm_login on

7. Complete SSH Hardening Script

#!/bin/bash
# ssh_hardening.sh - SSH security hardening script
set -euo pipefail

BACKUP_DIR="/root/ssh_backups"
mkdir -p "$BACKUP_DIR"

backup_config() {
    cp /etc/ssh/sshd_config "${BACKUP_DIR}/sshd_config.$(date +%Y%m%d_%H%M%S)"
    echo "Backup created in $BACKUP_DIR"
}

verify_sudo() {
    if [ "$(id -u)" -ne 0 ]; then
        echo "This script must be run as root"
        exit 1
    fi
}

verify_admin_user() {
    if ! id admin &>/dev/null; then
        echo "Creating admin user..."
        useradd -m -s /bin/bash admin
        usermod -aG sudo admin
    fi
    if [ ! -f "/home/admin/.ssh/authorized_keys" ]; then
        echo "WARNING: admin user has no SSH keys configured!"
        echo "Please add your public key to /home/admin/.ssh/authorized_keys before continuing"
        read -p "Press Enter to continue anyway..." dummy
    fi
}

apply_config() {
    cat > /etc/ssh/sshd_config <<'EOF'
# SSH Server Configuration
Port 2222
Protocol 2
ListenAddress 0.0.0.0

# Authentication
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no
PermitEmptyPasswords no
MaxAuthTries 3
LoginGraceTime 30

# Security
IgnoreRhosts yes
HostbasedAuthentication no
PermitUserEnvironment no
PrintMotd no
TCPKeepAlive yes

# Idle timeout
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable unused features
X11Forwarding no
AllowTcpForwarding no
PermitTunnel no
AllowAgentForwarding no

# SFTP configuration
Subsystem sftp internal-sftp -l INFO

# Override for SFTP users
Match Group sftpusers
    ChrootDirectory /var/sftp
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no
EOF
    echo "Configuration applied"
}

set_permissions() {
    chmod 644 /etc/ssh/sshd_config
    chown -R admin:admin /home/admin/.ssh
    chmod 700 /home/admin/.ssh
    chmod 600 /home/admin/.ssh/authorized_keys
}

verify_config() {
    echo "Verifying configuration..."
    sshd -t && echo "Configuration syntax OK"
}

restart_service() {
    echo "Restarting SSH service..."
    systemctl restart sshd
    systemctl status sshd --no-pager
}

main() {
    echo "SSH Security Hardening Script"
    echo "============================"
    verify_sudo
    verify_admin_user
    backup_config
    apply_config
    set_permissions
    verify_config
    echo ""
    echo "IMPORTANT: Keep your current SSH session open!"
    echo "Test a new connection before closing this session!"
    echo ""
    read -p "Restart SSH service now? (yes/no): " confirm
    if [ "$confirm" = "yes" ]; then
        restart_service
        echo "SSH service restarted. Please test a new connection."
    else
        echo "Service restart skipped. Run 'systemctl restart sshd' manually when ready."
    fi
}

main "$@"

8. Conclusion

SSH hardening must balance security and availability. Over‑restrictive settings can lock administrators out, while insufficient hardening leaves the service exposed. Core recommendations include changing the default port, disabling password authentication in favor of public‑key auth, disabling direct root login, deploying Fail2Ban, and rotating keys regularly. Always back up the configuration, keep an active session for recovery, apply changes incrementally, and have console or rescue‑mode access as a final fallback.

References

man sshd_config

man ssh_config

man ssh-keygen

OpenSSH official security page: https://www.openssh.com/security.html

NSA SSH Hardening Guide

CIS Benchmarks for SSH