How to Harden SSH Without Locking Yourself Out

Background and Problem

SSH is the default remote‑administration protocol for Linux servers. The default configuration leaves the service exposed: port 22 is scanned continuously, password authentication can be brute‑forced, root login is permitted, and keys never expire. Aggressive hardening can also lock administrators out.

1. SSH Protocol Basics

1.1 Protocol Versions

Two major versions exist: SSHv1 (deprecated) and SSHv2 (current). SSHv2 uses stronger ciphers (AES, ChaCha20), a full Diffie‑Hellman/ECDH key exchange, and supports public‑key authentication.

# View client version
ssh -V
# View server version
sshd -V
# Test protocol version (v1 should fail)
ssh -1 user@host
# Use SSHv2
ssh -2 user@host

1.2 Connection Process

The connection proceeds through TCP handshake, protocol version exchange, key exchange (Diffie‑Hellman/ECDH), server authentication, user authentication (password, public key, etc.), and finally session establishment.

# Detailed connection debug
ssh -vvv user@host

1.3 Key Types

Supported key types include RSA (commonly 4096 bits), Ed25519 (recommended for security and performance), and ECDSA (may have compatibility issues).

# Generate RSA key (4096 bits)
ssh-keygen -t rsa -b 4096 -C "[email protected]"
# Generate Ed25519 key (recommended)
ssh-keygen -t ed25519 -C "[email protected]"
# Generate ECDSA key (521 bits)
ssh-keygen -t ecdsa -b 521 -C "[email protected]"
# Store key at custom location
ssh-keygen -t ed25519 -f ~/.ssh/my_server_key -C "my_server"
# Add password protection (OpenSSH new format, 100 KDF rounds)
ssh-keygen -t ed25519 -o -a 100 -C "[email protected]"

2. SSH Server Configuration

2.1 Configuration File Locations

The server reads /etc/ssh/sshd_config; the client reads ~/.ssh/config. The focus is on the server file.

2.2 Core Security Settings

# Edit /etc/ssh/sshd_config
vim /etc/ssh/sshd_config
# Recommended settings
Port 2222                     # change default port
Protocol 2                    # disable SSHv1
ListenAddress 0.0.0.0        # or specific IP
PermitEmptyPasswords no
PasswordAuthentication no   # disable password auth
ChallengeResponseAuthentication no
PermitRootLogin no           # disable direct root login
AllowUsers admin [email protected]/24
AllowGroups sshusers
IgnoreRhosts yes
HostbasedAuthentication no
GSSAPIAuthentication no
PubkeyAuthentication yes
ClientAliveInterval 300
ClientAliveCountMax 2
MaxAuthTries 3
MaxSessions 10
LoginGraceTime 30
PermitUserEnvironment no
IgnoreUserKnownHosts yes
PrintMotd no
DisableForwarding yes
X11Forwarding no
Subsystem sftp internal-sftp -l INFO
# SFTP chroot for group sftpusers
Match Group sftpusers
    ChrootDirectory /var/sftp
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

2.3 Verify Configuration

# Syntax check
sshd -t
# Show effective options (example)
sshd -T | grep -E "^passwordauthentication|^permitrootlogin|^pubkeyauthentication"
# Restart service
systemctl restart sshd
systemctl status sshd

3. Public‑Key Authentication

3.1 Principle

The client holds a private key; the server stores the matching public key. During login the client signs random data with its private key, the server verifies the signature with the public key, eliminating password‑based attacks.

3.2 Server Configuration

# /etc/ssh/sshd_config
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
# Permissions
# .ssh directory 700
# authorized_keys file 600

3.3 User‑Side Management

# Generate a key pair
ssh-keygen -t ed25519
# Copy public key to server (simple)
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@host
# Manual copy
cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
# Verify permissions on server
ssh user@host "chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"
# List authorized_keys
ssh user@host "cat ~/.ssh/authorized_keys"
# Append additional keys
ssh user@host "cat >> ~/.ssh/authorized_keys" < ~/.ssh/new_key.pub
# Remove a specific key (example placeholder)
ssh user@host "grep -v 'ssh-rsa AAAAB3...' ~/.ssh/authorized_keys > /tmp/auth_keys && mv /tmp/auth_keys ~/.ssh/authorized_keys"
# Batch management script (example)
#!/bin/bash
AUTH_KEYS_FILE="$HOME/.ssh/authorized_keys"
USER_KEYS_DIR="$HOME/.ssh/user_keys"
mkdir -p "$USER_KEYS_DIR"
for user in alice bob charlie; do
    mkdir -p "$USER_KEYS_DIR/$user"
done
{
    echo "# Alice's keys"
    cat "$USER_KEYS_DIR/alice/"*.pub 2>/dev/null
    echo "# Bob's keys"
    cat "$USER_KEYS_DIR/bob/"*.pub 2>/dev/null
} > "$AUTH_KEYS_FILE"

4. Prevent Locking Yourself Out

4.1 Preparation Before Editing

Keep an existing SSH session open as a rescue channel, back up the current config, and verify syntax before applying changes.

# Backup current config
cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup.$(date +%Y%m%d)
cp /etc/ssh/sshd_config /root/sshd_config.backup
# Verify syntax
sshd -t
# Ensure console access (cloud console, IPMI, iLO, DRAC) and note credentials

4.2 Incremental Changes

Modify one setting at a time, test, then proceed.

# Step 1 – change port only
Port 2222
sshd -t && systemctl restart sshd
# Test new port while keeping the old session open
ssh -p 2222 user@host

4.3 Batch Changes with Ansible

# Change port on all hosts
ansible all -i inventory -m lineinfile \
    -a "path=/etc/ssh/sshd_config regexp='^Port' line='Port 2222'"
# Verify syntax on all hosts
ansible all -i inventory -m command -a "sshd -t"
# Restart service on all hosts
ansible all -i inventory -m systemd -a "name=sshd state=restarted"

4.4 Safe Root‑Login Disable Sequence

# 1. Create a sudo user
useradd -m -s /bin/bash admin
usermod -aG sudo admin
# 2. Add public‑key login for the new user
ssh-copy-id admin@host
# 3. Verify sudo works
ssh admin@host
sudo -i
# 4. After confirming sudo, disable root login
sed -i 's/^PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
# 5. Keep at least one out‑of‑band access method (console, rescue mode)

4.5 Fail2Ban for Brute‑Force Protection

# Install
apt-get install fail2ban   # Debian/Ubuntu
yum install fail2ban       # RHEL/CentOS
# Configure (/etc/fail2ban/jail.local)
[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
findtime = 600
bantime = 3600
# Enable and start
systemctl enable fail2ban
systemctl start fail2ban
# Check status
fail2ban-client status sshd

4.6 Emergency Recovery Methods

Use the cloud provider’s web console to log in as root.

Use a user‑data script at instance launch to fix the config.

Boot into single‑user mode via GRUB (add single or init=/bin/bash to the kernel line), mount the root filesystem read‑write, edit sshd_config, then reboot.

Use the provider’s rescue mode (boot from ISO, mount the original disk, edit the config, then reboot normally).

5. Key Management Best Practices

5.1 Storage and Permissions

# Private key 600
chmod 600 ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_rsa
# Public key 644
chmod 644 ~/.ssh/id_ed25519.pub
# .ssh directory 700
chmod 700 ~/.ssh
# Server‑side authorized_keys 600
chmod 600 ~/.ssh/authorized_keys
# Home directory must not be group‑writable
chmod go-w ~

5.2 Rotation Strategy

# Create a new key for 2024
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_2024 -C "rotation 2024"
# Add new public key to server
ssh user@host "cat >> ~/.ssh/authorized_keys" < ~/.ssh/id_ed25519_2024.pub
# Test new key
ssh -i ~/.ssh/id_ed25519_2024 user@host
# After confirming, remove old key from authorized_keys
ssh user@host "grep -v 'old_key_comment' ~/.ssh/authorized_keys > /tmp/auth && mv /tmp/auth ~/.ssh/authorized_keys"
# Delete old private key locally
mv ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.old

5.3 SSH Agent Forwarding

# Start agent
eval "$(ssh-agent -s)"
# Add key
ssh-add ~/.ssh/id_ed25519
# List keys
ssh-add -l
# Forward agent via jump host (config example)
Host jump-server
    HostName jumphost.example.com
    User admin
    Port 2222
    ForwardAgent yes
# Or use -A on the command line
ssh -A [email protected]

5.4 Managing Multiple Hosts with ~/.ssh/config

# Global defaults
Host *
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 300
    ServerAliveCountMax 2
    StrictHostKeyChecking ask
# Production servers
Host prod-*
    HostName %h.example.com
    User admin
    ForwardAgent no
    LogLevel INFO
# Jump host
Host jump
    HostName jumphost.example.com
    User admin
    Port 2222
    ForwardAgent yes
# Database behind jump host
Host db-1
    HostName 192.168.1.100
    User dbadmin
    ProxyJump jump
    LocalForward 3306 127.0.0.1:3306
# GitHub (password‑less)
Host github
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_ed25519

6. Connection Troubleshooting

6.1 Common Errors

Connection refused : SSH service not running or firewall blocks the port.

Permission denied : Wrong user, missing/incorrect key, or server rejects the key.

Connection timeout : Network unreachable, port blocked, or ACL restrictions.

# Check service status
systemctl status sshd
# Verify listening port
ss -tlnp | grep sshd
# Check firewall rules
iptables -L -n | grep 22
ufw status
firewall-cmd --list-all
# Debug authentication
ssh -vvv user@host
# View server logs
tail -f /var/log/auth.log
journalctl -u sshd -f
# Test port reachability
nc -zv host 22
telnet host 22
# Bind to specific source IP if ACLs apply
ssh -b bind_ip user@host

6.2 Detailed Logging

# Client side verbose output
ssh -vvv user@host
# Server side real‑time log
tail -f /var/log/auth.log
journalctl -u sshd -f
# Force specific KEX algorithm for debugging
ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 user@host

6.3 Service Status Checks

# Service status
systemctl status sshd
# Listening sockets
ss -tlnp | grep sshd
# Local connection test
ssh localhost
# SELinux context (RHEL/CentOS)
getsebool -a | grep ssh
setsebool -P ssh_sysadm_login on

7. Complete SSH Hardening Script

#!/bin/bash
# ssh_hardening.sh - SSH security hardening script
set -euo pipefail

BACKUP_DIR="/root/ssh_backups"
mkdir -p "$BACKUP_DIR"

backup_config() {
    cp /etc/ssh/sshd_config "$BACKUP_DIR/sshd_config.$(date +%Y%m%d_%H%M%S)"
    echo "Backup created in $BACKUP_DIR"
}

verify_sudo() {
    if [ "$(id -u)" -ne 0 ]; then
        echo "This script must be run as root"
        exit 1
    fi
}

verify_admin_user() {
    if ! id admin &>/dev/null; then
        echo "Creating admin user..."
        useradd -m -s /bin/bash admin
        usermod -aG sudo admin
    fi
    if [ ! -f "/home/admin/.ssh/authorized_keys" ]; then
        echo "WARNING: admin user has no SSH keys configured!"
        echo "Please add your public key to /home/admin/.ssh/authorized_keys before continuing"
        read -p "Press Enter to continue anyway..." dummy
    fi
}

apply_config() {
    cat > /etc/ssh/sshd_config <<'EOF'
# SSH Server Configuration
Port 2222
Protocol 2

ListenAddress 0.0.0.0

# Authentication
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no
PermitEmptyPasswords no
MaxAuthTries 3
LoginGraceTime 30

# Security
IgnoreRhosts yes
HostbasedAuthentication no
PermitUserEnvironment no
PrintMotd no
TCPKeepAlive yes

# Idle timeout
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable unused features
X11Forwarding no
AllowTcpForwarding no
PermitTunnel no
AllowAgentForwarding no

# SFTP configuration
Subsystem sftp internal-sftp -l INFO

# Override for SFTP users
Match Group sftpusers
    ChrootDirectory /var/sftp
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no
EOF
    echo "Configuration applied"
}

set_permissions() {
    chmod 644 /etc/ssh/sshd_config
    chown -R admin:admin /home/admin/.ssh
    chmod 700 /home/admin/.ssh
    chmod 600 /home/admin/.ssh/authorized_keys
}

verify_config() {
    echo "Verifying configuration..."
    sshd -t && echo "Configuration syntax OK"
}

restart_service() {
    echo "Restarting SSH service..."
    systemctl restart sshd
    systemctl status sshd --no-pager
}

main() {
    echo "SSH Security Hardening Script"
    echo "============================"
    verify_sudo
    verify_admin_user
    backup_config
    apply_config
    set_permissions
    verify_config
    echo ""
    echo "IMPORTANT: Keep your current SSH session open!"
    echo "Test a new connection before closing this session!"
    echo ""
    read -p "Restart SSH service now? (yes/no): " confirm
    if [ "$confirm" = "yes" ]; then
        restart_service
        echo "SSH service restarted. Please test a new connection."
    else
        echo "Service restart skipped. Run 'systemctl restart sshd' manually when ready."
    fi
}

main "$@"

Conclusion

Effective SSH hardening balances security and usability. Core measures include changing the default port, disabling password authentication in favor of public‑key authentication, prohibiting direct root login, deploying Fail2Ban, and rotating keys regularly. To avoid lockout, always back up the configuration, keep an active SSH session during changes, apply modifications incrementally, and retain an out‑of‑band recovery path such as console or rescue mode.

References

man sshd_config

man ssh_config

man ssh-keygen

OpenSSH official documentation: https://www.openssh.com/security.html

NSA SSH Hardening Guide

CIS Benchmarks for SSH