How to Harden Linux Servers: A Step‑by‑Step Guide to Secure SSH, Accounts, Firewalls, and Auditing

Overview

The article presents a systematic approach to Linux host hardening, emphasizing that security is a continuous risk curve rather than a binary state. It identifies three typical misconceptions: relying on antivirus/EDR alone, assuming a firewall is sufficient, and believing a single hardening step is permanent.

Baseline Comparison

A CIS‑style baseline table maps six hardening domains (account security, SSH, firewall, kernel parameters, file permissions, audit logs) to key points, CIS focus, typical benefits, and common risks.

Applicable Scenarios

Internet‑exposed jump hosts and edge nodes

Critical internal services (payment, order, account)

Pre‑deployment image standardisation

Compliance requirements (ISO27001, SOC2, etc.)

Post‑incident remediation

Environment Requirements

Supported OS: RHEL/CentOS 7‑9, Rocky/Alma, Ubuntu 20.04‑24.04, Debian 11/12

OpenSSH 7.4+ (recommended 8.0+)

Firewall: firewalld or nftables/iptables

Audit: auditd

Hardening Coordinate System

The hardening process is divided into five layers, each with a clear target, checkpoint, and evidence:

Entry layer – SSH, port exposure, source restrictions

Identity layer – Account policies, lockout, MFA

Privilege layer – sudo, SUID, file permissions

Kernel/Network layer – sysctl settings

Observability layer – auditd, logging, alerts

Detailed Steps

2.1 Default Values and Boundaries

Key default files are highlighted, e.g., /etc/login.defs for password policies and /etc/pam.d/* for PAM stacks. Sample verification commands are provided:

# List password‑policy files
ls -l /etc/login.defs /etc/security/pwquality.conf 2>/dev/null || true
# Check faillock for a user
faillock --user alice

2.2 Common Pitfalls and Risk Matrix

Typical dangerous actions include writing PasswordAuthentication no without testing key‑based login, disabling root login without a sudo fallback, and applying a default‑deny firewall without opening health‑check ports. Each pitfall is paired with a safer alternative.

2.3 Recommended Adjustments, Rollback, and Validation

For each of the six domains, concrete configuration snippets, rollback commands, and validation steps are listed. Highlights include:

Account security : set shells to /sbin/nologin, lock unused accounts, enforce password complexity via /etc/security/pwquality.conf, and configure /etc/login.defs for expiration.

SSH hardening : disable password auth, enforce public‑key only, set PermitRootLogin no, limit users/groups, tighten algorithms, and configure idle timeout.

Firewall : default‑deny zone, allow only required services/ports, and use rich rules for source‑IP whitelisting.

Kernel parameters : disable redirects, source routing, enable reverse‑path filtering, and apply SYN‑cookie and TCP backlog settings.

File permissions : enforce 600 for /etc/shadow, 440 for /etc/sudoers, and secure /etc/ssh/sshd_config.

Audit logging : audit only high‑value files and commands, set buffer size ( -b 8192) and failure action ( -f 1), and rotate logs.

2.4 Scripts for Automation

Five ready‑to‑use Bash scripts are provided: collect-hardening-evidence.sh – gathers current configuration, sysctl values, firewall state, and audit status. backup-hardening-configs.sh – creates timestamped backups of all critical files. verify-hardening.sh – runs a checklist of configuration tests and exits with a non‑zero status on failure. rollback-hardening.sh – restores files from a backup directory and reloads services. bucket-security-logs.sh – extracts SSH failures, sudo usage, account changes, and key audit events into separate files for incident analysis.

Real‑World Cases

Case 1 – SSH Brute‑Force on a Jump Host

Symptoms: SIEM alerts for many failed logins and high sshd CPU usage. Investigation revealed PasswordAuthentication yes and no source‑IP whitelist. Immediate mitigation used firewalld rich rules to allow only the operations subnet; long‑term fix disabled password auth, enabled faillock, and set MaxAuthTries 3.

Case 2 – rp_filter Causing Network Timeouts

After applying a strict rp_filter=1 on a multi‑NIC gateway, asymmetric return packets were dropped, leading to intermittent timeouts. The fix relaxed the setting to rp_filter=2 (loose) or per‑interface configuration, followed by sysctl --system and verification of traffic stability.

Case 3 – /tmp noexec Breaking Deployment Scripts

A deployment pipeline failed with “Permission denied” because /tmp was mounted with noexec. The resolution was either to remount /tmp without noexec for the affected hosts or to move temporary execution to a dedicated directory (e.g., /var/lib/app/tmp) and keep /dev/shm strict.

Case 4 – Over‑Auditing Leading to Disk Exhaustion

Auditd was configured to log every syscall, causing several gigabytes of logs per hour and filling /var. The remediation reduced the rule set to the critical items listed in section 2.3.20 and configured /etc/audit/auditd.conf with max_log_file = 50, num_logs = 10, and max_log_file_action = ROTATE.

Best Practices and Caveats

Always back up before any change (use the backup script).

Maintain an out‑of‑band console for SSH or sudo changes.

Template hardening per host role (jump host vs. business server vs. gateway).

Perform gray‑scale testing and performance validation for sysctl and mount options.

Prioritise attack‑surface reduction (IP whitelisting, port minimisation) before deep auditing.

Ensure audit rules are sustainable – focus on high‑value events and set rotation.

Monitoring and Alerting

Key metrics to expose to a monitoring system include SSH failure rate, unexpected successful logins, sudo usage spikes, audit log growth, and /var usage. Example Prometheus alert rules are provided for SSH burst failures and rapid audit‑log growth.

Verification Checklist After Hardening

# Verify sshd syntax
sshd -t
# Confirm critical sshd options
sshd -T | grep -E 'permitrootlogin|passwordauthentication|loglevel'
# Check firewall state (firewalld example)
firewall-cmd --state && firewall-cmd --list-all
# Validate sysctl settings
sysctl -a | egrep 'rp_filter|accept_redirects|send_redirects|tcp_syncookies|randomize_va_space|kptr_restrict|dmesg_restrict|fs.suid_dumpable'
# Verify file permissions
stat -c '%a %U:%G %n' /etc/shadow /etc/sudoers /etc/ssh/sshd_config
# Ensure auditd is active and rules loaded
systemctl is-active auditd && auditctl -s

Conclusion

Effective Linux hardening requires a repeatable, rollback‑able workflow, role‑based configuration templates, thorough validation, and continuous monitoring. By following the step‑by‑step adjustments, using the supplied automation scripts, and integrating the recommended alerts, operators can achieve a secure yet maintainable production environment.