Build an Impenetrable Linux Server: Complete Security Guide from Zero to One

This comprehensive guide walks you through a real intrusion case, then presents a multi‑layer defense strategy—including SSH hardening, Fail2Ban, firewall rules, IDS, file integrity monitoring, advanced techniques like port knocking and honeypots, plus monitoring, alerting, and emergency response—to dramatically improve Linux server security.

MaGe Linux Operations
MaGe Linux Operations
MaGe Linux Operations
Build an Impenetrable Linux Server: Complete Security Guide from Zero to One

Linux Server Security Complete Guide: Build a Fortress from Scratch

Preface: As an ops engineer, I've seen many servers compromised due to poor security. This article shares practical experience to build a complete Linux server security system.

Real Incident: A Dramatic Intrusion

Last year, a late‑night alert showed abnormal CPU usage on a web server. Investigation revealed suspicious processes and a mining trojan, highlighting the need for robust protection.

Attack Path Review:

Attacker brute‑forced SSH to obtain root.

Installed backdoor and established persistence.

Downloaded mining software consuming resources.

Attempted lateral movement within the internal network.

Core Protection Strategies: Multi‑Layer Defense

Layer 1: SSH Hardening

1. Change default port

# Edit SSH config file
vim /etc/ssh/sshd_config

# Change port (recommend 10000‑65535)
Port 22022

# Restart SSH service
systemctl restart sshd

2. Disable direct root login

# In sshd_config
PermitRootLogin no

# Create a regular user and add to sudo group
useradd -m -s /bin/bash admin
usermod -aG sudo admin

3. Configure key‑based authentication

# Generate SSH key pair
ssh-keygen -t ed25519 -C "[email protected]"

# Create authorized_keys on the server
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "your_public_key" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

# Disable password authentication
echo "PasswordAuthentication no" >> /etc/ssh/sshd_config
systemctl restart sshd

Layer 2: Fail2Ban Against Brute‑Force

Install and configure Fail2Ban

# Ubuntu/Debian
apt update && apt install fail2ban -y

# CentOS/RHEL
yum install epel-release -y && yum install fail2ban -y

Custom SSH protection rule

cat > /etc/fail2ban/jail.local <<'EOF'
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 3
ignoreip = 127.0.0.1/8 192.168.0.0/16

[sshd]
enabled = true
port = 22022
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 86400
EOF

systemctl enable fail2ban && systemctl start fail2ban

Check ban status

# View banned IPs
fail2ban-client status sshd

# Unban manually
fail2ban-client set sshd unbanip 192.168.1.100

Layer 3: Firewall Configuration

UFW simple firewall

# Enable UFW
ufw enable

# Default policies
ufw default deny incoming
ufw default allow outgoing

# Allow SSH on custom port
ufw allow 22022/tcp

# Allow web services
ufw allow 80/tcp
ufw allow 443/tcp

# View rules
ufw status verbose

iptables advanced rules

#!/bin/bash
# Flush existing rules
iptables -F
iptables -X
iptables -Z

# Default policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# SSH rate limiting
iptables -A INPUT -p tcp --dport 22022 -m state --state NEW -m recent --set --name SSH
iptables -A INPUT -p tcp --dport 22022 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP
iptables -A INPUT -p tcp --dport 22022 -j ACCEPT

# Web services
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Save rules
iptables-save > /etc/iptables/rules.v4

Layer 4: Intrusion Detection System

Deploy OSSEC‑HIDS

# Download and install OSSEC
wget https://github.com/ossec/ossec-hids/archive/3.6.0.tar.gz
tar -xzf 3.6.0.tar.gz && cd ossec-hids-3.6.0
./install.sh

# Edit /var/ossec/etc/ossec.conf to configure rules

Custom monitoring script

#!/bin/bash
LOG_FILE="/var/log/security_check.log"

check_suspicious_processes() {
    echo "[$(date)] Checking suspicious processes..." >> $LOG_FILE
    ps aux --sort=-%cpu | head -10 | while read line; do
        cpu=$(echo $line | awk '{print $3}')
        if (( $(echo "$cpu > 80" | bc -l) )); then
            echo "Warning: High CPU process: $line" >> $LOG_FILE
        fi
    done
}

check_network_connections() {
    echo "[$(date)] Checking network connections..." >> $LOG_FILE
    netstat -tlnp | grep -E ':(1234|4444|5555|8080)' && echo "Warning: Suspicious listening port" >> $LOG_FILE
}

check_suspicious_processes
check_network_connections

Layer 5: File Integrity Monitoring

Use AIDE

# Install AIDE
apt install aide -y   # Ubuntu/Debian
yum install aide -y   # CentOS/RHEL

# Initialize database
aide --init
mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

# Create check script
cat > /usr/local/bin/aide_check.sh <<'EOF'
#!/bin/bash
aide --check | tee /var/log/aide_check.log
if [ $? -ne 0 ]; then
    echo "File integrity check found issues, see log"
fi
EOF
chmod +x /usr/local/bin/aide_check.sh
# Add to cron (2 AM daily)
echo "0 2 * * * /usr/local/bin/aide_check.sh" | crontab -

Advanced Protection Techniques

Port knocking

# Install knockd
apt install knockd -y

# Configure /etc/knockd.conf
[options]
    UseSyslog

[openSSH]
    sequence    = 7000,8000,9000
    seq_timeout = 5
    command     = /sbin/iptables -A INPUT -s %IP% -p tcp --dport 22022 -j ACCEPT
    tcpflags    = syn

[closeSSH]
    sequence    = 9000,8000,7000
    seq_timeout = 5
    command     = /sbin/iptables -D INPUT -s %IP% -p tcp --dport 22022 -j ACCEPT
    tcpflags    = syn
EOF
systemctl enable knockd && systemctl start knockd

Honeypot deployment

# Install Cowrie SSH honeypot
pip3 install cowrie

# Configure Cowrie to listen on port 22 while real SSH runs on custom port

Log analysis automation

#!/bin/bash
LOGFILE="/var/log/auth.log"
ALERT_EMAIL="[email protected]"

# Detect many SSH failures
failed_attempts=$(grep "Failed password" $LOGFILE | grep "$(date '+%b %d')" | wc -l)
if [ $failed_attempts -gt 50 ]; then
    echo "Warning: $failed_attempts SSH failures today" | mail -s "SSH alert - $(hostname)" $ALERT_EMAIL
fi

# Detect new user creation
new_users=$(grep "new user" /var/log/auth.log | grep "$(date '+%b %d')")
if [ -n "$new_users" ]; then
    echo "Warning: New users created: $new_users" | mail -s "User alert - $(hostname)" $ALERT_EMAIL
fi

Monitoring and Alerting

Lightweight monitoring with Netdata

# Install Netdata
bash <(curl -Ss https://my-netdata.io/kickstart.sh)

# Configure email alerts in /etc/netdata/health_alarm_notify.conf
SEND_EMAIL="YES"
DEFAULT_RECIPIENT_EMAIL="[email protected]"

Custom alert script (Python)

#!/usr/bin/env python3
import psutil, smtplib, time
from email.mime.text import MIMEText

def check_system_health():
    alerts = []
    if psutil.cpu_percent(interval=1) > 80:
        alerts.append(f"CPU usage high: {psutil.cpu_percent()}%")
    if psutil.virtual_memory().percent > 85:
        alerts.append(f"Memory usage high: {psutil.virtual_memory().percent}%")
    for part in psutil.disk_partitions():
        usage = psutil.disk_usage(part.mountpoint).percent
        if usage > 90:
            alerts.append(f"Disk {part.mountpoint} usage high: {usage}%")
    return alerts

def send_alert(alerts):
    if not alerts:
        return
    msg = MIMEText("
".join(alerts))
    msg['Subject'] = f"Server health alert - {time.strftime('%Y-%m-%d %H:%M')}"
    msg['From'] = "[email protected]"
    msg['To'] = "[email protected]"
    print("Sending alert:", "
".join(alerts))
    # Add actual email sending logic here

if __name__ == "__main__":
    alerts = check_system_health()
    send_alert(alerts)

Emergency Response Plan

Steps after detecting intrusion

Immediate isolation : block all inbound traffic with iptables, keep your own IP allowed.

Preserve evidence : tar and compress critical logs.

Clean backdoors : review cron jobs, enabled services, and remove malicious files.

# Isolate network
iptables -A INPUT -j DROP
iptables -I INPUT 1 -s YOUR_IP -j ACCEPT

# Backup logs
tar -czf evidence_$(date +%Y%m%d_%H%M).tar.gz /var/log/auth.log /var/log/syslog /var/log/messages

# List cron jobs and services
crontab -l
cat /etc/crontab
ls -la /etc/cron.*
systemctl list-unit-files --state=enabled
ls -la /etc/init.d/

Summary and Recommendations

By applying the above multi‑layer measures, Linux server security can be significantly improved.

Security level assessment:

🔴 Basic – change SSH port and use key authentication.

🟡 Standard – add Fail2Ban and firewall rules.

🟢 Advanced – intrusion detection and file integrity monitoring.

🔵 Expert – honeypot and automated response.

Best‑practice suggestions:

Regularly update the system and packages.

Apply the principle of least privilege.

Conduct periodic audits and log analysis.

Maintain a documented incident response plan.

Perform security drills regularly.

firewallServer SecurityIDSSSH HardeningFail2Ban
MaGe Linux Operations
Written by

MaGe Linux Operations

Founded in 2009, MaGe Education is a top Chinese high‑end IT training brand. Its graduates earn 12K+ RMB salaries, and the school has trained tens of thousands of students. It offers high‑pay courses in Linux cloud operations, Python full‑stack, automation, data analysis, AI, and Go high‑concurrency architecture. Thanks to quality courses and a solid reputation, it has talent partnerships with numerous internet firms.

0 followers
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.