How to Detect and Mitigate SYN Flood Attacks on Linux Servers

SYN Flood Introduction

SYN Flood is one of the most common DoS/DDoS techniques that exploits a TCP handshake flaw by sending massive forged SYN packets, causing the server to hold many half‑open connections (SYN_RECV) and exhaust resources.

Diagnosis

When traffic drops, check the web server for high CPU, slow SSH, and examine /var/log/messages:

# tail -f /var/log/messages
Apr 18 11:21:56 web5 kernel: possible SYN flooding on port 80. Sending cookies.

Observe a surge in connections, especially SYN_RECV:

# netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
TIME_WAIT 16855
CLOSE_WAIT 21
SYN_SENT 99
FIN_WAIT1 229
FIN_WAIT2 113
ESTABLISHED 8358
SYN_RECV 48965
CLOSING 3
LAST_ACK 313

Normal values are much lower (e.g., SYN_RECV around 259). Record the netstat output to preserve evidence:

# netstat -na > /path/to/evidence.log

Emergency Handling

Identify attacking IP ranges from netstat and temporarily block them with iptables, e.g.,

# iptables -A INPUT -s 173.0.0.0/8 -p tcp --dport 80 -j DROP

Be careful not to block legitimate internal IPs; after analysis, unblock trusted addresses.

Using an F5 Load Balancer

If an F5 appliance is available, configure it to terminate the TCP handshake and forward only established connections to the backend, which dramatically reduces the impact of the attack.

Connection count increased by 5 million during the attack and returned to normal afterward.

F5 CPU usage rose by about 7 % during the attack.

Attack effectiveness dropped because attackers faced higher costs.

Adjusting System Parameters

When an F5 is not available, tune kernel parameters to limit half‑open connections:

Set tcp_synack_retries = 0 so the server does not retry SYN+ACK replies, freeing resources quickly. # sysctl -w net.ipv4.tcp_synack_retries=0 Increase the backlog size: # sysctl -w net.ipv4.tcp_max_syn_backlog=200000 Add the following to /etc/sysctl.conf and apply with sysctl -p:

# vi /etc/sysctl.conf
net.ipv4.tcp_synack_retries = 0
net.ipv4.tcp_max_syn_backlog = 200000

Raise the file‑descriptor limit for each user to handle many simultaneous connections:

# vi /etc/security/limits.conf
*               -       nofile          409600

Do not enable these aggressive settings for the public internet; they should be applied only during an active SYN flood.

Reference Materials

Linux kernel file‑handle limits: /usr/include/linux/fs.h (see blog.yufeng.info/archives/1380).

Detailed TCP variable explanations: frozentux.net/ipsysctl‑tutorial/chunkyhtml/tcpvariables.html.

Conclusion

There is no perfect solution to TCP flood attacks, but the diagnosis steps and mitigation techniques described above can help you quickly understand and reduce their impact on Linux servers.