Understanding SYN Flood Attacks and Effective Linux Mitigation Techniques

SYN Flood (SYN洪水) is a classic Denial‑of‑Service attack that exhausts a server’s TCP connection resources, causing it to stop responding to legitimate connection requests.

TCP Three‑Way Handshake

The TCP connection is established through a three‑way handshake: A sends SYN, B replies with SYN‑ACK, and A completes with ACK. This exchange synchronizes sequence numbers and opens duplex communication channels.

How SYN Flood Works

In a SYN Flood, the attacker (A) sends a large number of SYN packets to the target (B) but never completes the handshake, often spoofing source IP addresses. B allocates kernel resources for each half‑open connection, filling the SYN‑receive queue until it is exhausted.

Basic Mitigation on Linux

Increasing the size of the SYN‑receive queue and reducing the timeout for half‑open connections can raise the bar for attackers. Relevant sysctl parameters include:

/proc/sys/net/ipv4/tcp_max_syn_backlog

/proc/sys/net/ipv4/tcp_synack_retries

SYN Cache

SYN Cache stores half‑open connection information in a hash table, limiting the number of entries per bucket and evicting old entries when full, making it harder for an attacker to fill the queue.

SYN Cookies

SYN Cookies encode the connection state into the SYN‑ACK sequence number, so the server does not keep any state for half‑open connections. Only when the client returns a valid ACK does the server allocate resources.

Both techniques have trade‑offs: SYN Cache and SYN Cookies can reduce the effectiveness of legitimate connections under poor network conditions, and may require additional handling in the application layer.

Attackers often probe the target’s TCP parameters (e.g., timeout, queue limits) and adjust the flood rate to maximize impact while minimizing detection.