Understanding TCP Connection States and Linux Backlog Tuning

TCP Connection States Overview

The TCP protocol defines eleven states that a socket can be in during its lifetime:

LISTEN : server creates a socket, binds, listens and blocks on accept().

SYN_SENT : client actively opens a connection with connect(), sends a SYN and waits for SYN+ACK.

SYN_RECV : server receives the SYN, replies with SYN+ACK and waits for the client’s ACK; the request sits in the SYN (half‑open) queue.

ESTABLISHED : both sides have exchanged SYN, SYN+ACK and ACK; data transfer can begin.

FIN_WAIT1 : the active closer sends a FIN and waits for the peer’s ACK.

CLOSE_WAIT : the passive side has received the peer’s FIN, acknowledges it and waits for the application to close.

FIN_WAIT2 : after receiving the ACK for its FIN, the active side waits for the peer’s FIN.

LAST_ACK : the passive side sends its own FIN+ACK and waits for the final ACK.

TIME_WAIT : the active side has received the final ACK, stays in this state for 2 MSL to ensure the peer received the ACK.

CLOSING : both sides have sent FINs but are waiting for the final ACK.

CLOSED : the connection is fully terminated.

Three‑Way Handshake

The handshake proceeds as follows:

Client sends SYN=J → enters SYN_SENT .

Server replies SYN=K, ACK=J+1 → enters SYN_RECV and places the request in the SYN queue.

Client acknowledges with ACK=K+1 → both sides transition to ESTABLISHED and the request moves to the accept queue.

The handshake also negotiates options such as MSS (maximum segment size) and SACK_PERM (selective acknowledgment) to improve reliability and efficiency.

Half‑Open and Full‑Open Queues

When a SYN arrives, the kernel stores the request in the half‑open (SYN) queue whose length is controlled by net.ipv4.tcp_max_syn_backlog (default 2048). After the handshake completes, the socket moves to the accept (full‑open) queue whose size is the minimum of the listen() backlog argument and net.core.somaxconn (default 128). Typical commands to inspect these values are:

cat /proc/sys/net/ipv4/tcp_max_syn_backlog

cat /proc/sys/net/core/somaxconn

Both parameters can be tuned via sysctl or by editing /etc/sysctl.conf. For example:

echo '1' > /proc/sys/net/ipv4/tcp_abort_on_overflow

Increasing the backlog reduces the chance of connection drops under high load, while decreasing it can mitigate SYN‑flood attacks.

Common Issues and Diagnostics

SYN‑flood attacks fill the half‑open queue, causing legitimate connections to be dropped. When the accept queue overflows, the kernel may either silently discard new connections (default) or send a RST if tcp_abort_on_overflow is set to 1.

Useful diagnostic commands: netstat -s | egrep "listen|LISTEN" – shows listen‑queue overflow counters. netstat -s | grep TCPBacklogDrop – reports accept‑queue drops. ss -lnt – lists listening sockets with their current backlog usage.

When overflow counters increase, consider raising tcp_max_syn_backlog, somaxconn, or enabling SYN cookies ( net.ipv4.tcp_syncookies=1).

Linux TCP/IP Kernel Parameters

Key sysctl knobs that affect TCP connection handling:

tcp_abort_on_overflow – 0 (drop) or 1 (send RST) when the accept queue is full.

net.core.netdev_max_backlog – maximum packets queued per network interface when the kernel cannot keep up.

net.ipv4.tcp_max_orphans – limit of TCP sockets without a user‑space reference.

net.ipv4.tcp_max_syn_backlog – size of the SYN (half‑open) queue.

net.core.somaxconn – default maximum size of the accept (full‑open) queue.

net.ipv4.tcp_timestamps – enable/disable TCP timestamps.

net.ipv4.tcp_synack_retries – number of SYN+ACK retransmissions before giving up.

net.ipv4.tcp_syn_retries – number of SYN retransmissions by the client.

net.ipv4.tcp_syncookies – enable SYN cookies to defend against SYN floods.

net.ipv4.tcp_tw_reuse and net.ipv4.tcp_tw_recycle – allow reuse/recycling of TIME_WAIT sockets.

net.ipv4.tcp_fin_timeout – timeout for sockets in FIN_WAIT2.

net.ipv4.tcp_keepalive_time – interval for keep‑alive probes.

net.ipv4.ip_local_port_range – range of ephemeral ports for outbound connections.

net.ipv4.tcp_max_tw_buckets – maximum number of TIME_WAIT sockets.

Adjusting these parameters can improve connection throughput, reduce resource consumption, and mitigate denial‑of‑service conditions on high‑traffic servers.