TCP Connection Lifecycle: From SYN to FIN, State Machine & Troubleshooting

Global Context: Where TCP Lives

TCP operates at Layer 4 (Transport Layer) of both the OSI and TCP/IP models. It sits above the network layer (IP) and below application protocols such as HTTP, Redis, and MySQL. IP provides unreliable, best‑effort delivery — packets may be lost, reordered, or duplicated. TCP builds a reliable, ordered, byte‑stream, full‑duplex abstraction on top of IP.

Interview perspective: The fundamental difference between TCP and UDP is that TCP simulates a reliable pipe over unreliable IP using sequence numbers, acknowledgments, retransmissions, and windowing, while UDP merely adds ports and a checksum to IP packets.

TCP’s Four Core Problems & Mechanisms

Reliable delivery – solves packet loss via sequence numbers + ACK + timeout retransmission + fast retransmit.

Ordering – solves out‑of‑order arrival via sequence numbers + receive buffer reassembly.

Flow control – prevents sender overwhelming receiver via sliding window (rwnd).

Congestion control – prevents sender overwhelming the network via congestion window (cwnd) with slow start, congestion avoidance, fast retransmit, fast recovery.

Key distinction: Flow control is end‑to‑end (protect the receiver); congestion control is global (protect the network).

Connection Establishment: Three‑Way Handshake

Why a handshake?

“Connection‑oriented” means both ends negotiate parameters before data transfer: Initial Sequence Numbers (ISN), window size, MSS, SACK support, etc. The handshake exchanges these.

Step‑by‑step

Client → Server: SYN=1, seq=x → client enters SYN_SENT.

Server → Client: SYN=1, ACK=1, seq=y, ack=x+1 → server enters SYN_RCVD.

Client → Server: ACK=1, seq=x+1, ack=y+1 → both enter ESTABLISHED.

SYN and FIN each consume one sequence number, hence the +1 in ACKs.

Why three, not two?

Prevent stale connection requests: A delayed duplicate SYN could cause the server to create a spurious connection; the third ACK lets the client reject it with RST.

Verify both directions: Minimum exchange for each side to confirm the other’s send/receive capability.

Engineering note: The third ACK may carry data, but most implementations don’t because the receiver’s window isn’t known yet. HTTP/2’s connection preface is sent immediately after this ACK.

SYN Flood & Half‑Open Queue

The server places incoming SYNs in a half‑connection queue (SYN Queue) . Attackers sending SYNs without completing the handshake fill this queue. Defenses:

SYN Cookies: Encode connection state into the SYN‑ACK’s sequence number; allocate resources only after the final ACK arrives.

Increase tcp_max_syn_backlog.

Enable tcp_syncookies (Linux enables automatically under pressure).

Data Transfer: Reliability & Rate Control

Sequence & Acknowledgment Numbers

TCP views data as an unstructured byte stream. Every byte has a sequence number. ISN is randomized during handshake (not zero) to avoid collisions and improve security. seq = sequence number of first byte in this segment. ack = next expected byte from the peer (cumulative acknowledgment).

Example: Client sends seq=100 with 20 bytes; server replies ack=120 meaning “bytes 100‑119 received, next send from 120”.

Timeout Retransmission & RTO Calculation

If an ACK doesn’t arrive within RTO (Retransmission TimeOut) , the segment is retransmitted. RTO is dynamically derived from measured RTT using Jacobson’s algorithm:

SRTT ← (1 - α) × SRTT + α × RTT_sample   (α = 1/8)
RTTVAR ← (1 - β) × RTTVAR + β × |SRTT - RTT_sample|   (β = 1/4)
RTO = SRTT + 4 × RTTVAR

On timeout, TCP halves ssthresh and sets cwnd = 1, restarting slow start — a severe penalty.

Engineering tip: Use ss -ti to inspect per‑connection RTT, RTO, cwnd, etc. Example: ss -ti dst 10.0.0.1 .

Fast Retransmit

Waiting for RTO (hundreds of ms) is too slow. On receiving 3 duplicate ACKs (4 identical ACKs total), the sender retransmits immediately without waiting for timeout. Three duplicates distinguish loss from reordering.

After fast retransmit, TCP enters Fast Recovery : cwnd = ssthresh + 3 (each dupACK represents a packet that left the network), then grows linearly.

Sliding Window: From Stop‑and‑Wait to Pipeline

Without a window, each segment would wait for its ACK (stop‑and‑wait). The sliding window lets the sender transmit multiple segments as long as they fall within the window. Window slides right on new ACKs; right edge expands as the receiver processes data.

Crucial point: The sliding window is an efficiency mechanism for reliable delivery. Its actual size is constrained by two independent limits:

rwnd (flow control) – receiver’s available buffer.

cwnd (congestion control) – network’s estimated capacity. Effective window = min(rwnd, cwnd).

Flow Control: rwnd

The receiver advertises its free buffer space in the Window field of every ACK. If the buffer fills, rwnd becomes 0 and the sender stops. The sender then sends Zero Window Probes (1‑byte packets) periodically to detect window reopening.

Classic pitfall – Silly Window Syndrome: Receiver advertises tiny windows (e.g., 1 byte), causing the network to be flooded with tiny packets. Fix: receiver delays small window advertisements; sender uses Nagle’s algorithm to coalesce small writes.

Congestion Control: cwnd

Four intertwined algorithms:

Slow Start: cwnd starts at 1 MSS; each ACK increments cwnd by 1 → cwnd doubles every RTT (exponential growth). Despite the name, it’s fast.

Congestion Avoidance: When cwnd ≥ ssthresh, growth becomes additive: cwnd += 1 per RTT (linear). This is AIMD (Additive Increase, Multiplicative Decrease).

Fast Retransmit: On 3 duplicate ACKs, retransmit immediately.

Fast Recovery: After fast retransmit: ssthresh = cwnd / 2; cwnd = ssthresh + 3; each additional dupACK increments cwnd; on new ACK, cwnd = ssthresh and enter congestion avoidance.

Engineering note: Modern Linux defaults to CUBIC (RFC 8312), a cubic‑function‑based algorithm better suited for high‑bandwidth, high‑latency paths. Check with sysctl net.ipv4.tcp_congestion_control .

Unified View

Sliding window provides the pipeline framework.

rwnd matches sender rate to receiver capacity.

cwnd matches sender rate to network capacity.

Taking the minimum ensures neither end nor network is overwhelmed.

Thus reliable delivery, flow control, and congestion control are three co‑equal pillars that cooperate via the sliding window.

Connection Termination: Four‑Way Handshake & TIME_WAIT

Why four steps?

TCP is full‑duplex; each direction closes independently. The server may still have data to send after receiving the client’s FIN, so it ACKs first and sends its own FIN later.

Client → Server: FIN → FIN_WAIT_1.

Server → Client: ACK → client FIN_WAIT_2, server CLOSE_WAIT.

Server → Client: FIN → server LAST_ACK.

Client → Server: ACK → client TIME_WAIT, server CLOSED.

Client waits 2×MSL then CLOSED.

Why 2×MSL in TIME_WAIT?

Ensure the final ACK reaches the server; if lost, the server retransmits FIN and the client can re‑ACK.

Allow any stray packets from the old connection to expire before a new connection with the same 4‑tuple is created.

RFC suggests MSL = 2 minutes; Linux uses 30 seconds ( TCP_TIMEWAIT_LEN), so TIME_WAIT ≈ 1 minute.

Troubleshooting CLOSE_WAIT Pile‑up

Large numbers of CLOSE_WAIT mean the application never called close() . Common causes:

Application bug: exception path skips close.

Connection pool leak.

Peer closed, but local code blocks on read.

Diagnostic commands:

# Count connections per state
ss -ant | awk '{print $1}' | sort | uniq -c
# List CLOSE_WAIT sockets with owning process
ss -antp state close-wait

Engineering note: TIME_WAIT excess is usually harmless (Linux has tcp_tw_reuse ), but CLOSE_WAIT excess always indicates an application bug. Don’t confuse the two.

TCP Finite State Machine: 11 States

The entire lifecycle maps to a deterministic FSM with three phases: Establish → Transfer → Close . Understanding why each intermediate state exists (e.g., TIME_WAIT waits for final ACK and old packets) removes the need for rote memorization.

Using the FSM for Production Debugging

Scenario 1: CLOSE_WAIT flood

State machine says CLOSE_WAIT → LAST_ACK on close(). Stuck in CLOSE_WAIT ⇒ application didn’t call close. Check exception handling, pool leaks, blocked reads.

Scenario 2: SYN_RECV flood

SYN_RECV lives in the half‑connection queue. Surge = SYN flood attack. Inspect queue length:

ss -lnt | grep :80   # Send‑Q = current SYN_RECV count

Enable tcp_syncookies, raise tcp_max_syn_backlog.

Scenario 3: FIN_WAIT_2 flood

Active closer sent FIN, got ACK, but peer never sends FIN. Peer likely stuck in CLOSE_WAIT (peer bug) or crashed. Linux times out FIN_WAIT_2 after tcp_fin_timeout (default 60 s).

Scenario 4: TIME_WAIT exhaustion

High‑rate short connections can exhaust ephemeral ports. Mitigations: net.ipv4.tcp_tw_reuse = 1 – reuse TIME_WAIT ports for new outbound connections.

Widen net.ipv4.ip_local_port_range.

Prefer long‑lived connections (root fix).

Warning: tcp_tw_recycle was removed in Linux 4.12; do not use it.

Packet Analysis with Wireshark

Capture Commands

# Save full packets to file
sudo tcpdump -i any -s 0 -w http_capture.pcap 'tcp port 80'
# Live summary
sudo tcpdump -i any -nn 'tcp port 80'

TCP Header Fields (20 bytes base)

Key flags for analysis: SYN – synchronize sequence numbers (connection setup). ACK – acknowledgment field valid (set on all packets after handshake). FIN – sender finished, requests close. RST – abort connection (error). PSH – push data to application immediately. URG – urgent pointer valid.

Sample curl HTTP Request Sequence

SYN, seq=0

(relative; real ISN random). SYN+ACK, ack=1 (SYN consumes 1 sequence). ACK, seq=1. PSH+ACK, seq=1, 77 bytes data. ACK, ack=78 (1+77).

… data exchange … FIN+ACK, seq=78. ACK, ack=79 (FIN consumes 1 sequence).

Useful Wireshark Display Filters

# Handshake / teardown only
tcp.flags.syn == 1 or tcp.flags.fin == 1
# Retransmissions (network quality)
tcp.analysis.retransmission
# Zero window (receiver backlog)
tcp.analysis.zero_window
# Follow full TCP stream
Right‑click → Follow → TCP Stream

Pro tip: Filter with tcp.flags.syn==1 to isolate handshakes; use tcp.analysis.retransmission to spot loss‑induced retransmits.

Recap

TCP’s elegance lies in solving “reliable service over unreliable network” with a compact set of mechanisms: sequence numbers, cumulative ACKs, adaptive retransmission, sliding window, end‑to‑end flow control, and network‑wide congestion control (AIMD/CUBIC). Since RFC 793 (1981) it remains the internet’s backbone.

References

RFC 793 – TCP base specification

RFC 5681 – Congestion Control

RFC 8312 – CUBIC

TCP/IP Illustrated, Volume 1