Mastering TCP: Handshakes, Flow Control, Congestion Management and More

001. TCP vs UDP Differences

TCP is a connection‑oriented, reliable, byte‑stream transport protocol, while UDP is connection‑less.

Compared with UDP, TCP has three core characteristics:

Connection‑oriented : TCP establishes a connection with a three‑way handshake before data exchange; UDP does not.

Reliability : TCP tracks sent and acknowledged data, ensures ordered delivery, and provides stateful control.

Byte‑stream : TCP treats data as a continuous stream, whereas UDP sends discrete datagrams.

002. Three‑Way Handshake

Love Analogy

The handshake is likened to a mutual declaration of love, illustrating the three steps of SYN, SYN‑ACK, and ACK.

Actual Handshake

Both sides start in CLOSED. The server listens ( LISTEN), the client sends SYN ( SYN‑SENT), the server replies with SYN and ACK ( SYN‑RECEIVED), and the client finishes with ACK ( ESTABLISHED on both sides).

Any exchange that requires acknowledgment consumes a TCP sequence number.

Why Not Two or Four Handshakes?

Two handshakes cannot confirm the receiver’s ability; four handshakes add unnecessary delay.

Data Transfer During Handshake

Only the third handshake can carry data; the first two cannot, to avoid amplification attacks.

003. Four‑Way Termination

Process

Both ends start in ESTABLISHED. The client sends FIN ( FIN‑WAIT‑1), the server acknowledges ( CLOSED‑WAIT), the client moves to FIN‑WAIT‑2, the server sends its own FIN ( LAST‑ACK), the client replies with ACK ( TIME‑WAIT) and finally both sides reach CLOSED. The client must wait 2 MSL to ensure stray packets are discarded.

004. Half‑Open Queue and SYN Flood

Before the handshake, the server creates a half‑open (SYN) queue and a full‑open (ACCEPT) queue.

Half‑Open Queue

When a SYN arrives, the server replies with SYN‑ACK and moves the connection to the SYN queue ( SYN_RCVD).

Full‑Open Queue

After the final ACK, the connection moves to the ACCEPT queue, awaiting application acceptance.

SYN Flood Attack

Attackers send massive forged SYNs, filling the SYN queue and exhausting resources. Mitigations include increasing the SYN queue size, reducing SYN‑ACK retries, and using SYN cookies.

005. TCP Header Fields

The header (bytes) includes:

Source and destination ports (four‑tuple identification).

Sequence number (32‑bit, wraps at 2³²).

Initial Sequence Number (ISN) – dynamically generated to prevent prediction attacks.

Acknowledgment number.

Flags: SYN, ACK, FIN, RST, PSH.

Window size (with optional scaling).

Checksum.

Options (Timestamp, MSS, SACK, Window Scale, etc.).

006. TCP Fast Open (TFO)

TFO reduces latency by allowing data in the initial SYN after a SYN cookie is exchanged. The client caches the cookie, sends it with the next SYN and the HTTP request, and the server can reply with data before the three‑way handshake completes.

007. TCP Timestamp Option

The 10‑byte option contains timestamp and timestamp echo. It solves two problems:

Accurate RTT measurement by embedding send times.

Preventing sequence‑number wrap‑around ambiguity.

008. Retransmission Timeout (RTO) Calculation

Classic Method

Smoothed RTT (SRTT) is updated: SRTT = α·SRTT + (1‑α)·RTT (α≈0.8‑0.9). RTO = clamp(β·SRTT, lbound, ubound) (β≈1.3‑2.0).

Standard (Jacobson/Karels) Method

1. Update SRTT: SRTT = (1‑α)·SRTT + α·RTT (α=1/8). 2. Compute RTT variance: RTTVAR = (1‑β)·RTTVAR + β·|RTT‑SRTT| (β=0.25). 3. Final RTO: RTO = μ·SRTT + δ·RTTVAR (μ=1, δ=4).

009. TCP Flow Control

Flow control uses the receiver’s advertised window to limit the sender. The sender’s effective window is min(rwnd, cwnd). The receiver shrinks its window when its buffer fills, informing the sender via ACK.

010. TCP Congestion Control

Two key variables: congestion window (cwnd) and slow‑start threshold (ssthresh). Algorithms:

Slow start: cwnd doubles each RTT until ssthresh.

Congestion avoidance: cwnd increases by 1 per RTT (≈1/cwnd per ACK).

Fast retransmit: three duplicate ACKs trigger immediate retransmission.

Selective ACK (SACK): informs sender which segments arrived.

Fast recovery: halve ssthresh, set cwnd to ssthresh, then increase linearly.

011. Nagle Algorithm and Delayed ACK

Nagle batches small segments until either a full MSS is accumulated or all previous data is ACKed. Delayed ACK waits up to 200‑500 ms to combine ACKs, unless urgent conditions (large segment, quick‑ack mode, out‑of‑order packet) require immediate acknowledgment.

012. TCP Keep‑Alive

Keep‑alive probes detect dead peers. Linux defaults: probe every 7200 s, up to 9 probes, interval 75 s. Many applications disable it because the interval is too long for most use cases.