Why TCP Handshake Needs Three Steps and Teardown Four – Wireshark Packet Analysis

Overview

The article starts by stating that TCP three‑way handshake and four‑step termination are common interview questions, but many candidates only memorize the steps without understanding the underlying packet details. It aims to bridge that gap by dissecting every packet with Wireshark and tcpdump.

TCP Header Structure

The minimal TCP header is 20 bytes, up to 60 bytes with options. Key fields relevant to connection management are:

Sequence Number : 32‑bit ISN generated randomly.

Acknowledgment Number : Valid only when ACK flag is set.

Flags : SYN, ACK, FIN, RST, PSH, URG.

Window Size : 16‑bit, can be scaled with the Window Scale option.

Three‑Way Handshake

Step‑by‑step packet flow:

Client → Server: SYN, Seq=x
Server → Client: SYN+ACK, Seq=y, Ack=x+1
Client → Server: ACK, Seq=x+1, Ack=y+1

Wireshark shows the flags as [S], [S.], and [.]. The article notes that the displayed sequence numbers are relative by default and can be switched to absolute values in Preferences.

Why Three Steps?

Two reasons are given:

Prevent old, delayed SYN packets from establishing a connection that the client never saw.

Both sides must confirm each other's send and receive capabilities.

Four‑Step Connection Termination

Four packets are required because TCP is full‑duplex. The typical flow is:

Client → Server: FIN, Seq=u
Server → Client: ACK, Ack=u+1
Server → Client: FIN, Seq=w
Client → Server: ACK, Ack=w+1

In some cases the ACK and FIN can be merged, resulting in a three‑step termination (FIN+ACK).

TIME_WAIT Details

After the final ACK, the active closer enters TIME_WAIT for 2 MSL (≈60 s on Linux). This state ensures the last ACK reaches the peer and that delayed packets from the old connection expire before a new connection reuses the same 4‑tuple.

Each TIME_WAIT socket consumes ~0.25 KB; large numbers can exhaust the ephemeral port range.

Optimization options include net.ipv4.tcp_tw_reuse=1, increasing net.ipv4.ip_local_port_range, and using connection pools. tcp_tw_recycle has been removed since Linux 4.12.

Wireshark Practical Tips

Filter SYN packets: tcp.flags.syn == 1 && tcp.flags.ack == 0 Filter FIN packets: tcp.flags.fin == 1 Filter RST packets: tcp.flags.reset == 1 Show retransmissions: tcp.analysis.retransmission Follow a TCP stream via Follow → TCP Stream to see the full lifecycle.

tcpdump Command Cheat Sheet

Capture SYN packets:

sudo tcpdump -i any 'tcp[tcpflags] & tcp-syn != 0' -nn

Capture RST packets:

sudo tcpdump -i any 'tcp[tcpflags] & tcp-rst != 0' -nn

Show absolute sequence numbers and timestamps: tcpdump -i any -nn -S -tttt Limit capture size:

sudo tcpdump -i eth0 -s 0 -w /tmp/capture.pcap port 443 -G 3600 -W 24

TCP Retransmission Mechanisms

Two main mechanisms:

Timeout Retransmission (RTO) : Calculated from RTT samples using Jacobson’s algorithm. Minimum RTO is 200 ms, maximum 120 s. Exponential back‑off doubles the timeout on each retry.

Fast Retransmit : Triggered after three duplicate ACKs. The retransmitted segment is marked as [TCP Fast Retransmission] in Wireshark.

SACK : Allows the receiver to inform the sender which blocks have been received, so only missing data is retransmitted.

Congestion Control: BBR vs Cubic

Cubic (Linux default) is loss‑based; it grows the congestion window until a packet loss occurs. It can under‑utilize bandwidth on high‑latency links and cause bufferbloat.

BBR (Google, kernel 4.9+, v3 in 6.x) measures bottleneck bandwidth and minimum RTT, setting the sending rate to the measured bandwidth. Advantages include higher throughput on long‑fat pipes and lower latency. BBR v3 improves fairness when co‑existing with Cubic.

Choosing an Algorithm

Cross‑region or high‑latency traffic: BBR.

Data‑center internal traffic: Cubic (fairness).

Video streaming: BBR.

Mixed Cubic flows: Cubic or BBR v3.

Best‑Practice Recommendations

Connection Management

Increase net.core.somaxconn and net.ipv4.tcp_max_syn_backlog for large listen backlogs.

Enable SYN cookies: net.ipv4.tcp_syncookies=1.

Reuse TIME_WAIT ports: net.ipv4.tcp_tw_reuse=1.

Set appropriate tcp_fin_timeout and tcp_max_tw_buckets.

Use HTTP/2, gRPC, or connection pools to reduce connection churn.

Graceful close: shutdown(SHUT_WR) then close().

Packet Capture

Apply BPF filters to limit capture size.

In production, capture a limited number of packets ( -c) or rotate files ( -G / -W).

Use tshark for batch analysis, e.g.,

tshark -r capture.pcap -q -z io,stat,0,"tcp.analysis.retransmission"

Common TCP Issues and Debugging Steps

Connection timeout : Verify the server is listening ( ss -lnt), capture SYN packets on client and server, check firewall rules.

SYN Flood : Monitor netstat -s | grep "SYNs to LISTEN", enable SYN cookies, increase backlog, add iptables rate‑limit.

RST packets : Identify source, check for services not listening, application‑level aborts, firewall/NAT timeouts.

Full accept queue : Observe netstat -s | grep overflowed, raise somaxconn and application backlog.

Interview Quick Reference

Why three‑way handshake? Prevent stale connections and confirm both sides' capabilities.

Why four‑step termination? Full‑duplex close; can be merged to three when ACK and FIN are combined.

Purpose of TIME_WAIT? Ensure final ACK delivery and let old packets expire.

How to defend against SYN Flood? SYN cookies, larger backlog, rate‑limit.

Difference between RST and FIN? FIN is graceful, RST is abrupt.

BBR vs Cubic? Loss‑based vs bandwidth‑delay‑based.

What happens when accept queue overflows? By default ACK is dropped (client retries); setting tcp_abort_on_overflow=1 makes the kernel send RST.

References

RFC 9293 – TCP specification (2022).

Google BBR Congestion Control research.

Wireshark TCP Analysis documentation.

Linux kernel networking – TCP implementation.

tcpdump manual page.