Why TCP Can Still Lose Messages: A Deep Dive into Packet Loss and Troubleshooting

Although the story begins with a personal anecdote about a programmer helping a friend understand why her boyfriend’s messages seem delayed, the core of the article is a technical walkthrough of how a data packet is transmitted in a chat application and where loss can happen.

Packet Sending Process

When a user sends a message, the text is copied from the user‑space chat client to the kernel’s send buffer, then traverses the transport (TCP), network, and data‑link layers before reaching the physical NIC. The packet passes through flow‑control queues (qdisc) and a RingBuffer before being emitted onto the network.

On the receiving side, the NIC triggers a DMA transfer to place the packet into a receive RingBuffer, generates a hardware interrupt, and the kernel’s soft‑irq handler (ksoftirqd) moves the data up the stack until it is copied back into the user‑space chat client.

Common Scenarios of Packet Loss

Connection‑establishment loss

During the TCP three‑way handshake, half‑open connections are stored in a backlog queue. If the queue overflows, new SYN packets are dropped.

# netstat -s | grep overflowed
# netstat -s | grep -i "SYNs to LISTEN sockets dropped"

Flow‑control loss

If the transmit queue (txqueuelen) is too short for the burst of outgoing data, packets are dropped. The ifconfig output shows the TX dropped counter.

# ifconfig eth0

Increasing the queue length (e.g., txqueuelen 1500) can alleviate this.

NIC‑related loss

RingBuffer overflow, faulty cables, or insufficient NIC speed can cause drops. The ethtool -S eth0 command reveals rx_queue_0_drops. Adjusting RingBuffer size with ethtool -G eth0 rx 4096 tx 4096 helps.

# ethtool -S eth0 | grep rx_queue_0_drops
# ethtool -G eth0 rx 4096 tx 4096

Receive‑buffer loss

When the TCP receive window reaches zero, the sender receives a zero‑window advertisement and may drop packets if data continues to arrive. The kernel’s TcpExt: TCPRcvQDrop counter tracks such events.

# cat /proc/net/netstat | grep TCPRcvQDrop

End‑to‑end network loss

Intermediate routers or links can drop packets. Use ping to check basic reachability and packet loss percentage, and mtr (or mtr -u for UDP) to see per‑hop loss.

# ping baidu.com
# mtr -r baidu.com
# mtr -u -r baidu.com

TCP Reliability Limits

TCP guarantees delivery only up to the transport layer. After the receiver’s TCP stack acknowledges a segment, the application must still read the data. If the app crashes or the device runs out of memory, the message may never reach the user, effectively “lost” despite TCP’s retransmission.

Why a Server Is Needed

Removing the server simplifies the model but introduces problems: each client would need a separate connection for every peer, higher resource consumption, security concerns, and version‑compatibility issues. A central server allows a single persistent connection, handles authentication, and can reconcile missing messages by comparing message IDs.

Conclusion

Packet loss can occur at many points in the communication path, and while TCP’s retransmission covers many cases, it does not guarantee application‑level reliability. Monitoring tools (ping, mtr) and proper configuration of queues, buffers, and server‑side reconciliation are essential for robust messaging.