Why Is Your Network Dropping Packets? A Complete Troubleshooting Guide

Introduction

Packet loss is a frequent network problem; if a ping fails or returns incomplete data, it often indicates that packets are being dropped somewhere in the transmission path. This guide presents common packet‑loss scenarios and systematic methods to locate and fix them.

What Is Packet Loss?

Data travels across the Internet in packets (bytes). Due to network devices, link quality, or other factors, the amount of data received can be less than the amount sent, resulting in packet loss.

Packet Transmission Principles

Sending a packet:

1. Application data is encapsulated with a TCP header. 2. An IP header is added, forming an IP packet. 3. The NIC driver adds a 14‑byte MAC header, creating a frame (including source and destination MAC addresses). 4. The frame is copied to the NIC buffer, which adds synchronization info and CRC, then the packet is placed on the wire.

Receiving a packet:

1. The NIC checks CRC, removes the frame header, and verifies the destination MAC address. 2. The frame is copied to a ring buffer. 3. The driver notifies the kernel, which processes the packet through the TCP/IP stack. 4. The application reads the data from the socket buffer.

Core Idea

Understanding the send/receive process reveals that packet loss can stem from three main layers: the NIC hardware, the NIC driver, and the kernel protocol stack. The troubleshooting approach follows a bottom‑up layer analysis, checking relevant information at each layer to pinpoint the cause.

Common Packet‑Loss Scenarios

Hardware NIC loss

NIC driver loss

Ethernet link‑layer loss

IP‑layer loss

Transport‑layer (UDP/TCP) loss

Application‑layer socket loss

Hardware NIC Loss

Ring Buffer Overflow

When the NIC writes incoming frames to its internal Ring Buffer faster than the kernel can consume them, the buffer fills and new packets are dropped.

Check:

$ ethtool -S eth0 | grep rx_fifo<br/>$ cat /proc/net/dev

Solution: increase NIC buffer sizes. $ ethtool -G eth0 rx 4096 tx 4096 Port Negotiation Loss

Verify link speed and mode: $ ethtool -S eth0<br/>$ ethtool eth0 Solution: renegotiate or force the speed.

$ ethtool -r eth0<br/>$ ethtool -s eth0 speed 1000 duplex full autoneg off

Flow Control Loss

Check flow‑control counters: $ ethtool -S eth0 | grep control Solution: disable flow control.

$ ethtool -A eth0 autoneg off<br/>$ ethtool -A eth0 tx off<br/>$ ethtool -A eth0 rx off

MAC Address Mismatch

If the destination MAC does not match the NIC’s MAC (e.g., stale ARP entries), packets are dropped.

Solution: refresh ARP tables or set static ARP entries.

Driver Loss

Key counters (view with ifconfig or ethtool) include:

RX errors (including CRC, FIFO overruns, etc.)

RX dropped (buffer shortage)

RX overruns (kernel cannot keep up)

Solution: adjust net.core.netdev_max_backlog or increase driver queue sizes.

$ sysctl -w net.core.netdev_max_backlog=2000

Kernel Protocol‑Stack Loss

ARP Issues

Incorrect arp_ignore or arp_filter settings can cause drops.

$ sysctl -a | grep arp_ignore<br/>$ sysctl -a | grep arp_filter

Solution: set appropriate values (e.g., arp_ignore=1, arp_filter=0).

Routing Drops

Missing or incorrect routes lead to packet drops.

$ ip r get <destination_ip><br/>$ netstat -s | grep "dropped because of missing route"

Solution: correct routing tables.

Reverse‑Path Filtering $ cat /proc/sys/net/ipv4/conf/eth0/rp_filter Set to 0 or 2 if the network environment causes legitimate packets to be rejected. $ sysctl -w net.ipv4.conf.all.rp_filter=2 Firewall Drops $ iptables -nvL | grep DROP Adjust firewall rules accordingly.

Connection‑Tracking Overflows

$ cat /proc/sys/net/netfilter/nf_conntrack_max<br/>$ sysctl -w net.netfilter.nf_conntrack_max=3276800

Increase limits or reduce timeout values.

Transport‑Layer Loss

TCP SYN Flood

$ sysctl -w net.ipv4.tcp_syncookies=1<br/>$ sysctl -w net.ipv4.tcp_max_syn_backlog=4096

TIME_WAIT Exhaustion

$ sysctl -w net.ipv4.tcp_tw_reuse=1<br/>$ sysctl -w net.ipv4.tcp_tw_recycle=0

PAWS / Timestamp Issues

$ sysctl -w net.ipv4.tcp_timestamps=1<br/>$ sysctl -w net.ipv4.tcp_tw_recycle=0

TCP Queue Overflows

$ sysctl -w net.core.somaxconn=1024<br/>$ sysctl -w net.ipv4.tcp_max_syn_backlog=4096

UDP Loss

UDP has no flow control; packet loss often reflects system overload.

$ sysctl -w net.ipv4.udp_mem=65536 131072 262144<br/>$ sysctl -w net.ipv4.udp_rmem_min=65536<br/>$ sysctl -w net.ipv4.udp_wmem_min=65536

Application design should limit send rates and use asynchronous processing.

Application‑Layer Socket Loss

Increase socket buffers:

$ sysctl -w net.core.rmem_max=67108864<br/>$ sysctl -w net.core.wmem_max=67108864

Adjust per‑socket options with setsockopt if needed.

Summary of Kernel‑Stack Packet Loss

Tools for Diagnosis

dropwatch – monitors kernel drop events and prints the call stack where the drop occurred.

tcpdump / tshark – captures packets for detailed analysis; use Wireshark for visual inspection.

Conclusion

Packet loss can originate from any component in the network stack, especially in complex cloud environments. By understanding the fundamentals of packet transmission and following a layered diagnostic approach, you can quickly locate the problematic node and apply the appropriate fix to restore service reliability.