Why Is My Linux Server Dropping Packets? A Step‑by‑Step Diagnosis

1. Background

Packet loss can occur anywhere in the network protocol stack, from the virtual‑machine link between two hosts down to the application layer, meaning every layer must be examined.

Transmission failures between VMs (congestion, line errors)

Ring‑buffer overflow on the NIC

Link‑layer frame check failures or QoS drops

IP‑layer routing failures or MTU oversize

Transport‑layer port not listening or kernel limits

Socket‑buffer overflow

Application‑layer process crashes

iptables filtering

2. Link Layer

The NIC records error counters that can be inspected with ethtool or netstat -i. The columns RX‑OK , RX‑ERR , RX‑DRP , RX‑OVR show received packets, errors, drops and overruns respectively; the TX columns report the same for transmitted packets.

netstat -i
Iface   MTU   RX-OK  RX-ERR  RX-DRP  RX-OVR  TX-OK  TX-ERR  TX-DRP  TX-OVR  Flg
eth0    100    31      0       0       0      0       0       8      0   BMRU
lo     65536    0      0       0       0      0       0       0      0

On the examined host a netem qdisc was configured with a 30 % random loss rule, which explains the observed drops:

tc qdisc show dev eth0
qdisc netem 800d: root refcnt 2 limit 1000 loss 30%

Removing the rule restores normal NIC behavior:

tc qdisc del dev eth0 root netem loss 30%

3. Network and Transport Layers

Linux provides per‑protocol statistics via netstat -s. The TCP section revealed multiple failures:

11 failed connection attempts

4 retransmitted segments

11 SYN‑RECV resets

4 SYN retransmissions

7 time‑outs

These numbers indicate frequent three‑way‑handshake failures, but they do not pinpoint the root cause without deeper inspection.

4. iptables

Connection tracking was checked first; the count (182) was far below the maximum (262 144), so it was not the culprit. The filter table was then listed, revealing two DROP rules that use the statistic module with a 30 % random probability, matching all source and destination addresses:

iptables -t filter -nvL
Chain INPUT (policy ACCEPT 25 packets)
 pkts bytes target prot opt in out source destination
   6 240 DROP all -- 0.0.0.0/0 0.0.0.0/0 statistic mode random probability 0.29999999981

Chain OUTPUT (policy ACCEPT 15 packets)
 pkts bytes target prot opt in out source destination
   6 264 DROP all -- 0.0.0.0/0 0.0.0.0/0 statistic mode random probability 0.30

Both rules were deleted:

iptables -t filter -D INPUT -m statistic --mode random --probability 0.30 -j DROP
iptables -t filter -D OUTPUT -m statistic --mode random --probability 0.30 -j DROP

5. tcpdump

Capturing traffic on port 80 with tcpdump -i eth0 -nn port 80 showed that the interface was listening but verbose output was suppressed. A comparison between hping3 (which only sends SYN packets) and curl (which sends a full HTTP GET after the SYN) revealed that the loss only manifested when larger packets were transmitted.

6. MTU Misconfiguration

The netstat -i output displayed an MTU of 100 for eth0, far below the Ethernet default of 1500. This caused fragmentation problems for packets larger than 100 bytes, such as the HTTP GET generated by curl. The MTU was corrected: ifconfig eth0 mtu 1500 After the change, a curl request succeeded and Nginx returned the expected HTML page, confirming that packet loss was fully resolved.