How to Diagnose and Fix Network Packet Loss on a Linux Nginx Server

Understanding Packet Loss

Packet loss occurs when network packets are discarded before reaching the application, and the loss rate (dropped packets ÷ total packets) is a key performance metric. High loss degrades TCP performance, causing retransmissions, increased latency, and reduced throughput.

Where Packet Loss Can Occur

Using Nginx as an example, the article demonstrates how to verify connectivity with hping3 -c 10 -S -p 80 192.168.0.30. The output shows 50% loss and large RTT variations, indicating a loss problem.

The possible loss points span the entire protocol stack:

Transmission errors between VMs (congestion, line errors)

Ring‑buffer overflow on the NIC

Link‑layer frame checksum failures or QoS drops

IP‑layer routing failures or MTU issues

Transport‑layer port not listening or kernel limits

Socket‑layer buffer overflow

Application‑layer crashes

iptables filtering rules

Assuming the second VM (VM2) is healthy, the investigation proceeds layer by layer.

Link‑Layer Inspection

Check NIC statistics with netstat -i and ethtool. The example shows no NIC errors. However, a tc qdisc show dev eth0 reveals a netem rule with loss 30%, which explains the 50% loss observed.

Remove the netem rule: tc qdisc del dev eth0 root netem loss 30% Re‑run hping3; loss drops to 0% but the problem persists at higher layers.

Network and Transport Layer Stats

Run netstat -s to view protocol counters. TCP shows multiple failed connections, retransmissions, and SYN retries, indicating handshake failures but not pinpointing the root cause.

iptables Check

Inspect filter tables: iptables -t filter -nvL The output contains two DROP rules using the statistic module with a 30% random drop probability, affecting both INPUT and OUTPUT chains. Deleting them:

iptables -t filter -D INPUT -m statistic --mode random --probability 0.30 -j DROP
iptables -t filter -D OUTPUT -m statistic --mode random --probability 0.30 -j DROP

After removal, hping3 again shows 0% loss.

Using tcpdump and MTU Diagnosis

Capture traffic on port 80: tcpdump -i eth0 -nn port 80 While the three‑way handshake succeeds, the HTTP GET never appears. Checking netstat -i reveals a high receive‑drop count (RX‑DRP 344) and an unusually low MTU of 100 on eth0. The small MTU truncates larger TCP packets (e.g., HTTP GET), causing drops.

Fix the MTU: ifconfig eth0 mtu 1500 Retest with curl --max-time 3 http://192.168.0.30/; the server now returns the expected HTML page, confirming that packet loss is resolved.

Final Resolution

The loss was caused by a stray netem rule and an incorrect MTU setting. Removing the netem rule eliminated artificial drops, and correcting the MTU allowed full‑size packets to be transmitted, restoring normal Nginx HTTP responses.