How to Diagnose and Reduce Linux Network Latency with hping3, traceroute, and Wireshark
This article explains how to identify the root causes of Linux network latency—including transmission delays, kernel processing speed, and application handling—by using tools such as hping3, traceroute, wrk, tcpdump, and Wireshark, and offers practical steps to mitigate the delays.
In Linux servers, kernel tuning, DPDK, and XDP can improve resistance to attacks and reduce DDoS impact, while application‑level caches, WAF, and CDN help mitigate DDoS effects on services.
However, once DDoS traffic reaches the server, network latency typically spikes dramatically, so professional traffic‑scrubbing and firewall devices are often employed.
Other common reasons for increased latency include slow network transmission, slow Linux kernel packet processing, and slow application data handling.
When faced with these issues, how can we locate the root cause? Let’s discuss network latency.
Linux Network Latency
Network latency (RTT) refers to the round‑trip time of packets between source and destination. Application latency adds the time the application spends processing the request, and is often called round‑trip latency as well.
People usually use ping to test latency; ping relies on ICMP echo request/reply. Because ICMP can be abused, many services disable it, so you can use traceroute or hping3 in TCP/UDP mode to measure latency.
Example with hping3:
# -c: 3 requests
# -S: Set TCP SYN
# -p: Set port to 80
$ hping3 -c 3 -S -p 80 google.com
HPING google.com (eth0 142.250.64.110): S set, 40 headers + 0 data bytes
len=46 ip=142.250.64.110 ttl=51 id=47908 sport=80 flags=SA seq=0 win=8192 rtt=9.3 ms
len=46 ip=142.250.64.110 ttl=51 id=6788 sport=80 flags=SA seq=1 win=8192 rtt=10.9 ms
len=46 ip=142.250.64.110 ttl=51 id=37699 sport=80 flags=SA seq=2 win=8192 rtt=11.9 ms
--- baidu.com hping statistic ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 9.3/10.9/11.9 msExample with traceroute:
$ traceroute --tcp -p 80 -n google.com
traceroute to google.com (142.250.190.110), 30 hops max, 60 byte packets
1 * * *
2 240.1.236.34 0.198 ms * *
3 * * 243.254.11.5 0.189 ms
4 * 240.1.236.17 0.216 ms 240.1.236.24 0.175 ms
5 241.0.12.76 0.181 ms 108.166.244.15 0.234 ms 241.0.12.76 0.219 ms
...
24 142.250.190.110 17.465 ms 108.170.244.1 18.532 ms 142.251.60.207 18.595 msCase Study
We set up two hosts: host1 (192.168.0.30) runs two Nginx containers—one official and one with added latency; host2 (192.168.0.2) is the analyzer.
host1 preparation
# Official nginx
$ docker run --network=host --name=good -itd nginx
# Latency version of nginx
$ docker run --name nginx --network=host -itd feisky/nginx:latencyVerify both containers serve traffic with curl requests.
$ curl http://127.0.0.1
<!DOCTYPE html>...
<p><em>Thank you for using nginx.</em></p>host2 preparation
Use hping3 to test latency of ports 80 and 8080.
$ hping3 -c 3 -S -p 80 192.168.0.30
HPING 192.168.0.30 (eth0 192.168.0.30): S set, 40 headers + 0 data bytes
len=44 ip=192.168.0.30 ttl=64 DF id=0 sport=80 flags=SA seq=0 win=29200 rtt=7.8 ms
len=44 ip=192.168.0.30 ttl=64 DF id=0 sport=80 flags=SA seq=1 win=29200 rtt=7.7 ms
len=44 ip=192.168.0.30 ttl=64 DF id=0 sport=80 flags=SA seq=2 win=29200 rtt=7.6 ms
--- 192.168.0.30 hping statistic ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 7.6/7.7/7.8 ms# Test 8080 port latency
$ hping3 -c 3 -S -p 8080 192.168.0.30
HPING 192.168.0.30 (eth0 192.168.0.30): S set, 40 headers + 0 data bytes
len=44 ip=192.168.0.30 ttl=64 DF id=0 sport=8080 flags=SA seq=0 win=29200 rtt=7.7 ms
len=44 ip=192.168.0.30 ttl=64 DF id=0 sport=8080 flags=SA seq=1 win=29200 rtt=7.6 ms
len=44 ip=192.168.0.30 ttl=64 DF id=0 sport=8080 flags=SA seq=2 win=29200 rtt=7.3 ms
--- 192.168.0.30 hping statistic ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 7.3/7.6/7.7 msRun wrk to generate concurrent load.
$ wrk --latency -c 100 -t 2 --timeout 2 http://192.168.0.30/
Running 10s test @ http://192.168.0.30/
2 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 9.19ms 12.32ms 319.61ms 97.80%
Req/Sec 6.20k 426.80 8.25k 85.50%
Latency Distribution
50% 7.78ms
75% 8.22ms
90% 9.14ms
99% 50.53ms
123558 requests in 10.01s, 100.15MB read
Requests/sec: 12340.91
Transfer/sec: 10.00MB$ wrk --latency -c 100 -t 2 --timeout 2 http://192.168.0.30:8080/
Running 10s test @ http://192.168.0.30:8080/
2 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 43.60ms 6.41ms 56.58ms 97.06%
Req/Sec 1.15k 120.29 1.92k 88.50%
Latency Distribution
50% 44.02ms
75% 44.33ms
90% 47.62ms
99% 48.88ms
22853 requests in 10.01s, 18.55MB read
Requests/sec: 2283.31
Transfer/sec: 1.85MBCapture packets on host1 with tcpdump and analyze them in Wireshark. $ tcpdump -nn tcp port 8080 -w nginx.pcap Wireshark’s flow graph shows a 40 ms delayed ACK after the first HTTP response, indicating the client’s delayed acknowledgment mechanism (TCP delayed ACK) is active.
Inspecting socket options with strace reveals only TCP_NODELAY is set, not TCP_QUICKACK, which explains the extra delay.
$ strace -f wrk --latency -c 100 -t 2 --timeout 2 http://192.168.0.30:8080/
... setsockopt(52, SOL_TCP, TCP_NODELAY, [1], 4) = 0 ...Conclusion
Use hping3 and wrk to verify latency of single and concurrent requests.
Use traceroute to confirm routing and per‑hop latency.
Use tcpdump and Wireshark to ensure packets are transmitted correctly.
Use strace to observe socket calls.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Open Source Linux
Focused on sharing Linux/Unix content, covering fundamentals, system development, network programming, automation/operations, cloud computing, and related professional knowledge.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.