Master Server Log Analysis: 20 Essential Linux Commands to Uncover Traffic, Errors, and Performance

Running a personal website on Alibaba Cloud ECS, you can gain deep insight into traffic, errors, and performance by analyzing server access logs with a collection of practical Linux commands.

Basic IP and page statistics

awk '{print $1}' log_file | sort | uniq | wc -l

Count the number of unique visitor IPs. grep "/index.php" log_file | wc -l Show how many times a specific page was accessed.

awk '{++S[$1]} END {for (a in S) print a, S[a]}' log_file | sort -n

List each IP together with the number of pages it requested.

awk '{++S[$1]} END {for (a in S) print S[a], a}' log_file | sort -n

Sort IPs by the number of pages visited, from fewest to most.

grep ^111.111.111.111 log_file | awk '{print $1,$7}'

Show all pages accessed by a particular IP.

awk '{print $12,$1}' log_file | grep ^"Mozilla" | awk '{print $2}' | sort | uniq | wc -l

Exclude search‑engine crawlers and count remaining unique visitors.

awk '{print $4,$1}' log_file | grep 16/Aug/2015:14 | awk '{print $2}' | sort | uniq | wc -l

Count unique IPs that accessed the site during a specific hour.

awk '{print $1}' log_file | sort | uniq -c | sort -nr | head -10

List the top 10 IP addresses by request count.

Page‑level statistics

cat access.log | awk '{print $11}' | sort | uniq -c | sort -nr | head -10

Show the ten most requested files or pages.

cat log_file | awk '{print $11}' | sort | uniq -c | sort -nr | head -20

Rank URLs by request volume for a given day.

cat www.access.log | awk '($7~/\.php/){print $10 " " $1 " " $4 " " $7}' | sort -nr | head -100

List the 100 PHP pages with the largest transfer size.

cat www.access.log | awk '($NF > 60 && $7~/\.php/){print $7}' | sort -n | uniq -c | sort -nr | head -100

Identify PHP pages that took more than 60 seconds to serve.

Bandwidth and status code analysis

cat access.log | awk '{sum+=$10} END {print sum/1024/1024/1024}'

Calculate total traffic in gigabytes.

awk '($9 ~/404/)' access.log | awk '{print $9,$7}' | sort

Extract all 404 errors and the requested URLs.

cat access.log | awk '{counts[$(9)]+=1}; END {for(code in counts) print code, counts[code]}'

Count occurrences of each HTTP status code.

Connection and process monitoring (netstat)

netstat -an | grep ESTABLISHED | wc -l

Show the number of established connections on all ports. netstat -nat|grep -i "80"|wc -l Count total connections to port 80.

netstat -n | awk '/^tcp/ {++S[$NF]}; END {for(a in S) print a, S[a]}'

Display a summary of TCP connection states.

netstat -anlp | grep 80 | awk '{print $7}' | cut -d/ -f1

Show the process IDs listening on port 80.

uniq -c groups identical lines and prefixes each group with its count.

LAST_ACK, SYN_RECV, ESTABLISHED, FIN_WAIT1, FIN_WAIT2, TIME_WAIT are typical TCP states indicating connection progress or termination.

These commands together provide a solid foundation for routine log inspection, traffic analysis, and performance troubleshooting on Linux‑based web servers.