Essential Shell Commands for Analyzing Apache Access Logs

This guide provides a toolbox of command‑line snippets for extracting valuable metrics from Apache access logs.

Basic IP and page statistics

awk '{print $1}' log_file | sort | uniq | wc -l

– count distinct IP addresses. grep "/index.php" log_file | wc -l – count accesses to a specific page.

awk '{++S[$1]} END {for (a in S) print a, S[a]}' log_file | sort -k2 -n

– list each IP with the number of pages it requested.

awk '{++S[$1]} END {for (a in S) print S[a], a}' log_file | sort -n

– sort IPs by request count.

Advanced filtering and sorting

Show pages requested by a particular IP: grep ^111.111.111.111 log_file | awk '{print $1, $7}' Exclude search‑engine crawlers:

awk '{print $12,$1}' log_file | grep '^"Mozilla' | awk '{print $2}' | sort | uniq | wc -l

Count requests within a specific hour (e.g., 14:00 on 16 Aug 2015):

awk '{print $4,$1}' log_file | grep 16/Aug/2015:14 | awk '{print $2}' | sort | uniq | wc -l

Top‑N analyses

Top 10 IPs by request volume:

awk '{print $1}' access_log | sort | uniq -c | sort -nr | head -10

Top 10 most requested URLs:

awk '{print $11}' log_file | sort | uniq -c | sort -nr | head -10

Top 20 IPs by total traffic (bytes transferred):

awk '{print $11}' log_file | sort | uniq -c | sort -nr | head -20

Performance‑related queries

Pages with transfer size >200 KB:

awk '($10>200000 && $7~/\.php/){print $7}' log_file | sort | uniq -c | sort -nr | head -100

Requests taking more than 60 seconds:

awk '($NF>60 && $7~/\.php/){print $7}' log_file | sort | uniq -c | sort -nr | head -100

HTTP status code distribution:

awk '{counts[$9]++} END {for (code in counts) print code, counts[code]}' access.log

Network connection monitoring

Current ESTABLISHED connections on port 80: netstat -an | grep ESTABLISHED | wc -l Count connections per TCP state:

netstat -n | awk '/^tcp/ {++S[$NF]} END {for (a in S) print a, S[a]}'

Top IPs with most TIME_WAIT sockets:

netstat -n | grep TIME_WAIT | awk '{print $5}' | sort | uniq -c | sort -rn | head -20

These snippets can be combined, piped, or adapted to specific log formats, enabling quick diagnostics of traffic patterns, bottlenecks, and potential abuse without requiring additional tools.