Master Apache Log Analysis with 30 Essential Shell Commands

Apache Log Analysis Commands

1. Count unique IPs : awk '{print $1}' log_file | sort | uniq | wc -l 2. Count visits to a specific page : grep "/index.php" log_file | wc -l 3. Count pages visited per IP :

awk '{++S[$1]} END {for (a in S) print a,S[a]}' log_file | sort -n -t ' ' -k 2

4. Sort IPs by number of pages visited (ascending) :

awk '{++S[$1]} END {for (a in S) print S[a],a}' log_file | sort -n

5. List pages accessed by a specific IP : grep ^111.111.111.111 log_file | awk '{print $1,$7}' 6. Exclude search‑engine crawlers :

awk '{print $12,$1}' log_file | grep ^"Mozilla" | awk '{print $2}' | sort | uniq | wc -l

7. Count IPs in a specific hour (e.g., 08:00 on 26/Feb/2025) :

awk '{print $4,$1}' log_file | grep 26/Feb/2025:08 | awk '{print $2}' | sort | uniq | wc -l

8. Top 10 IP addresses by request count :

awk '{print $1}' access_log | sort | uniq -c | sort -nr | head -10

9. Top 10 most requested files/pages :

cat log_file | awk '{print $11}' | sort | uniq -c | sort -nr | head -10

10. Requests per sub‑domain (based on Referer) :

cat access.log | awk '{print $11}' | sed -e 's/http:\/\///' -e 's/\/.*//' | sort | uniq -c | sort -rn | head -20

11. Largest transferred files :

cat www.access.log | awk '($7~/\.php/){print $10 " " $1 " " $4 " " $7}' | sort -nr | head -100

12. Pages larger than 200 KB and their frequencies :

cat www.access.log | awk '($10 > 200000 && $7~/\.php/){print $7}' | sort -n | uniq -c | sort -nr | head -100

13. Slowest pages (based on transfer time column) :

cat www.access.log | awk '($7~/\.php/){print $NF " " $1 " " $4 " " $7}' | sort -nr | head -100

14. Pages taking more than 60 s :

cat www.access.log | awk '($NF > 60 && $7~/\.php/){print $7}' | sort -n | uniq -c | sort -nr | head -100

15. Files with transfer time > 30 s :

cat www.access.log | awk '($NF > 30){print $7}' | sort -n | uniq -c | sort -nr | head -20

16. Process count per server process (descending) :

ps -ef | awk -F ' ' '{print $8 " " $9}' | sort | uniq -c | sort -nr | head -20

17. Current Apache concurrent connections : netstat -an | grep ESTABLISHED | wc -l 18. Apache process count (example shows 1388) : ps -ef | grep httpd | wc -l 19. Connections per IP and total state counts :

netstat -n | awk '/^tcp/ {n=split($(NF-1),array,":");if(n<=2)++S[array[1]];else++S[array[4]];++s[$NF];++N} END {for(a in S){printf("%-20s %s
", a, S[a]);++I} printf("%-20s %s
","TOTAL_IP",I); for(a in s) printf("%-20s %s
",a, s[a]); printf("%-20s %s
","TOTAL_LINK",N);}'

20. Top 20 URLs on 26/Feb/2025 :

cat access.log | grep '26/Feb/2025' | awk '{print $11}' | sort | uniq -c | sort -nr | head -20

21. IPs that accessed www.linuxyz.cn :

cat access_log | awk '($11~/www.linuxyz.cn/){print $1}' | sort | uniq -c | sort -nr

22. Top 10 IPs overall (any time) :

cat linewow-access.log | awk '{print $1}' | sort | uniq -c | sort -nr | head -10

23. Log entries within a time range (example 15/Aug/2024‑16/Aug/2024) :

cat log_file | egrep '15/Aug/2024|16/Aug/2024' | awk '{print $1}' | sort | uniq -c | sort -nr | head -10

24. IPs for a specific request path sorted by count :

cat log_file | egrep '15/Aug/2024|16/Aug/2024' | awk '{if($7 == "/index.php?g=Member&m=Public&a=sendValidCode") print $1,$7}' | sort | uniq -c | sort -nr

25. Total traffic in GB :

cat access.log | awk '{sum+=$10} END {print sum/1024/1024/1024}'

26. Count of 404 responses : awk '($9 ~/404/)' access.log | awk '{print $9,$7}' | sort 27. HTTP status code distribution :

cat access.log | awk '{counts[$(9)]+=1} END {for(code in counts) print code, counts[code]}'

28. Requests per second (watch example) :

watch "awk '{if($9~/200|30|404/)COUNT[$4]++}END{for(a in COUNT) print a,COUNT[a]}' log_file | sort -k 2 -nr | head -n10"

29. Bandwidth usage (client requests) :

cat apache.log | awk '{if($7~/GET/) count++} END {print "client_request="count}'

30. TCP connection state summary :

netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'

These commands together form a practical toolbox for administrators to monitor traffic patterns, identify heavy hitters, detect abnormal connection states, and assess server performance directly from Apache log files.