Solve 90% of Linux Log Issues with Three Command‑Line Tools

Why only three tools?

Linux follows the "Do One Thing Well" philosophy, so log analysis can be broken into three steps: log filtering → field analysis → data cleaning. The corresponding tools are grep for filtering, awk for analysis, and sed for cleaning.

Tool 1: grep (log‑filtering king)

Most log‑analysis workflows start with grep.

Find error lines: grep ERROR app.log Output example:

ERROR database connect failed
ERROR timeout

Show context around errors (5 lines before/after): grep -C 5 ERROR app.log Real‑time monitoring: tail -f app.log | grep ERROR This streams error lines as they appear.

Tool 2: awk (log‑analysis engine)

awk

splits each line into fields (space‑delimited). Example log line: 192.168.1.1 - - [10/Mar] "GET /login" 200 Fields: $1=IP, $4=time, $5=request, $9=status.

Count most frequent IPs:

awk '{print $1}' access.log | sort | uniq -c | sort -nr

Sample output:

120 192.168.1.1
 98 10.0.0.2

Count most accessed endpoints:

awk '{print $7}' access.log | sort | uniq -c | sort -nr

Sample output:

500 /api/login
320 /api/order

Count HTTP status codes: awk '{print $9}' access.log | sort | uniq -c Sample output:

1000 200
  23 500
   8 404

Tool 3: sed (log‑cleaning wizard)

sed

is used for simple text transformations.

Delete empty lines: sed '/^$/d' app.log Replace an IP address: sed 's/127.0.0.1/localhost/g' app.log Remove debug logs:

sed '/DEBUG/d' app.log

Real production log‑analysis cases

Case 1 – Count ERROR lines: grep ERROR app.log | wc -l Case 2 – Find most common error messages:

grep ERROR app.log | awk '{print $5}' | sort | uniq -c | sort -nr

Output example:

200 TimeoutException
120 NullPointerException
 50 DBConnectionFailed

Case 3 – Find most active IP:

awk '{print $1}' access.log | sort | uniq -c | sort -nr | head

Case 4 – Count endpoint hits:

awk '{print $7}' access.log | sort | uniq -c | sort -nr

Case 5 – Identify slowest API calls (sample log format "GET /api/user 120ms"):

awk '{print $2, $3}' access.log | sort -k2 -nr | head

CLI pipeline philosophy

Combining tools with pipes yields powerful one‑liners, e.g.:

grep ERROR app.log | awk '{print $5}' | sort | uniq -c | sort -nr

The flow is: grep finds errors → awk extracts the error message field → sort orders them → uniq -c counts occurrences → final sort -nr shows the top errors.

Ultimate routine

Find errors: grep ERROR Extract fields: awk Count: sort | uniq -c Show top:

sort -nr

Golden combination rating

grep   ⭐⭐⭐⭐⭐
awk    ⭐⭐⭐⭐⭐
tail   ⭐⭐⭐⭐⭐
sort   ⭐⭐⭐⭐
uniq   ⭐⭐⭐⭐
sed    ⭐⭐⭐

Final command to remember

If you keep only one command, use:

grep ERROR app.log | awk '{print $5}' | sort | uniq -c | sort -nr

This one‑liner can handle Java logs, Nginx logs, micro‑service logs, and Kubernetes logs, covering roughly 90% of online issues.