Speed Up Log Inspection: A Live Demo of awk, tail, grep, and sed Combo

tail

For large log files, cat scrolls the screen and can freeze the terminal. tail -f logs/application.log follows new lines in real time, useful for monitoring service restarts.

Scenario A: Service start‑up monitoring

# -f (follow): show appended lines in real time
tail -f logs/application.log

Scenario B: Reproducing a bug during testing

Only the latest output is needed, so view the last 200 lines while following:

# Show last 200 lines and keep refreshing
tail -n 200 -f logs/application.log

less

less

loads files on demand, allowing smooth navigation of multi‑gigabyte logs without exhausting memory, and supports backward scrolling.

Scenario: Tracing a customer complaint order

Search backwards for order ORD12345678: less logs/application.log Press Shift+G to jump to the end.

Enter ?ORD12345678 to search upward.

Press n to find the previous occurrence if needed.

Press Shift+F to switch to a tail -f -like live mode; Ctrl+C returns to normal browsing.

grep

grep

is the most common search tool, but simple keyword matches often miss context.

Scenario A: Reconstructing an error scene

Show 20 lines before and after a NullPointerException:

# Show 20 lines of context
grep -C 20 "NullPointerException" logs/application.log

Scenario B: Full‑trace search by TraceId

Search all rotated logs for a specific TraceId:

# Search all files starting with app.log
grep "TraceId-20251219001" logs/app.log*

Scenario C: Counting exception occurrences

# -c counts matching lines
grep -c "RedisConnectionException" logs/application.log

Scenario D: Filtering out noise

# -v inverts match to exclude health‑check lines
grep -v "HealthCheck" logs/application.log

sed

When a log is huge (e.g., 10 GB) and the incident time window is known, sed can extract that slice.

Scenario: Exporting a time‑window segment

# Extract lines between start and end timestamps
sed -n '/2025-12-19 14:00/,/2025-12-19 14:05/p' logs/application.log > error_segment.log

The resulting error_segment.log is a few megabytes and easy to download or share.

awk

awk

excels at column‑based processing, useful for structured logs such as Nginx access logs.

Scenario A: Finding malicious IPs during a possible CC attack

# Extract IP column, sort, count, and show top 10
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head -n 10

Scenario B: Identifying slow endpoints

Assuming response time is the last column and URL is column 7, list requests taking more than 1 second:

# Print URL and response time > 1.000
awk '$NF > 1.000 {print $7, $NF}' access.log

Conclusion

The commands above are common tools that backend engineers should keep handy; copying the exact snippets for each scenario can dramatically speed up production troubleshooting.