Master Real‑Time Log Analysis with tail, less, grep, sed & awk

tail – Real‑time log monitoring

For large log files, cat scrolls the whole file and can freeze the terminal. tail -f logs/application.log follows the file, showing new lines as they are written, which is ideal for watching service start‑up messages or test‑triggered errors.

# -f (follow): show appended content in real time
tail -f logs/application.log

less – Efficient paging and back‑search

less

loads data on demand, allowing you to open multi‑gigabyte logs without exhausting memory. Use Shift+G to jump to the end, then ?pattern to search backwards, n to repeat, and Shift+F to switch to a tail‑like follow mode.

# Open the log for paging
less logs/application.log

grep – Powerful pattern searching

grep

finds lines matching a keyword. Adding -C 20 shows 20 context lines before and after the match, while -c counts occurrences. Use -v to exclude noisy lines such as health‑check logs.

# Show 20 lines around each NullPointerException
grep -C 20 "NullPointerException" logs/application.log

# Count Redis timeout exceptions
grep -c "RedisConnectionException" logs/application.log

# Exclude health‑check logs
grep -v "HealthCheck" logs/application.log

sed – Extract a time window

When a failure occurs within a known time range, sed -n '/START/,/END/p' extracts only those lines, creating a small file for offline analysis.

# Extract logs between 14:00 and 14:05
sed -n '/2025-12-19 14:00/,/2025-12-19 14:05/p' logs/application.log > error_segment.log

awk – Column‑based processing

awk

excels at handling structured logs. The examples below show how to find the top attacking IPs and the slowest HTTP requests.

# Top 10 IPs by request count (Nginx access log)
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head -n 10

# URLs with response time > 1 s (assuming $NF is time, $7 is URL)
awk '$NF > 1.000 {print $7, $NF}' access.log

Putting it together

Combine these commands to quickly locate the root cause of production incidents without downloading entire log files. Memorise or bookmark the snippets, and adapt the parameters (file paths, patterns, line counts) to your own environment.