Master Log Analysis: Fast Techniques to Pinpoint Errors in Massive Logs

Dynamic Log Viewing

Use tail -f catalina.out to follow a log file in real time, or cat catalina.out to display its entire content.

Extracting Subsets with tail and head

Common patterns: tail -n number catalina.out – show the last number lines. tail -n +number catalina.out – display from line number to the end. head -n number catalina.out – show the first number lines. head -n -number catalina.out – show all lines except the last number .

Finding Line Numbers with grep

First locate the line number of a keyword, then view surrounding lines:

# cat -n catalina.out | grep "keyword"
# cat -n catalina.out | tail -n +13230539 | head -n 10

Here tail -n +13230539 starts from line 13230539, and head -n 10 limits the output to ten lines.

Time‑Range Queries with sed

Search logs within a specific time window:

# grep '11:07 18:29:20' catalina.out
# grep '11:07 18:31:11' catalina.out
# sed -n '/11:07 18:29:20/,/11:07 18:31:11/p' catalina.out

Counting Matches

Count how many times a string appears:

# grep '1175109632' catalina.out | wc -l
# Output: 154

Combining tail and grep

Search the last n lines for a keyword: # tail -n 20 catalina.out | grep 'INFO Takes:1' Use --color to highlight matches, and -a2 to show two context lines before and after each match.

Pagination with more / less

When the result set is large, pipe through a pager:

# tail -n 2000 catalina.out | grep 'INFO Takes:1' --color -a2 | more
# tail -n 2000 catalina.out | grep 'INFO Takes:1' --color -a2 | less

Navigation Shortcuts (for less )

Full‑screen navigation: Ctrl+F (forward a screen), Ctrl+B (back a screen), Ctrl+D (forward half‑screen), Ctrl+U (back half‑screen).

Line navigation: j (down one line), k (up one line).

Other commands: G (go to end), g (go to start), q or ZZ (quit).