Master grep, awk, and sed: Essential Linux Text‑Processing Tricks

Linux Three Musketeers

grep

-n: print line numbers
-A: after, print N lines after a match
-B: before, print N lines before a match
-C: center, print N lines before and after a match
-E: enable extended regular expressions (e.g., grep -E 'root|nginx' /etc/passwd)
-v: invert match
-o: print only the matched portion
-w: match whole words only
-P: enable Perl-compatible regular expressions
-i: ignore case
-r: recursive search (e.g., grep -r 'www.baidu.com' ./)
-R: recursive search including symbolic links
-l: list only matching file names
-h: suppress file name output
-f: compare file contents, showing differences when combined with -v
-c: count matching lines (similar to wc -l)
-G: basic regular expressions
-m: limit output to the first N lines (similar to head -n)

# Example usage
grep -r 'www.zls.com' /root/web/
# Replace all occurrences of www.zls.com with www.baidu.com
grep -lr 'www.zls.com' /root/web/ | xargs sed -i.zls 's#www.zls.com#www.baidu.com#g'

awk

Built‑in variables, options and actions

Key built‑in variables include:

NR : record number (line number)

RS : record separator (default newline)

FS : field separator (default space)

NF : number of fields in the current record

Common options:

-F : specify field separator

-v : assign a variable before processing begins

Typical actions:

print : output fields or strings

gsub : global substitution within a field

variable assignment and arithmetic

Awk processing consists of three stages:

BEGIN block – executed before any input is read.

Pattern‑action statements – applied to each input line.

END block – executed after the last line has been processed.

Examples:

# Print the first field of /etc/passwd
awk -F: '{print $1}' /etc/passwd

# Count lines and sum a column
awk 'NR>1 {sum+=$2} END {print sum}' data.txt

# Show status‑code statistics from an nginx log
zcat access.log.gz | awk '{code[$10]++; bytes[$10]+=$11} END {for(c in code) print c, code[c], bytes[c]}'

sed

Core operations are grouped into four categories: delete, insert, change, and print.

Delete

# Delete line 2 (output to stdout)
sed '2d' file.txt

# Delete the last line
sed '$d' file.txt

# Delete from a line containing "zls" up to a line containing "cls"
sed '/zls/,/cls/d' file.txt

Insert / Append / Change

# Replace the entire second line
sed '2c2,huanglong,438' file.txt

# Append a new line after the last line
sed '$a5,huanglong,438' file.txt

# Insert a new line before the first line
sed '1i2,huanglong,438' file.txt

Substitute (replace)

# Basic substitution
sed 's#zls#ZLS#g' file.txt

# Use a regular expression to replace all digits with 666
sed 's#[0-9]#666#g' file.txt

# Capture groups with back‑references
ifconfig eth0 | sed -nr 's#^.*inet (.*)  net.*#\1#gp'

Advanced concepts include the pattern space (reading multiple lines with N) and the hold space for temporary storage.