Mastering AWK: Quick Commands to Clean, Filter, and Analyze Text Files

Awk is a powerful text‑processing utility that scans files line by line, splits each line into fields, matches patterns, and executes actions.

Preparing a sample file

The following heredoc creates content.txt with several space‑separated records:

cat << EOF >> content.txt
hitesh engineer sales 30000
jayesh director account 25000
vyom manager purchase 20000

bhavesh engineer sales 30000
rajesh directory sales 40000
niraj clerk account 20000
jay peon purchase 23000
deep clerk sales 20000
EOF

Delete all blank lines

Use the built‑in variable NF to filter out empty records:

awk NF content.txt

Trim leading and trailing spaces or tabs

Remove leading whitespace with sub: awk '{ sub(/^[ \t]+/, ""); print }' content.txt Remove trailing whitespace with sub again: awk '{ sub(/[ \t]+$/, ""); print }' content.txt Handle both ends in a single command using gsub:

awk '{ gsub(/^[ \t]+|[ \t]+$/, ""); print }' content.txt

Reverse the order of all lines

Store each line in an array and print it backwards in the END block:

awk '{ b[i++] = $0 } END { for (j=i-1; j>=0; j--) print b[j] }' content.txt

Add line numbers to non‑empty lines

Use the NF variable to skip blank lines and prepend a counter:

awk 'NF { $0 = ++a ": " $0 }; { print }' content.txt

Count lines containing a specific string

Count occurrences of the word engineer:

awk '/engineer/{n++} END {print n+0}' content.txt

Print lines that match or do not match a pattern

Matching lines: awk '/engineer/' content.txt Non‑matching lines (e.g., exclude jayesh):

awk '!/jayesh/' content.txt

Replace strings

Replace the word engineer with doctor:

awk '{ gsub(/engineer/, "doctor"); print }' content.txt

Replace any of jayesh, hitesh, or bhavesh with mahesh:

awk '{ gsub(/jayesh|hitesh|bhavesh/, "mahesh"); print }' content.txt

Combine AWK with other commands

Show device name, available space, and usage percentage from df -h:

df -h | awk '{ printf("%-24s\t%-6s\t%-4s
", $1, $4, $5) }'

List the number of open connections per IP (useful for spotting attacks):

netstat -ntu | awk '{print $5}' | cut -d: -f1 | awk '/[0-9]/' | sort | uniq -c | sort -n

Conclusion

Through concrete examples we have shown how a single AWK command can perform a wide range of everyday text‑processing and system‑administration tasks efficiently.