Mastering awk: The Ultimate Guide to Text Processing in Linux

1. Introduction to awk

awk is a powerful text‑processing language used for analyzing and transforming data files. Its name comes from the initials of its creators—Alfred Aho, Peter Weinberger, and Brian Kernighan. In the Unix toolbox, awk joins grep and sed as the three "swordsmen" for text handling.

1.1 Basic Role

awk reads input line by line, splits each line into fields, and applies user‑defined actions.

2. awk Usage

2.1 Syntax

awk [options] 'pattern {action}' file

2.2 Common Options

-F

: specify the input field separator (default is whitespace).

2.3 Example

awk -F':' '{print $1}' /etc/passwd

3. Predefined Variables

$0

: the entire current line. $n: the nth field. NF: number of fields in the current line. NR: record number (line number). FS: input field separator. OFS: output field separator.

3.1 Example

awk -F':' '{print $NF, $1}' /etc/passwd

4. Execution Flow

awk processes scripts in the order: BEGIN block → pattern/action pairs → END block. At least one of these blocks must be present.

5. Functions

print

: prints fields with OFS between them. printf: formatted output using C‑style specifiers ( %s, %d, width, alignment, etc.).

5.1 Example

awk -F':' 'BEGIN{OFS=" >>> "}{printf "%+15s|%-15s
", $NF, $1}' /etc/passwd

6. Pattern Matching and Positioning

Regular expressions can be used to select lines, e.g., /root/ {print $0} prints lines containing "root".

7. Comparison Expressions

>

, <, >=,

<=

~

: matches regex; !~: does not match.

7.1 Example

awk -F':' '$4 > $3 {print $0}' /etc/passwd

8. Conditional Expressions

Use ==, !=, etc., to test values. Example: NR == 3 {print $0} prints the third line.

9. Logical Expressions

&&

: logical AND ||: logical OR !: logical NOT

9.1 Example

awk -F':' '$3+$4 > 2000 && $3*$4 > 2000 {print $0}' /etc/passwd

10. Arithmetic Expressions

Supported operators: +, -, *, /, %. Example: NR % 2 == 0 {print $0} prints even‑numbered lines.

11. Flow Control

if (condition) { … } else { … }

for (i=10; i>0; i--) { print $0 }

while (condition) { … }

11.1 Example

awk -F':' '{if($3 > $4) print "greater"; else print "less or equal"}' /etc/passwd

12. Practical Case

Print a separator line every five lines:

awk -F':' '{if(NR%5==0) print "----------"; print $0}' /etc/passwd

For further details, refer to the original source.