Master Awk: Unlock Powerful Text Processing on the Command Line

Command‑line Syntax

Awk supports two forms of command‑line syntax similar to sed: awk [-F ERE] [-v assignment] ... program [argument ...] and

awk [-F ERE] -f progfile ... [-v assignment] ... [argument ...]

. The program is a series of pattern‑action pairs; patterns trigger actions when matched.

Records and Fields

Awk treats a text file as a database of records and fields. By default, records are separated by newline (RS) and fields by whitespace (FS). These can be changed with -F, -v FS=, or -v RS=. Fields are accessed via $1, $2, …, $0 (the whole record), and NF holds the number of fields.

Script Structure

An Awk script consists of multiple pattern { action } blocks. If the pattern is omitted, the action runs for every input line. Functions can be defined with function name(parameters) { statements }. Example: echo -e "line1\nline2" | awk '{print}' Functions can be called within actions, and long statements can be split with a backslash ( \).

Patterns

/regular expression/ – extended regular expression.

relational expression – e.g., $1 > 10.

BEGIN – runs before the first record.

END – runs after the last record.

pattern, pattern – a range of records.

Patterns can be combined with && (and) or || (or). Negation is done with !.

Regular Expressions

Awk uses POSIX extended regular expressions; refer to external resources for a full reference.

Expressions and Operators

Awk supports arithmetic, string, and logical operators. See the man page for the complete precedence table.

Statements

print

and printf – output data, with optional field separator OFS. if, while, for – control flow. break, continue – loop control. delete – remove array elements. exit – terminate processing (END block still runs). next – skip to the next record.

Built‑in Variables

Key variables include ARGC, ARGV, FS, RS, NF, NR, FNR, OFS, ORS, CONVFMT, OFMT, ENVIRON, RSTART, and RLENGTH. They control input parsing, output formatting, and environment access.

Functions

Mathematical Functions

Awk provides atan2, cos, sin, exp, log, sqrt, int, rand, and srand. Example to generate a random integer between 0 and n‑1:

function randint(n) { return int(n*rand()); }
BEGIN { srand(); print randint(10); }

String Functions

Common functions include sub, gsub, index, length, match, split, sprintf, substr, tolower, and toupper. Example:

BEGIN { var="kodango"; sub(/kodango/, "hello, &", var); print var }

I/O Functions

getline

reads input in several forms, optionally assigning to a variable. close can close opened files or pipes, and system executes external commands.

Conclusion

The article consolidates information from the Awk man page and the “Sed & Awk” appendix, providing a solid overview of Awk for text manipulation, scripting, and data extraction on Linux systems.