Mastering Awk: How to Filter Text with Regular Expressions on Linux

When using Unix/Linux commands to read or edit text, you often need to filter the output to obtain the parts you are interested in; regular expressions are perfect for this purpose.

What is a regular expression?

A regular expression is a string that represents a set of character sequences, allowing you to filter command or file output, edit parts of text, or process configuration files.

Features of regular expressions

Regular expressions consist of ordinary characters and meta‑characters.

Ordinary characters , such as spaces, underscores, A‑Z, a‑z, 0‑9.

Meta‑characters include: (.) – matches any single character except a newline. (*) – matches zero or more occurrences of the preceding character. [character(s)] – matches any character listed inside the brackets; ranges can be specified with a hyphen, e.g., [a-f] or [1-5]. ^ – matches the beginning of a line. $ – matches the end of a line. \ – escape character.

Using awk as a text‑filtering tool

Awk can be used as a programming language for text filtering. Its general syntax is: # awk 'script' filename In the script, the format is /pattern/ action, where pattern is a regular expression and action is the command executed when the pattern matches a line.

Basic awk examples

Print all lines of /etc/hosts (no pattern specified):

# awk '//{print}' /etc/hosts

Print lines containing localhost:

# awk '/localhost/{print}' /etc/hosts

Use the wildcard (.) to match any single character (e.g., l.c matches loc, localhost, localnet):

# awk '/l.c/{print}' /etc/hosts

Use the asterisk (*) to match zero or more occurrences (e.g., l*c matches lc, lcc, llc):

# awk '/l*c/{print}' /etc/hosts

Use a character set [al1] to match any of the listed characters:

# awk '/[al1]/{print}' /etc/hosts

Match lines starting with K or k followed by T:

# awk '/[Kk]T/{print}' /etc/hosts

Use ranges to specify character classes, e.g., [0-9] for digits, [a-z] for lowercase letters, [A-Z] for uppercase letters, [a-zA-Z] for any letter, [a-zA-Z0-9] for alphanumeric.

# awk '/[0-9]/{print}' /etc/hosts

Use the caret ^ to match the start of a line:

# awk '/^fe/{print}' /etc/hosts

Use the dollar sign $ to match the end of a line:

# awk '/ab$/{print}' /etc/hosts

Escape special characters with \. For example, to match a literal dollar sign:

# awk '/$25.00/{print}' deals.txt

# awk '/\$25.00/{print}' deals.txt

Summary

The examples above cover the basic operations of awk as a filtering tool. More advanced features will be introduced in later sections.