Master Shell Regex: Boost Your Script Efficiency with Powerful Patterns

Regular Expressions 1

Regular Expressions (REGEXP) are patterns composed of special and literal characters used to match and process text. Unlike wildcards, regex can match text content, not just filenames, and are supported by many tools like vim, less, grep, sed, awk, nginx, mysql.

Main uses of regular expressions : match specific patterns in strings such as command output or file contents.

Main uses of wildcards : match filenames or directory names, e.g., files ending with .sh.

Common wildcard usage

*

: matches any sequence of one or more characters, e.g., ls *.txt matches all files ending with .txt. ?: matches any single character, e.g., ls ?.txt matches filenames with a single character before .txt. []: matches any single character from the set, e.g., ls [a-z].txt matches files starting with a lowercase letter.

Wildcard / Escape / Metacharacter / Hyphen

Wildcard

Wildcard characters are used to match any character or set in filenames. Common wildcards include: *: matches any length of characters, including empty. ?: matches any single character. []: matches any character or range inside brackets.

Escape character

The escape character (backslash \) is used to treat metacharacters such as *, ?, $ as literal characters.

Example : \* matches the asterisk character itself, not a wildcard.

Metacharacters

Metacharacters define special matching rules. Common ones are ., *, +, ?, |, [], ().

Example : . matches any single character; [a-z] matches any lowercase letter.

Hyphen

The hyphen - is usually used to denote a range inside brackets, e.g., [a-z] matches all lowercase letters, [0-9] matches digits.

Types of regular expressions

Basic Regular Expressions (BRE)

Extended Regular Expressions (ERE)

Regular expression examples

Match any character

Match a line where a character is followed by any character ls /opt | grep "t." Matches file or directory names where t is followed by any character.

Match a line where a character is followed by any two characters grep 'r..t' /etc/passwd Matches lines containing r, any two characters, then t.

Use grep with . to match any character echo abc | grep 'a.c' Matches strings where a is followed by any single character and then c.

Match character set

Use grep to match any character in a set: ls | grep '[fhtx].txt' Matches files starting with f, h, t, or x and ending with .txt.

Use a range: ls [a-d].txt Matches files whose names start with a‑d (or A‑D) and end with .txt.

Use grep with a lowercase range: ls | grep '[a-d].txt' Same as above for lowercase letters.

Use grep with a negated set: ls | grep '[^a-z].txt' Matches files not starting with a lowercase letter.

Use grep with a negated set containing a and z: ls | grep '[^a.z].txt' Matches files not starting with a or z.

Match special characters

Match files beginning with rc. (dot escaped): ls /etc/ | grep 'rc\.' Match a literal dot in a string:

echo abc | grep 'a\.c'

Match character class

Match whitespace characters: grep [[:space:]] 123.txt Match letters (both cases):

ls | grep '[a-zA-Z]'

Match mixed character sets

Match rc followed by a dot or a digit 0‑6: ls /etc/ | grep 'rc[.0-6]' Match rc followed by a dot or digit and then any character:

ls /etc/ | grep 'rc[.0-6].'

Quantifiers (repetition)

Quantifiers specify how many times a character or sub‑expression may appear. *: zero or more (greedy). .*: any length of any characters, including empty. \?: zero or one. \+: one or more. \{n\}: exactly n times. \{m,n\}: between m and n times. \{,n\}: up to n times. \{n,\}: at least n times.

Examples of quantifier usage

Match exactly two o characters: echo google | grep 'go\{2\}gle' Match at least two o characters: echo gooooogle | grep 'go\{2,\}gle' Match between two and five o characters: echo gooooogle | grep 'go\{2,5\}gle' Match up to five o characters (no output if more):

echo goooooogle | grep 'go\{,5\}gle'

Wildcard matching examples

Match zero or more o characters: echo goooooogle | grep 'go*gle' Match string without o: echo ggle | grep "go*gle" Match one or more g characters:

echo gggle | grep "go*gle"

Any‑character matching example

Match any characters between g and gle:

echo gdadadadgle | grep "g.*gle"

Occurrence matching example

Match zero or one o: echo ggle | grep "go\?gle" Match one or more o:

echo google | grep "go\+gle"

IP address matching example

Extract IPv4 addresses from ifconfig output:

ifconfig ens33 | grep netmask | grep -o '[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+'

Match IPv4 address components with 1‑3 digits:

ifconfig ens33 | grep netmask | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}'