Master Essential Linux Shell Tools: find, grep, awk, and More

Introduction

The article introduces the most frequently used Linux shell tools for text processing, offering practical examples and parameters.

find – File Search

Search for txt and pdf files

find . ( -name "*.txt" -o -name "*.pdf" ) -print

Regex search for .txt and .pdf

find . -regex ".*(.txt|.pdf)$"

Use -iregex for case‑insensitive regex.

Negate pattern – find all non‑txt files

find . ! -name "*.txt" -print

Limit search depth (depth 1)

find . -maxdepth 1 -type f

Custom search by type, time, size, permissions, user, etc.

find . -type d -print

find . -atime 7 -type f -print

find . -type f -size +2k

find . -type f -perm 644 -print

find . -type f -user weber -print

Post‑search actions

Delete all .swp files

find . -type f -name "*.swp" -delete

Execute a command on each match

find . -type f -user root -exec chown weber {} \;

Note: {} is replaced by the current file name.

grep – Text Search

Common options: -o output only matching parts, -v invert match -c count matches -n show line numbers -i ignore case -l list matching file names

grep -c "text" filename

Recursive search in multiple directories:

grep "class" . -R -n

Match multiple patterns:

grep -e "class" -e "virtual" file

Use -Z to output NUL‑terminated file names:

grep "test" file* -lZ | xargs -0 rm

xargs – Convert Input to Command‑Line Arguments

Transform multi‑line output to a single line:

cat file.txt | xargs

Convert a single line to multiple lines (e.g., three arguments per command):

cat single.txt | xargs -n 3

Define a delimiter:

xargs -d '\n' command

Example – count lines in C++ files:

find source_dir/ -type f -name "*.cpp" -print0 | xargs -0 wc -l

sort – Sorting

Key options: -n numeric sort, -d dictionary order -r reverse -k N sort by the N‑th column

sort -nrk 1 data.txt

sort -bd data

uniq – Remove Duplicate Lines

Delete duplicate lines

sort unsort.txt | uniq

Count occurrences of each line

sort unsort.txt | uniq -c

Show only duplicate lines

sort unsort.txt | uniq -d

Options -s (start position) and -w (compare width) allow fine‑grained control.

tr – Translate or Delete Characters

General usage

echo 12345 | tr '0-9' '9876543210'

cat text | tr '\t' ' '

Delete characters

cat file | tr -d '0-9'

Complement set

cat file | tr -c '0-9'

Compress repeated characters

cat file | tr -s ' '

Character classes such as [:lower:], [:upper:], [:digit:] can be used.

cut – Column Extraction

Extract fields 2 and 4

cut -f2,4 filename

Remove column 3

cut -f3 --complement filename

Specify delimiter

cut -f2 -d ";" filename

Range specifications (e.g., N‑M)

cut -c1-5 file

paste – Merge Files Column‑wise

Default delimiter is a tab; use -d to change.

paste file1 file2

paste file1 file2 -d ","

wc – Count Lines, Words, Characters

Line count: wc -l file Word count: wc -w file Byte/character count:

wc -c file

sed – Stream Editor

Replace first occurrence

sed 's/text/replace_text/' file

Global replacement

sed 's/text/replace_text/g' file

In‑place editing

sed -i 's/text/replace_text/g' file

Delete empty lines

sed '/^$/d' file

Variables can be used with double quotes.

awk – Powerful Text‑Processing Language

Typical script structure:

awk 'BEGIN{...} { ... } END{...}'

Print current line: print Print specific fields: awk '{print $2, $3}' file Count lines: awk 'END{print NR}' file Sum first column:

awk 'BEGIN{sum=0} {sum+=$1} END{print sum}' file

Common built‑in functions: index(), sub(), match(), length(), printf().

Iterating Over Files

Line‑by‑line with while read line; do ...; done < file.txt Word iteration: for word in $line; do ...; done Character iteration using Bash substring syntax.

This article is a study note from the book "Linux Shell Script Guide".