Master Essential Linux Shell Tools: find, grep, awk, and More

Why Linux Shell Matters

Linux shell is a fundamental skill for developers and system administrators; mastering its core utilities reveals many details of the underlying operating system and enables powerful one‑liner solutions.

Common Text‑Processing Tools

The article focuses on the following commands: find, grep, xargs, sort, uniq, tr, cut, paste, wc, sed, awk . Each tool is introduced with its most useful options and practical examples.

1. find – File Search

Basic pattern search:

find . \( -name "*.txt" -o -name "*.pdf" \) -print

Regex search (case‑insensitive): find . -regex ".*\(\.txt|\.pdf\)$" Negation, depth control, size, permissions, and user filters are demonstrated, e.g.:

find . ! -name "*.txt" -print

find . -maxdepth 1 -type f

find . -type f -size +2k

find . -type f -perm 644 -print

find . -type f -user weber -print

Post‑search actions such as deletion and exec are shown:

find . -type f -name "*.swp" -delete

find . -type f -user root -exec chown weber {} \;

Note: {} is replaced by each matched filename.

find . -type f -mtime +10 -name "*.txt" -exec cp {} OLD \;

When multiple commands are needed, wrap them in a script and call it with -exec ./script.sh {}.

2. grep – Text Search

Basic usage: grep "pattern" file Common options: -o output only the matching part (vs -v for non‑matches) -c count matches -n show line numbers -i ignore case -l list matching filenames

Recursive search in directories: grep "class" . -R -n Search for multiple patterns: grep -e "class" -e "virtual" file Use -Z to output null‑terminated filenames for safe piping:

grep "test" file* -lZ | xargs -0 rm

3. xargs – Build Command Lines

Convert input lines to arguments, e.g.: cat file.txt | xargs Limit arguments per command with -n: cat single.txt | xargs -n 3 Key options: -d define a delimiter (default space, newline is \n) -n number of arguments per invocation -I {} replace placeholder with each input item -0 treat null byte as delimiter

cat file.txt | xargs -I {} ./command.sh -p {} -1

4. sort – Ordering Lines

Important flags: -n numeric sort (vs -d dictionary order) -r reverse -k N sort by the N‑th column

sort -nrk 1 data.txt

sort -bd data   # ignore leading blanks

5. uniq – Remove Duplicate Lines

Typical pipeline: sort unsort.txt | uniq Count occurrences: sort unsort.txt | uniq -c Show only duplicated lines: sort unsort.txt | uniq -d Limit comparison to specific fields with -s (start) and -w (width).

6. tr – Translate or Delete Characters

Basic translation: echo 12345 | tr '0-9' '9876543210' Delete characters: cat file | tr -d '0-9' Complement set: cat file | tr -c '0-9' Squeeze repeated characters (commonly spaces): cat file | tr -s ' ' Character classes (e.g., [:lower:], [:upper:]) can be used:

tr '[:lower:]' '[:upper:]'

7. cut – Extract Columns

Extract specific fields: cut -f2,4 filename Exclude a field: cut -f3 --complement filename Specify delimiter: cut -f2 -d ";" filename Field ranges: N- from field N to end M first M fields N-M fields N through M

Units: -b bytes -c characters -f fields (default delimiter is tab)

cut -c1-5 file   # first five characters

cut -c-2 file    # first two characters

8. paste – Merge Columns

Combine files side by side (default delimiter is tab): paste file1 file2 Use a custom delimiter:

paste file1 file2 -d ","

9. wc – Count Lines, Words, Bytes

wc -l file   # lines

wc -w file   # words

wc -c file   # bytes

10. sed – Stream Editor

Single substitution (first match per line): sed 's/text/replace_text/' file Global substitution: sed 's/text/replace_text/g' file In‑place editing: sed -i 's/text/replace_text/g' file Delete empty lines: sed '/^$/d' file Use captured groups: sed 's/hello\([0-9]\)/\1/' Variable interpolation with double quotes:

p=pattern; r=replace; echo "line with pattern" | sed "s/$p/$r/g"

11. awk – Powerful Text‑Processing Language

Program structure:

awk 'BEGIN{ statements } statements END{ statements }' file

Execution flow: BEGIN block → per‑line processing → END block.

Common built‑in variables: NR record (line) number NF number of fields $0 entire line $1, $2 … individual fields

Examples:

Print specific columns: awk '{print $2, $3}' file Count lines: awk 'END{print NR}' file Sum first column: awk '{sum+=$1} END{print sum}' file Head‑like output: awk 'NR<=10{print}' file Tail‑like output (last 10 lines):

awk '{buffer[NR%10]=$0} END{for(i=0;i<10;i++) print buffer[(NR+i)%10]}' file

Set field separator with -F (default space): awk -F: '{print $NF}' /etc/passwd Read command output via getline:

echo | awk '{"grep root /etc/passwd" | getline cmdout; print cmdout}'

Loop constructs:

for(i=0;i<10;i++){print $i}

for(i in array){print array[i]}

12. Iterating Over Files

Line‑by‑line with while read: while read line; do echo $line; done < file.txt Same using a subshell:

cat file.txt | (while read line; do echo $line; done)

Word iteration: for word in $line; do echo $word; done Character iteration (bash string slicing):

for ((i=0;i<${#word};i++)); do echo ${word:i:1}; done