Master Essential Linux Shell Tools: find, grep, awk, sed & More

Linux shell is a fundamental skill; mastering its core utilities greatly enhances text processing and system navigation.

1. find – File Search

Common examples:

find . \( -name "*.txt" -o -name "*.pdf" \) -print

find . -regex ".*\(\.txt|\.pdf\)$"

find . ! -name "*.txt" -print

find . -maxdepth 1 -type f

Search by type, time, size, permissions, user, etc. Example – files accessed in the last 7 days: find . -atime 7 -type f -print Delete all swap files: find . -type f -name "*.swp" -delete Execute actions with -exec:

find . -type f -user root -exec chown weber {} \;

{} is replaced by each matched filename.

2. grep – Text Search

Basic usage and useful options:

-o: output only matching parts

-v: invert match

-c: count matches

-n: show line numbers

-i: ignore case

-l: list matching file names

grep -c "text" filename

grep "class" . -R -n

grep -e "class" -e "virtual" file

grep "test" file* -lZ | xargs -0 rm

3. xargs – Build Command Lines

Convert input into command arguments, often combined with grep or find:

cat file.txt | xargs

cat single.txt | xargs -n 3

Key options:

-d: define delimiter (default space, \n for lines)

-n: number of arguments per command line

-I {}: replace placeholder with input

-0: use \0 as delimiter

cat file.txt | xargs -I {} ./command.sh -p {} -1

find source_dir/ -type f -name "*.cpp" -print0 | xargs -0 wc -l

4. sort – Sorting

Options:

-n: numeric sort

-d: dictionary order

-r: reverse

-k N: sort by column N

sort -nrk 1 data.txt

sort -bd data   # ignore leading blanks

5. uniq – Remove Duplicate Lines

sort unsort.txt | uniq

sort unsort.txt | uniq -c   # count occurrences

sort unsort.txt | uniq -d   # show duplicates only

6. tr – Translate Characters

Common uses:

echo 12345 | tr '0-9' '9876543210'

cat text | tr '\t' ' '

cat file | tr -d '0-9'   # delete digits

cat file | tr -c '0-9'   # keep only digits

cat file | tr -s ' '   # squeeze spaces

7. cut – Column Extraction

cut -f2,4 filename

cut -f3 --complement filename

cut -f2 -d ";" filename

8. paste – Merge Columns

Combine files side‑by‑side, default delimiter is a tab; use -d to change:

paste file1 file2 -d ","

9. wc – Count Lines, Words, Bytes

wc -l file   # lines

wc -w file   # words

wc -c file   # bytes

10. sed – Stream Editing

Replace first occurrence: sed 's/text/replace_text/' file Global replacement: sed 's/text/replace_text/g' file In‑place edit: sed -i 's/text/replace_text/g' file Delete empty lines:

sed '/^$/d' file

11. awk – Powerful Text Processor

Basic script structure: awk 'BEGIN{...} { ... } END{...}' file Print current line: awk '{print}' file Print specific fields: awk '{print $2, $3}' file Count lines: awk 'END{print NR}' file Sum a column: awk '{sum+=$1} END{print sum}' file Set field separator: awk -F: '{print $NF}' /etc/passwd Read command output:

awk '{"grep root /etc/passwd" | getline cmd; print cmd}'

12. Iterating Over Files

Line‑by‑line loop: while read line; do echo $line; done < file.txt Word iteration: for word in $line; do echo $word; done Character iteration (bash substring):

for ((i=0;i<${#word};i++)); do echo ${word:i:1}; done

Source: 大CC, http://www.cnblogs.com/me115/p/3427319.html