Master Essential Linux Shell Tools for Text Processing and File Management

1. find File Search

Examples of using find to locate *.txt and *.pdf files, regex search, negation, depth control, type, time, size, permission, user, and actions such as -delete and -exec for copying or changing ownership.

find . \( -name "*.txt" -o -name "*.pdf" \) -print
find . -regex ".*\(\.txt|\.pdf\)$"
find . ! -name "*.txt" -print
find . -maxdepth 1 -type f
find . -type d -print
find . -atime 7 -type f -print
find . -type f -size +2k
find . -type f -perm 644 -print
find . -user weber -type f -print
find . -type f -name "*.swp" -delete
find . -user root -type f -exec chown weber {} \;
find . -mtime +10 -name "*.txt" -exec cp {} OLD \;

2. grep Text Search

Basic grep usage with common options: -o (only matching part), -v (invert match), -c (count), -n (line number), -i (ignore case), -l (list filenames), and recursive search.

grep -c "text" filename
grep -R -n "class" .
grep -e "class" -e "virtual" file
grep -lZ "test" * | xargs -0 rm

3. xargs Argument Conversion

xargs

turns input data into command‑line arguments, enabling pipelines such as grep + find. Useful flags include -d (delimiter), -n (max args per command), -I {} (replace token), and -0 (null delimiter).

cat file.txt | xargs -I {} ./command.sh {}
find source_dir -type f -name "*.cpp" -print0 | xargs -0 wc -l

4. sort Sorting

Sorting options: -n (numeric), -d (dictionary), -r (reverse), -k N (column N), and examples.

sort -nrk 1 data.txt
sort -bd data.txt

5. uniq Duplicate Removal

Remove duplicate lines, count occurrences, or list only repeated lines.

sort unsort.txt | uniq
sort unsort.txt | uniq -c
sort unsort.txt | uniq -d

6. tr Character Translation

Common tr usages: substitution, deletion, complement, squeezing, and character classes.

echo 12345 | tr '0-9' '9876543210'
cat file | tr '\t' ' '
cat file | tr -d '0-9'
cat file | tr -c '0-9' '
'
cat file | tr -s ' '

7. cut Column Extraction

Extract specific fields or characters with cut, set delimiter with -d, and choose units ( -b, -c, -f).

cut -f2,4 filename
cut -f3 --complement filename
cut -d ";" -f2 filename
cut -c1-5 file

8. paste Column Merging

Combine files column‑wise; default delimiter is a tab, customizable with -d.

paste file1 file2
paste -d "," file1 file2

9. wc Word/Line/Byte Count

Count lines ( -l), words ( -w), or bytes ( -c).

wc -l file
wc -w file
wc -c file

10. sed Stream Editing

Single‑line ( s/text/replace/) and global ( s/text/replace/g) replacements, in‑place editing with -i, removing empty lines, back‑references, and inserting characters.

sed 's/text/replace_text/' file
sed 's/text/replace_text/g' file
sed -i 's/text/replace_text/g' file
sed '/^$/d' file
sed 's/^.{3}/&\//g' file

11. awk Data‑Stream Processing

Awk script structure with BEGIN, main block, and END. Built‑in variables ( NR, NF, $0, $1 …), pattern matching, field printing, aggregations, external variable passing, loops, and implementing head / tail functionality.

awk 'BEGIN{print "start"} {print} END{print "end"}' file
awk '{print NR":"$0"-"$1"-"$2}' file
awk 'END{print NR}' file
awk 'NR<=10{print}' file   # head
awk '{buffer[NR%10]=$0} END{for(i=0;i<10;i++) print buffer[i]}' file   # tail

12. Iterating Lines, Words, Characters

Loop over file lines with while read or awk, iterate words with for word in $line, and iterate characters using Bash parameter expansion.

while read line; do echo $line; done < file.txt
awk '{print}' file.txt
for word in $line; do echo $word; done
for ((i=0;i<${#word};i++)); do echo ${word:i:1}; done