Master Linux Text Processing: Essential find, grep, sort, awk, and More
This guide walks through the most common Linux shell tools for text processing—including find, grep, xargs, sort, uniq, tr, cut, paste, wc, sed, and awk—explaining their core options, practical examples, and how to combine them for powerful command‑line workflows.
Common Linux Shell Text Processing Tools
Tools covered: find, grep, xargs, sort, uniq, tr, cut, paste, wc, sed, awk.
01 find – File Search
Search for files by name, pattern, depth, type, time, size, permission, or owner.
find . ( -name "*.txt" -o -name "*.pdf" ) -printfind . -regex ".*(\.txt|\.pdf)$"find . ! -name "*.txt" -printfind . -maxdepth 1 -type ffind . -type d -printfind . -atime 7 -type f -printfind . -type f -size +2kfind . -type f -perm 644 -printfind . -type f -user weber -printDelete files: find . -type f -name "*.swp" -delete Execute a command on each match (strong exec):
find . -type f -user root -exec chown weber {} \;find . -type f -mtime +10 -name "*.txt" -exec cp {} OLD \;Use -print0 for null‑terminated output when filenames contain spaces.
02 grep – Text Search
Basic pattern matching and useful options.
grep pattern filegrep -c "text" filenameRecursive search with line numbers: grep "class" -R -n . Match multiple patterns: grep -e "class" -e "virtual" file Combine with xargs to delete matching files:
grep "test" file* -lZ | xargs -0 rmxargsconverts input into command‑line arguments, useful with many commands such as grep and find.
cat file.txt | xargscat single.txt | xargs -n 303 sort – Sorting
Numeric vs. dictionary order, reverse, column selection.
sort -nrk 1 data.txtsort -bd data04 uniq – Remove Duplicate Lines
Basic deduplication and counting.
sort unsort.txt | uniqsort unsort.txt | uniq -csort unsort.txt | uniq -dSpecify start position and width with -s and -w.
05 tr – Translate Characters
General translation, deletion, complement, and squeezing.
echo 12345 | tr '0-9' '9876543210'cat text | tr '\t' ' 'cat file | tr -d '0-9'cat file | tr -c '0-9'cat file | tr -d -c '0-9 'cat file | tr -s ' 'tr '[:lower:]' '[:upper:]'06 cut – Column Extraction
Extract specific fields, complement, set delimiters, and select ranges.
cut -f2,4 filenamecut -f3 --complement filenamecut -d';' -f2 filenamecut -c1-5 filecut -c-2 file07 paste – Merge Columns
Combine two files side‑by‑side; default delimiter is a tab, can be changed.
paste file1 file2paste file1 file2 -d ','08 wc – Count Lines, Words, Bytes
wc -l filewc -w filewc -c file09 sed – Stream Editing
Single substitution, global substitution, in‑place edit, delete empty lines, capture groups, variable expansion.
sed 's/text/replace_text/' filesed 's/text/replace_text/g' filesed -i 's/text/replace_text/g' filesed '/^$/d' filesed 's/^.{3}/&/g' fileUse double quotes for variable expansion:
p=pattern; r=replace; echo "line with $p" | sed "s/$p/$r/g"10 awk – Data‑Stream Processing
Program structure, built‑in variables (NR, NF, $0, $1…), pattern actions, printing, loops, functions, external variable passing, and file/command input.
awk 'BEGIN{print "start"} {print} END{print "End"}'awk '{print NR ":" $0 "-" $1 "-" $2}'awk '{print $2, $3}' fileawk 'END{print NR}' fileawk 'BEGIN{sum=0} {sum+=$1} END{print sum}' filevar=1000; awk -v v=$var 'BEGIN{print v}'awk 'NR<5' fileawk '/linux/' fileawk -F: '{print $NF}' /etc/passwdawk '{"grep root /etc/passwd" | getline cmdout; print cmdout}'awk 'for(i=0;i<10;i++){print i}'awk 'for(i in array){print array[i]}'Implement head and tail with awk:
awk 'NR<=10{print}' filename # head
awk '{buffer[NR%10]=$0} END{for(i=0;i<10;i++) print buffer[i]}' filename # tailPrint specific columns or ranges:
ls -lrt | awk '{print $6}'awk '{print $2, $3}' fileawk 'NR==4,NR==6{print}' fileawk '/start_pattern/,/end_pattern/' filenameCommon built‑in functions: index, sub, match, length, printf.
awk '{"grep root /etc/passwd" | getline cmdout; print length(cmdout)}'Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
