Boost Your Development Efficiency with Essential Shell Commands

Background

The author shares a personal collection of command‑line utilities that can speed up routine development tasks on Linux or macOS. Rather than teaching shell scripting, the focus is on using existing tools to solve common problems quickly.

Common Tools and Environment

zsh with oh‑my‑zsh and useful plugins (git, autojump, etc.)

Frequently used shortcuts via

bindkey

Basic File and Directory Operations

Standard commands include rm, mkdir, mv, cp, cd, ls, ln, file, stat, wc, head, tail, cat, etc.

Shell Text‑Processing Commands

find

Typical options: -name, -type, -maxdepth, time filters ( -ctime, -atime, -mtime), and action execution with -exec.

find ./ -name "*.json"
find . -maxdepth 7 -name "*.json" -type f
find . -name "*.log.gz" -ctime +7 -size +1M -delete
find . -name "*.scala" -atime -7 -exec du -h {} \;

grep

Common flags: -v (invert), -c (count), -n (line number), -i (ignore case), -l, -L, -R, -e.

grep 'partner' ./*.scala -l
grep -e 'World' -e 'first' -i -R .

xargs

Useful flags: -n (items per command), -I (replace placeholder), -d (delimiter, not supported on macOS).

find . -type f -name "*.jpg" | xargs -n1 -I {} du -sh {}

cut

Options: -b (bytes), -c (characters), -f (fields) with -d (delimiter). Example:

echo "helloworldhellp" | cut -c1-10
cut -d, -f2-8 csu.db.export.csv

paste

Options: -d (delimiter), -s (serial). Example:

paste -d, file1 file2
paste -s -d: file1 file2

join

SQL‑like inner join using -t to set the field separator.

join -1 1 -2 3 j1 j2

comm

Compare two sorted files; flags -1, -2, -3 control which columns to output.

comm -1 file11 file22

sort

Key options: -d, -n, -r, -b, -k.

sort -b -k2 -r file2

uniq

Works with sort to filter duplicates; flags -c, -d, -u, -f.

sort file4 | uniq -c
sort file4 | uniq -d

tr

Translate or delete characters. Examples:

echo '1111234444533hello' | tr '[1-3]' '[a-c]'
tr -d '[1-3]'
tr -s '[0-9]'
tr '[:lower:]' '[:upper:]'

sed

Stream editor for in‑place editing. Common flags: -d (delete), -s (substitute, with g for global), -e (multiple commands), -i (in‑place, backup optional).

sed "2,3d" file2
sed '/one/d' file2
sed 's/one/111/g' file2
sed -e 's/one/111/g' -e '/two/d' file2

awk

Powerful pattern‑action language. Useful variables: NR (record number), NF (field count), $1 … for fields. Field separator set with -F. Typical workflow:

Execute BEGIN block.

Process each line matching a pattern.

Execute END block.

awk '{print NR"("NF"):", $3}' file5
awk -F"xxxx" '{print $1, $2}' mobile.csv
awk '$1>=22 {print NR":" $3}' file5
seq 36 | awk 'BEGIN{sum=0; print "question:"} {print $1" +"; sum+=$1} END{print "="; print sum}' | xargs | sed 's/+ =/=/'

Practical Applications

Log Analysis

Using awk, grep, sort, and uniq to extract top‑10 HTTP 404 paths or hourly request counts from an nginx log.

head -n 10000 std.nginx.log | awk '{print $8, $10}' | grep ',404' | sort | uniq -c | sort -nr -k1 | head -n 10

Database URL Correction

Export rows, replace outdated URLs with sed, then generate SQL update statements with awk.

# Test replacement
head -n 5 customers.csv | sed 's|https://www.tanglei.name/upload/photos/[0-9]\{1,\}/|http://www.tanglei.me|g'
# Apply in place
sed -i '' 's|https://www.tanglei.name/upload/photos/[0-9]\{1,\}/|http://www.tanglei.me|g' customers.csv
# Build SQL
awk -F, '{print "update sometable set photo_url_1 = " $2 ", photo_url_2 = " $3 ", photo_url_3 = " $4 " where id = " $1 ";"}' customers.csv > customer.sql

Other Handy Uses

Generate random passwords from /dev/urandom and tr.

Batch resize images with sips on macOS.

Process JSON efficiently using the jq command‑line tool.