Master the Linux Command Line: Essential Tips and Tricks for Engineers

Proficient use of the command line is often overlooked, yet it dramatically increases an engineer's flexibility and productivity. This article summarizes a collection of useful command‑line tricks discovered while working on Linux, ranging from basic to advanced techniques.

Must‑Read

Scope:

This article is useful for both newcomers and experienced users. It aims for broad coverage, concrete examples, and conciseness, ensuring each tip saves time in specific contexts. Although written for Linux, most content applies to other Unix‑like systems, including macOS and Cygwin.

The focus is on interactive Bash, though many tips work with other shells or Bash scripts. Both standard Unix commands and those requiring additional packages are included.

Basics

Learn Bash fundamentals by running man bash and reading it fully. Bash is powerful and universally available, unlike other shells that may limit portability.

Master a text editor—Vim ( vi) is highly recommended for terminal editing, though Emacs or modern IDEs are also options.

Use man and apropos to read documentation, and help or help -d for built‑in Bash commands.

Redirect output and input with >, >>, <, and pipe with |. Understand stdout and stderr.

Employ wildcards like *, ?, [...] and understand quoting differences between ' and ".

Familiarize yourself with Bash job control: &, Ctrl‑Z , Ctrl‑C , jobs, fg, bg, kill.

Learn ssh basics and use ssh-agent and ssh-add for password‑less authentication.

Basic file management commands include ls, ls -l, less, head, tail, tail -f, ln, ln -s, chown, chmod, du, df, mount, fdisk, mkfs, lsblk, and understanding inodes with ls -i and df -i.

Network management basics: ip or ifconfig, and dig.

Regular expressions with grep / egrep and useful options such as -i, -o, -v, -A, -B, -C.

Install packages using the appropriate package manager for your distribution: apt-get, yum, dnf, pacman, pip, brew, etc.

Daily Use

Use Tab for auto‑completion and Ctrl‑R to search command history.

Editing shortcuts: Ctrl‑W delete the previous word, Ctrl‑U delete the whole line, Alt‑B / Alt‑F move by word, Ctrl‑A go to line start, Ctrl‑E go to line end, Ctrl‑K delete to end of line, Ctrl‑L clear screen. See man readline for more.

Switch to vi‑style key bindings with set -o vi and back with set -o emacs.

Edit long commands in your preferred editor: set EDITOR=vim then press Ctrl‑X Ctrl‑E (or Esc v in vi mode).

View command history with history. Use shortcuts like !$ (last argument) and !! (last command), though Ctrl‑R and Alt‑. are often preferred.

Return to the previous directory with cd -. Comment out a partially typed command by pressing Alt‑# or inserting # at the line start.

Combine commands with xargs (or parallel) and control arguments per line with -L and parallelism with -P. Test with xargs echo. Use -I{} for placeholder substitution.

Text and Data Processing

Find files by name: find . -iname '*something*' or globally with locate something (remember to run updatedb).

Search source code with ag (better than grep -r).

Convert HTML to plain text: lynx -dump -stdin.

Convert between Markdown, HTML, and other formats with pandoc.

Process XML using xmlstarlet.

Handle JSON with jq.

Manipulate CSV/Excel files with csvkit tools such as in2csv, csvcut, csvjoin, csvgrep.

Interact with Amazon S3 using s3cmd, s4cmd, the official aws CLI, or saws.

Sort and deduplicate data with sort and uniq (options -u, -d), and compare files with comm.

Use cut, paste, and join for column manipulation; many overlook join.

Count lines, words, bytes with wc ( -l, -w, -c, -m).

Duplicate input to a file with tee, e.g., ls -al | tee file.txt.

Set locale for consistent sorting; use export LC_ALL=C to ignore locale‑specific ordering.

Simple data processing with awk and sed, e.g., awk '{ x += $3 } END { print x }' myfile is often faster than equivalent Python.

Batch rename files with rename or more complex patterns with repren.

Randomly select lines using shuf.

Advanced sort usage: numeric sort with -n, human‑readable sizes with -h, stable sort with -s, and specify key fields with -k.

Insert literal tabs in Bash with Ctrl‑V Tab or $'\t'.

Compare and merge source code using diff and patch; diffstat provides a summary. Use diff -r for recursive directory comparison.

Inspect binary files with hd, bvi, or extract strings with strings. Use xdelta3 for binary delta compression.

Convert text encodings with iconv or the more powerful uconv.

System Debugging

Web debugging with curl (or curl -I), wget, or httpie.

Monitor system resources with iostat, netstat, top / htop, and dstat. Use glances for an overview of multiple subsystems.

Check network connections with netstat or ss.

For deep system insight, use glances, free, vmstat, and pay attention to cached memory.

Java debugging tools include kill -3 <pid>, jps, jstat, jstack, jmap, and SJK tools.

Trace routes with mtr, view disk usage with ncdu, and monitor bandwidth with iftop or nethogs.

Load‑test web servers with ab (ApacheBench) or siege. Capture network traffic with wireshark, tshark, or ngrep.

Debug programs with strace and ltrace (use -c for profiling, -p to attach to a running process).

Inspect shared libraries with ldd and debug with gdb.

Explore the /proc filesystem for process information (e.g., /proc/cpuinfo, /proc/meminfo, /proc/<pid>/fd/).

Historical performance data is available via sar. For deeper analysis, consider perf, SystemTap, or sysdig.

Identify the running kernel and distribution with uname -a and lsb_release -a. View kernel messages with dmesg.

One‑Liners

Set operations on text files can be performed with sort / uniq pipelines, e.g., union, intersection, and difference using appropriate options ( -u, -d, -c).

Search all files in a directory with grep . * (useful for configuration directories like /etc, /proc, /sys).

Calculate the sum of the third column in a file efficiently with awk '{ x += $3 } END { print x }' myfile.

Recursively list files with size and date using find . -type f -ls as a more readable alternative to ls -lR.

Count occurrences of a parameter in a log file (e.g., acct_id) with a pipeline of cat, egrep, cut, sort, and uniq -c.

Obscure but Useful Tools

expr

: evaluate expressions or regex matches m4: macro processor yes: repeat a string cal: display a calendar env and printenv: run commands with modified environment or display variables look: find words with a given prefix cut, paste, join: manipulate columns of data fmt, pr, fold, column: format text expand / unexpand: convert tabs to spaces and vice versa nl, seq, bc, factor: various utilities for numbering, calculations, and factorization gpg: encrypt and sign files nc, socat: network debugging and data transfer slurm: network visualization dd, file, tree, stat, time, lockfile, logrotate, watch, tac, shuf, comm, pv, hd, hexdump, xxd, bvi, strings, tr, iconv, uconv, split, csplit, sponge, units, apg, 7z, ldd, nm, ab, strace, mtr, cssh, rsync, wireshark, tshark, ngrep, host, dig, lsof, dstat, iostat, mpstat, vmstat, htop, last, w, id, sar, iftop, nethogs, ss, dmesg, sysctl, hdparm, lsb_release, lsblk, lshw, lscpu, lspci, lsusb, dmidecode, lsmod, modinfo, fortune, ddate, sl provide specialized functionality for various tasks.

macOS‑Only Tips

Use brew (Homebrew) or port (MacPorts) for package management on macOS.

Copy command output to the clipboard with pbcopy and paste with pbpaste.

Set the Option key as Meta in Terminal preferences for Alt‑style shortcuts.

Open files or applications with open or open -a <AppName>.

Search files with mdfind and view metadata (e.g., EXIF) with mdls.

Be aware that many macOS commands differ slightly from their GNU counterparts; consult the BSD manual pages for differences, or use GNU versions (e.g., gawk, gsed) when needed.