Master Linux Commands: From Basics to Advanced Ops – The Ultimate One‑Page Guide

Introduction

As a seasoned operations engineer with a decade of experience, I share the most essential Linux commands that resolve 80% of production incidents, helping newcomers and veterans alike avoid years of trial and error.

1. File and Directory Operations

1.1 Basic Navigation Commands

# pwd - display current directory
$ pwd
/home/admin/logs

# cd - change directory (90% of people don’t know these tricks)
$ cd -               # quickly return to previous directory
$ cd ~username       # go to a specific user’s home
$ cd !$              # go to the last argument of the previous command

# ls - list directory contents (advanced usage)
$ ls -lhtr           # list by time descending, human‑readable
$ ls -ld */          # show only directories
$ ls -la | grep "^d" # another way to list directories

Practical tip: Use pushd and popd to maintain a directory stack when switching between log directories.

$ pushd /var/log/nginx   # save current dir and switch
$ pushd /var/log/mysql
$ dirs -v                # view stack
$ popd                    # return to previous directory

1.2 Advanced File Operations

# find - essential for locating files
$ find /var/log -name "*.log" -mtime +7 -size +100M -exec rm {} \;
# delete log files older than 7 days larger than 100M

$ find . -type f -name "*.conf" -exec grep -l "error" {} \;
# find config files containing "error"

$ find /data -type f -mmin -5
# find files modified in the last 5 minutes (troubleshooting gem)

# Safe cp/mv/rm usage
$ cp -av source/ dest/   # preserve attributes, verbose
$ mv -i file1 file2      # interactive move to avoid mistakes
$ rm -I *.log            # prompt before deleting multiple files

# rsync - efficient synchronization (100× faster than cp/scp)
$ rsync -avzP --delete source/ dest/
# -a archive, -v verbose, -z compress, -P progress, --delete remove extraneous files

$ rsync -avz --exclude='*.log' --exclude='cache/' source/ dest/
# exclude specific files or directories

2. Text Processing Trio: grep, sed, awk

2.1 grep – Powerful Text Search

# grep advanced tricks
$ grep -B 3 -A 3 "ERROR" app.log   # show 3 lines before and after matches
$ grep -v "^#\|^$" config.conf      # filter out comments and empty lines
$ grep -r --include="*.java" "TODO" .   # recursive search in specific file types

# useful regex
$ grep -E "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" access.log   # match IP addresses
$ grep -oP '(?<=user_id=)[0-9]+' access.log | sort -u   # extract unique user_id values

# performance tip
$ LC_ALL=C grep pattern file   # use C locale for ~30% speed boost
$ grep -F "fixed_string" file   # fixed‑string search, faster

2.2 sed – Stream Editor

# sed practical examples
$ sed -i.bak 's/old/new/g' file.txt   # replace and backup original
$ sed -n '100,200p' huge_file.txt      # display lines 100‑200 only
$ sed '/^$/d' file.txt                # delete empty lines

# batch modify config files
$ sed -i 's/^#\(.*nginx.*\)/\1/g' config.conf   # uncomment lines containing nginx

# advanced usage
$ sed -n '/START/,/END/p' file.txt   # print content between START and END
$ sed '1~2d' file.txt                # delete odd lines
$ sed -e 's/^/PREFIX_/' -e 's/$/_SUFFIX/' file.txt   # add prefix and suffix

2.3 awk – Data Processing Expert

# core awk usage
$ awk '{sum+=$5} END {print sum/NR}' data.txt   # average of column 5
$ awk '$3 > 1000 {print $1, $3}' access.log   # rows where column 3 > 1000

# log analysis example
$ awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10   # top 10 IPs

# field separator and join-like operation
$ awk -F: '$3 >= 1000 {print $1}' /etc/passwd   # users with UID >= 1000
$ awk 'BEGIN{OFS=","} {$1=$1; print}' file.txt   # convert spaces to commas
$ awk 'NR==FNR{a[$1]=$2;next} {print $0, a[$1]}' file1 file2   # simple join

3. System Monitoring and Performance Analysis

3.1 Process Management

# ps useful tricks
$ ps aux --sort=-%cpu | head -10   # top CPU consumers
$ ps aux --sort=-%mem | head -10   # top memory consumers
$ ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head -10   # custom output

# find specific process
$ ps -ef | grep "[n]ginx"   # avoid grepping the grep itself
$ pgrep -f "java.*tomcat"   # elegant way to find Java/Tomcat processes
$ pidof nginx                # get PID of nginx

# proper kill usage
$ kill -TERM $pid   # graceful termination (default)
$ kill -HUP $pid     # reload configuration
$ kill -USR1 $pid    # user‑defined signal (e.g., nginx log rotation)
$ timeout 5 some_command   # limit execution time

3.2 System Resource Monitoring

# top/htop advanced usage
$ top -b -n 1 | head -20   # batch mode for scripting
$ top -p $(pgrep -d ',' java)   # monitor all java processes

# vmstat – virtual memory stats
$ vmstat 1 10   # output every second, 10 times

# iostat – I/O statistics
$ iostat -x 1   # detailed I/O stats, %util shows device utilization

# sar – system activity report
$ sar -u 1 10   # CPU usage
$ sar -r 1 10   # memory usage
$ sar -n DEV 1 10   # network traffic
$ sar -d 1 10   # disk I/O

3.3 Network Diagnostics

# ss – faster than netstat
$ ss -tulpn | grep :80   # check listeners on port 80
$ ss -ant | awk '{print $1}' | sort | uniq -c   # count TCP states

# netstat – legacy tool
$ netstat -an | grep ESTABLISHED | wc -l   # count established connections

# tcpdump – packet capture
$ tcpdump -i eth0 -nn host 192.168.1.100 and port 80   # capture specific host/port
$ tcpdump -i any -w capture.pcap -C 100 -W 10   # rotate 10 files of 100 MB each

# curl performance test
$ curl -w "@curl-format.txt" -o /dev/null -s https://example.com   # detailed HTTP metrics

# port scanning
$ nc -zv hostname 22-80   # quick port range scan
$ nmap -sV -p- hostname   # full port scan with service version detection

4. Log Analysis and Fault Diagnosis

4.1 Log Analysis Techniques

# count error occurrences
$ grep "ERROR" app.log | awk '{print $5}' | sort | uniq -c | sort -rn

# time‑range analysis
$ awk '$1 >= "2024-01-01" && $1 <= "2024-01-31"' access.log

# real‑time monitoring of multiple logs
$ tail -f /var/log/nginx/*.log /var/log/mysql/*.log

# compressed log search
$ zcat access.log.*.gz | grep "pattern"
$ zless access.log.gz

# advanced error block extraction
$ awk '/ERROR/{print $0; for(i=1;i<=3;i++){getline; print}}' app.log   # print ERROR line + next 3 lines

4.2 Fault‑Diagnosis Practice

# disk space issues
$ df -h | awk '$5+0 > 80'   # partitions >80% usage
$ du -sh /* 2>/dev/null | sort -rh | head -10   # top 10 largest directories
$ find / -size +1G -type f 2>/dev/null   # files larger than 1 GB

# memory problems
$ free -h | awk '/^Mem:/ {print "Used: " $3 "/" $2 " (" $3/$2*100 "% )"}'
$ ps aux | awk '{sum+=$6} END {print "Total RSS: " sum/1024 " MB"}'

# CPU bottleneck
$ mpstat -P ALL 1   # per‑CPU usage
$ pidstat -u 1 10   # per‑process CPU usage

# network connection issues
$ ss -s   # connection summary
$ conntrack -L | wc -l   # size of connection tracking table

5. Shell Scripting Best Practices

5.1 Script Writing Techniques

#!/bin/bash
set -euo pipefail
IFS=$'
\t'

# error handling
trap 'echo "Error on line $LINENO"' ERR

# logging function
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a /var/log/script.log; }

# argument check
if [ $# -lt 1 ]; then
  echo "Usage: $0 <argument>"
  exit 1
fi

# concurrency control
MAX_JOBS=5
for item in "${items[@]}"; do
  while [ $(jobs -r | wc -l) -ge $MAX_JOBS ]; do sleep 1; done
  process_item "$item" &
  done
wait

5.2 Useful Shell Function Library

# retry mechanism
retry() {
  local max_attempts="$1" delay="$2"; shift 2
  local count=0
  until "$@"; do
    count=$((count+1))
    if [ $count -ge $max_attempts ]; then
      echo "Command failed after $count attempts"
      return 1
    fi
    echo "Attempt $count failed, retrying in ${delay}s..."
    sleep "$delay"
  done
}

# colored output
red() { echo -e "\033[31m$*\033[0m"; }
green() { echo -e "\033[32m$*\033[0m"; }
yellow() { echo -e "\033[33m$*\033[0m"; }

# progress bar
progress_bar() {
  local duration=$1 steps=50
  local sleep_time=$(echo "scale=3; $duration / $steps" | bc)
  for i in $(seq 1 $steps); do
    printf "\r["; printf "%0.s=" $(seq 1 $i); printf "%0.s " $(seq $i $((steps-1)));
    printf "] %d%%" $((i*100/steps))
    sleep $sleep_time
done
  echo
}

6. Container and Cloud‑Native Operations

6.1 Docker Common Commands

# list containers with custom format
$ docker ps -a --format "table {{.Names}}	{{.Status}}	{{.Ports}}"

# view resource usage
$ docker stats --no-stream --format "table {{.Container}}	{{.CPUPerc}}	{{.MemUsage}}"

# clean up unused resources
$ docker system prune -a --volumes

# view logs
$ docker logs -f --tail 100 container_name

# batch stop and remove
$ docker stop $(docker ps -aq)
$ docker rm $(docker ps -aq -f status=exited)

# debugging
$ docker exec -it container_name /bin/bash
$ docker cp container:/path/to/file ./
$ docker diff container_name   # view filesystem changes

6.2 Kubernetes Operations

# pod management
$ kubectl get pods -A --sort-by='.status.startTime'
$ kubectl top pods --all-namespaces | sort -k3 -rn | head
$ kubectl describe pod pod_name -n namespace

# log viewing
$ kubectl logs -f deployment/app --all-containers=true
$ kubectl logs pod_name --previous   # view previous crashed container logs

# resource management
$ kubectl get events --sort-by='.lastTimestamp' -A
$ kubectl rollout history deployment/app
$ kubectl set image deployment/app app=image:tag --record

# debugging
$ kubectl exec -it pod_name -- /bin/bash
$ kubectl port-forward pod/pod_name 8080:80
$ kubectl debug pod_name -it --image=busybox

7. Security Hardening and Auditing

7.1 System Security Checks

# find SUID/SGID files
$ find / -perm -4000 -type f 2>/dev/null   # SUID
$ find / -perm -2000 -type f 2>/dev/null   # SGID

# orphaned files
$ find / -nouser -o -nogroup 2>/dev/null

# login audit
$ last -10
$ lastb
$ who -H
$ w

# file integrity
$ find /etc -type f -mtime -1   # config files changed in last 24h
$ rpm -Va   # RedHat package verification
$ debsums   # Debian package verification

7.2 Network Security Monitoring

# firewall rules
$ iptables -L -n -v --line-numbers
$ iptables -I INPUT 1 -s 192.0.2.123 -j DROP   # block IP
$ iptables -D INPUT 1

# connection monitoring
$ ss -tupn | grep LISTEN
$ lsof -i :80
$ netstat -tulpn | grep -v 127.0.0.1

# intrusion detection
$ grep "Failed password" /var/log/secure | awk '{print $11}' | sort | uniq -c | sort -rn   # SSH brute‑force stats
$ tcpdump -nn -c 100 'tcp[tcpflags] == tcp-syn'   # monitor SYN packets (possible SYN flood)

8. Performance Optimization in Practice

8.1 System Tuning Commands

# kernel parameters
$ sysctl -a | grep net.ipv4
$ echo "net.ipv4.tcp_fin_timeout = 30" >> /etc/sysctl.conf
$ sysctl -p

# file descriptor limits
$ ulimit -a
$ ulimit -n 65535
$ echo "* soft nofile 65535" >> /etc/security/limits.conf

# I/O scheduler
$ cat /sys/block/sda/queue/scheduler
$ echo deadline > /sys/block/sda/queue/scheduler

# CPU affinity
$ taskset -c 0-3 command   # bind to CPUs 0‑3
$ taskset -p -c 0,1 $PID   # bind existing process

8.2 Service Optimization Examples

# Nginx
$ nginx -t   # syntax check
$ nginx -T   # full config dump
$ ab -n 10000 -c 100 http://localhost/   # load test

# MySQL
$ mysql -e "SHOW VARIABLES LIKE '%buffer%'"
$ mysql -e "SHOW STATUS LIKE 'Threads%'"
$ mysqladmin processlist
$ mysqltuner   # tuning suggestions

# Redis
$ redis-cli --latency   # latency monitor
$ redis-cli --bigkeys   # find large keys
$ redis-cli monitor   # real‑time command monitor

9. Automation Tools

9.1 Cron Job Management

# view active crontab entries (ignore comments/blank lines)
$ crontab -l | grep -v '^#' | grep -v '^$'

# backup and clear crontab
$ crontab -l > backup.cron && crontab -r

# special time expressions
@reboot   # run at system boot
@daily    # run daily
@hourly   # run hourly

# redirect output
*/5 * * * * /path/to/script.sh >> /var/log/cron.log 2>&1
*/5 * * * * /path/to/script.sh > /dev/null 2>&1

9.2 Batch Operation Techniques

# parallel SSH execution
$ cat hosts.txt | xargs -P 10 -I {} ssh {} "uptime"
$ parallel -j 10 ssh {} uptime :::: hosts.txt

# bulk file processing
$ for f in *.txt; do mv "$f" "${f%.txt}.bak"; done
$ rename 's/\.txt$/\.md/' *.txt

# SSH batch commands
$ pssh -h hosts.txt -l root -i "hostname; uptime"
$ pdsh -w node[01-10] "systemctl restart nginx"

10. Fault Recovery and Data Backup

10.1 Backup Strategies

# tar backup
$ tar -czf backup-$(date +%F).tar.gz --exclude='*.log' /data/
$ tar --listed-incremental=snapshot.snar -czf incremental.tar.gz /data/

# rsync incremental backup
$ rsync -avz --backup --backup-dir=/backup/$(date +%F) source/ dest/
$ rsync -avz --link-dest=/backup/yesterday/ source/ /backup/today/

# database dump
$ mysqldump -u root -p --all-databases --single-transaction > backup.sql
$ pg_dumpall -U postgres > postgres_backup.sql

# backup verification
$ tar -tzf backup.tar.gz | head
$ mysql test < backup.sql   # test restore

10.2 Disaster Recovery

# file recovery
$ extundelete /dev/sda1 --restore-file /path/to/file
$ testdisk /dev/sda   # partition recovery

# process core dump
$ gcore -o dump $PID
$ gdb /path/to/program dump.$PID

# system rescue modes
$ systemctl rescue
$ systemctl emergency
$ init 1   # single‑user mode

# network restoration
$ ip link set eth0 up
$ dhclient eth0
$ ip addr add 192.168.1.100/24 dev eth0
$ ip route add default via 192.168.1.1

Case Study: Production Incident Resolution

At 3 AM an alert indicated extremely slow website response. The engineer quickly checked system load, identified a MySQL process consuming 300 % CPU, examined the slow‑query log, killed the offending query, analyzed Nginx access logs to spot a malicious IP, blocked it with iptables, and monitored recovery with watch until performance normalized—all within 15 minutes.