Master Linux Shell Scripting: From Basics to Real-World Automation

1. Why Learn Linux Shell Programming

Shell scripting is a versatile tool for system administrators, developers, and hobbyists, enabling automation of repetitive tasks such as file management, user permissions, performance monitoring, and deployment.

2. Understanding Linux Shell

2.1 What is a Shell

A shell is the bridge between the user and the kernel; commands entered in a terminal are interpreted by the shell and executed by the system.

2.2 Common Shell Types

bash : default on most Linux distributions, supports command line editing, history, job control, and tab completion.

zsh : offers advanced completion and theming (e.g., Oh My Zsh), with a slightly higher learning curve.

3. Core Shell Commands

(1) File and Directory Operations

pwd

– print current working directory. ls – list directory contents; ls -l for detailed view, ls -a to include hidden files. cd – change directory; cd .. to go up, cd - to return to previous directory. mkdir – create directories; mkdir -p creates parent directories as needed. rmdir – remove empty directories. cp – copy files ( cp src dest) or directories ( cp -r src dest). mv – move or rename files and directories.

(2) Viewing and Editing Files

cat

– display file contents. more – paginate output. less – advanced pagination with search. head – show the first N lines. tail – show the last N lines; tail -f follows a growing file. vi/vim – classic text editors; basic navigation and editing commands are described.

(3) System Information and Process Management

top

– real‑time resource monitor. ps – list processes; ps aux shows all users. kill – terminate a process by PID; kill -9 forces termination. free – display memory usage. df – show disk space usage.

(4) Permissions and User Management

chmod

– change file mode (e.g., chmod 744 file). chown – change owner and group. sudo – execute commands as root. useradd, userdel – create and delete users.

4. Advanced Shell Programming

4.1 Variables and Data Types

Variables are defined without spaces ( name=Alice). By default they are strings; arithmetic requires $((...)) or declare -i. String length: ${#var}. Substring: ${var:offset:length}.

4.2 Operators and Expressions

Arithmetic operators (+, -, *, /, %) use $((...)) or expr. Relational operators ( -eq, -gt, etc.) compare integers. Logical operators ( -a, -o, !) combine conditions.

4.3 Flow Control

if statements with then, else, elif, fi.

for loops over lists or numeric ranges.

while loops repeat while a condition is true.

case statements for multi‑branch selection.

4.4 Functions

Define with func_name() { commands; }. Return values via return (0‑255) or capture output with command substitution. Use local for scoped variables.

5. Practical Case Studies

5.1 System Monitoring Script

#!/bin/bash
cpu_usage=$(top -bn1 | grep '%Cpu(s)' | awk '{print $2 + $4}')
mem_total=$(free -m | awk '/Mem:/{print $2}')
mem_used=$(free -m | awk '/Mem:/{print $3}')
mem_usage=$(echo "scale=2; ($mem_used/$mem_total)*100" | bc)
disk_usage=$(df -h / | awk 'NR==2{print $5}' | sed 's/%//')
cpu_warn=80
mem_warn=80
disk_warn=80
if (( $(echo "$cpu_usage > $cpu_warn" | bc -l) )); then
    echo "CPU usage high: $cpu_usage%" | mail -s "CPU Alert" [email protected]
fi
if (( $(echo "$mem_usage > $mem_warn" | bc -l) )); then
    echo "Memory usage high: $mem_usage%" | mail -s "Memory Alert" [email protected]
fi
if (( $(echo "$disk_usage > $disk_warn" | bc -l) )); then
    echo "Disk usage high: $disk_usage%" | mail -s "Disk Alert" [email protected]
fi

5.2 Log Analysis Script

#!/bin/bash
log_file="access.log"
ip_count=$(awk '{print $1}' "$log_file" | sort | uniq -c | sort -nr)
echo "Top 10 IPs:"
echo "$ip_count" | head -10

5.3 Automated Backup Script

#!/bin/bash
source_dir="/home/user/data"
backup_dir="/backup"
backup_file="${backup_dir}/data_$(date +%Y%m%d).tar.gz"
keep_days=7
tar -czf "$backup_file" "$source_dir"
find "$backup_dir" -type f -name "data_*.tar.gz" -mtime +$keep_days -exec rm {} \;

6. Advanced Tips and Safety

6.1 Error Handling

Use set -e to abort on non‑zero exit codes, set -o pipefail for pipelines, and trap to clean up temporary files on EXIT, INT, or TERM.

6.2 Performance Optimisation

Prefer built‑in commands, minimise pipelines, and use tools like awk for efficient text processing.

6.3 Secure Scripting

Validate variables ( ${VAR:?msg}), sanitize user input with regex, avoid command injection by quoting arguments and avoiding eval.