Master Shell Scripting: From Basics to Advanced Debugging Techniques

Preface

When a server runs out of memory and hangs, the usual answer is to kill the process using the most CPU/memory. The following one‑liner kills the two processes that consume the most memory:

ps aux | sort -rn -k +3 | head -2 | awk '{print $2}' | xargs kill -9

To avoid repeating this command, we can encapsulate it in a reusable shell library.

Creating a Shell Function Library

1. Create a file shell-libs.sh to hold reusable functions.

2. Define a function killProcessByMemory that accepts a number n and kills the top n memory‑consuming processes.

function killProcessByMemory() {
  params=($*)
  killCmd="kill -9"
  memoryInfoCmd='ps aux | sort -rn -k +3 | head -${1} | awk "{print $2}"'
  needkill=$(arr_includes params kill)
  if boolean $needkill; then
    echo "Need to delete"
    handledCmd="$memoryInfoCmd | $killCmd"
  else
    echo "Only query"
    handledCmd=$memoryInfoCmd
  fi
  echo "Final command:"
  echo $handledCmd
}

After sourcing the library ( source shell-libs.sh), you can run:

# Query only
killProcessByMemory 2
# Delete
killProcessByMemory 2 kill

This one‑time setup gives you a reusable tool for memory‑intensive processes.

Extending Shell Capabilities

Just as JavaScript libraries (lodash, axios) encapsulate common logic, we can create shell libraries for tasks such as OS initialization, remote server configuration, and automated cron jobs.

Initialize OS tools (git, nvm, node, etc.) with a single command.

Configure password‑less SSH for remote servers.

Wrap web‑crawling actions into scheduled tasks.

What Is a Shell?

A shell is a command‑line interpreter that lets users send requests to the Linux kernel. It can launch, suspend, stop programs, and also serves as a powerful scripting language with features like variables, control flow, and pipelines.

Shell Basics

Variables

Define with name=value (no spaces around =).

Use let or arithmetic expansion (( )) for numeric calculations.

Export with export VAR=value to make it visible to child processes.

Unset with unset VAR.

Variable naming rules: start with a letter or underscore, contain only letters, numbers, and underscores, and be no longer than 255 characters.

# Example of string concatenation
x=1
y=2
z=3
k=$x+$y+$z   # Result: 1+2+3

Arrays

colors=(yellow red blue)
echo ${colors[@]}   # All elements
echo ${#colors[@]}   # Length
echo ${colors[0]}    # First element

Declaring Types

Use declare to set attributes: -a – array -i – integer -r – read‑only -x – environment variable

declare -i count=5   # Integer
declare -r PI=3.14   # Read‑only

Environment Variables and Configuration Files

Common variables include $PATH, $PS1, $?, $$, and positional parameters $1 … $9, ${10}, etc.

System‑wide files ( /etc/profile, /etc/bashrc) and user‑specific files ( ~/.bash_profile, ~/.bashrc) are sourced depending on whether the shell is a login shell.

Shell Statements

Conditional Tests

String equality: [ "$a" = "$b" ] Numeric comparisons: -eq, -gt, -lt, etc., or [[ $a -gt $b ]].

File tests: -e (exists), -f (regular file), -d (directory), -x (executable).

Branching

# if‑else example
if [ $UID -eq 0 ]; then
  echo "root user"
else
  echo "regular user"
fi

Loops

Standard for loop:

for i in {1..10}; do
  echo $i
done

C‑style loop:

for ((i=1; i<=10; i++)); do
  echo $i
done

While loop:

a=1
while [ $a -lt 10 ]; do
  ((a++))
  echo $a
done

Until loop (runs until condition becomes true):

until [ $a -ge 10 ]; do
  ((a++))
  echo $a
done

Break and continue can control loop flow.

Functions

Define with function name { … } or name() { … }. Use local for scoped variables.

function cdls() {
  cd "$1"
  ls
}
cdls /tmp

Pass arguments as $1, $2, … ${10}. Return values via echo because return only supports integers.

function push() {
  local name=$1
  eval local innerArr=(\${${name}[@]})
  local item=$2
  innerArr+=($item)
  echo ${innerArr[@]}
}
arr=(a b c)
arr=($(push arr d))   # arr now contains a b c d

Source a library to make its functions available in the current shell:

source ./libs/os.sh
platform=$(getPlatform)
echo $platform

Special Shell Features

Signal Trapping

# Print a message when SIGTERM (15) is received
trap "echo sig 15" 15
while :; do :; done

Job Scheduling

One‑time jobs (at) : Use at now + 1 minute and type commands, ending with Ctrl‑D. Manage with atq and atrm.

Cron (periodic) : Edit with crontab -e. Format is minute hour day month weekday command. Example to log the date every minute: * * * * * /usr/bin/date >> /tmp/timestamp.txt Anacron (delayed periodic) : Handles jobs that may be missed when the machine is off. Configuration in /etc/anacrontab includes period, delay, and command.

flock (locking) : Prevents concurrent execution of a script.

flock -xn "/tmp/f.lock" -c "/root/a.sh"

Pipelines and Redirection

Use | to pipe output of one command to another. Example: ps aux | grep bash | wc -l Redirection symbols: > – overwrite file. >> – append. 2> – redirect stderr. &> – redirect both stdout and stderr.

# Write "hello" to a file
echo hello > /tmp/hello.txt
# Append variable content
read a < /tmp/hello.txt
echo $a >> /tmp/hello.txt
# Capture error output
nocmd 2> error.log

Debugging Shell Scripts

Logging

Insert echo statements or use VS Code snippets that generate log lines automatically.

set Options

Place at the top of a script:

set -ux   # Show each command and treat unset variables as errors

Common flags: -e – exit on any error. -u – error on unset variables. -x – print each command before execution. -o pipefail – make pipelines fail if any component fails.

Interactive Debugger

Install the VS Code "Bash Debug" extension (requires Bash 4+). Add a launch.json configuration:

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "bashdb",
      "request": "launch",
      "name": "Bash Debug",
      "program": "${file}"
    }
  ]
}

Run the debugger to step through scripts, inspect variables, and set breakpoints.

Conclusion

This guide covered shell fundamentals, function libraries, variable handling, control flow, job scheduling, pipelines, redirection, and debugging techniques, providing a solid foundation for writing maintainable Bash scripts.