Master Bash Scripting: Essential Safety, Functions, and Debugging Tips

Introduction

The techniques below are an expanded version of Google’s “Testing on the Toilet” (TOTT) approach for Bash scripting.

Script Safety

Start every Bash script with the following lines:

#!/bin/bash
set -o nounset
set -o errexit

This prevents two common problems:

Referencing undefined variables (treated as empty strings).

Ignoring failed commands.

Be aware that some commands (e.g., mkdir -p, rm -f) can suppress errors, and errexit does not catch every failure.

Script Functions

Define reusable functions to improve readability. Examples:

ExtractBashComments() {
  egrep "^#"
}

cat myscript.sh | ExtractBashComments | wc
comments=$(ExtractBashComments < myscript.sh)

SumLines() {
  # iterating over stdin – similar to awk
  local sum=0
  local line=""
  while read line; do
    sum=$((sum + line))
  done
  echo $sum
}

SumLines < data_one_number_per_line.txt

log() {
  # classic logger
  local prefix="[$(date +%Y/%m/%d\ %H:%M:%S)]: "
  echo "${prefix}$@" >&2
}

log "INFO" "a message"

Place only global variables, constants, and the call to a main function at the top level.

Variable Annotations

Use local for function‑scoped variables and readonly for constants. Example:

# a useful idiom: DEFAULT_VAL can be overwritten
#       with an environment variable of the same name
readonly DEFAULT_VAL=${DEFAULT_VAL:-7}

myfunc() {
  # initialize a local variable with the global default
  local some_var=${DEFAULT_VAL}
  ...
}

Convert a mutable variable to read‑only:

x=5
x=6
readonly x
x=7   # failure

Annotate all script variables with local or readonly whenever possible.

Modern Command Substitution

Prefer $(...) over backticks for readability and nesting.

# both commands below print out: A-B-C-D
echo "A-`echo B-\`echo C-\\\`echo D\\\`\`"
echo "A-$(echo B-$(echo C-$(echo D)))"

Double Brackets [[ ]] vs Single Brackets [ ]

Using [[ ... ]] avoids many pitfalls and adds new operators. Common operators: || – logical OR (inside [[ ]]) && – logical AND (inside [[ ]]) < – string comparison (no escaping needed) -lt – numeric comparison = – string equality == – globbing string comparison =~ – regex comparison -n – non‑empty string -z – empty string -eq – numeric equality -ne – numeric inequality

Example comparison:

[[ "${name}" > "a" && "${name}" < "m" ]]

Regular Expressions / Globbing

Examples demonstrating globbing and regex inside [[ ]]:

t="abc123"
[[ "$t" == abc* ]]   # true (globbing)
[[ "$t" == "abc*" ]] # false (literal)
[[ "$t" =~ [abc]+[123]+ ]] # true (regex)
[[ "$t" =~ "abc*" ]] # false (literal)

# Store a regex with spaces in a variable
r="a b+"
[[ "a bbb" =~ $r ]]   # true

Globbing can also be used in case statements:

case $t in
  abc*) <action> ;;
esac

String Operations

Common string manipulations:

f="path1/path2/file.ext"
len="${#f}"                # length = 20
slice1="${f:6}"            # "path2/file.ext"
slice2="${f:6:5}"        # "path2"
slice3="${f: -8}"          # "file.ext"
pos=6; len=5
slice4="${f:${pos}:${len}}" # "path2"

Pattern‑based replacement (globbing):

single_subst="${f/path?/x}"   # "x/path2/file.ext"
global_subst="${f//path?/x}" # "x/x/file.ext"

Delete prefixes or suffixes:

extension="${f#*.}"   # "ext"
filename="${f##*/}"    # "file.ext"
dirname="${f%/*}"   # "path1/path2"
root="${f%%/*}"       # "path1"

Avoid Temporary Files

Use process substitution <( ... ) when a command expects a filename:

# Compare two web pages without creating files
diff <(wget -O - url1) <(wget -O - url2)

Here‑documents allow multi‑line input:

command << MARKER
...
${var}
$(cmd)
MARKER

If no variable expansion is needed, quote the delimiter:

command << 'MARKER'
...
MARKER

Built‑in Variables

$0

– script name $n – nth argument $$ – script PID $! – PID of most recent background command $? – exit status of last command $# – number of arguments $@ – all arguments (preserves spacing) $* – all arguments as a single string (rarely useful)

Prefer $@ (quoted) over $* for correct handling of spaces.

Debugging

Syntax check: bash -n myscript.sh Verbose execution (print each command): bash -v myscript.sh Execution trace (show commands with arguments): bash -x myscript.sh You can set set -o verbose and set -o xtrace at the script’s top for permanent tracing, which is handy on remote machines.

When Not to Use Bash Scripts

Scripts become very long (hundreds of lines).

Complex data structures beyond arrays are needed.

Escaping becomes overly complicated.

Heavy string manipulation.

Little interaction with other programs or pipelines.

Performance concerns.

In such cases, consider a higher‑level scripting language such as Python or Ruby.

References

Advanced Bash‑Scripting Guide: http://tldp.org/LDP/abs/html/

Bash Reference Manual