Mastering sed: Powerful Text Manipulation and Automation Techniques

sed

sed operates on lines and can modify file contents without opening the file.

Basic syntax:

sed [options] ... {script} filename

Common Options

-i        # edit files in place
-i.bak    # edit in place after creating a .bak backup
-n        # suppress automatic printing
-r        # use extended regular expressions
-p        # print selected lines
-q        # quit early (e.g., 3q prints up to line 3 then exits)
-d        # delete lines (e.g., 3d deletes line 3)
-a        # append after a line (e.g., 3a text)
-i        # insert before a line (e.g., 3i text)
-c        # replace a line (e.g., 3c new text)
=        # print line numbers

Extracting Odd and Even Lines

Use range and command combinations to print odd or even lines, or to print lines between two patterns (e.g., lines between a line starting with b and a line starting with f).

Search and Replace

Standard form: s/old/new/flags. Alternative delimiters such as # or @ can be used.

g   # global replacement on the line
p   # print lines where substitution succeeded
w   # write substituted lines to a file
i,I # ignore case

Replacement Exercise

Replace SELINUX=enforcing with SELINUX=enable. Remember to back up the original file, e.g.,

sed -i.bak 's/SELINUX=enforcing/SELINUX=enable/' /etc/selinux/config

.

Group References

Only extended regular expressions support grouping. Example syntax uses parentheses to capture groups and back‑references.

Practice 1 – Extract IP Address

Use a pattern like .*inet\s+([^ ]+) to capture the IP address after inet.

Practice 2 – Extract Version Numbers

Split on - and capture the numeric part before .jar.

Practice 3 – Extract File Permission Digits

Capture the four‑digit permission string from the fourth line using a suitable regex.

Non‑Interactive Input

Multi‑line redirection example:

cat > target_file <<EOF
...multiple lines...
EOF

expect

Use expect to automate interactive shell scripts. Install with yum install expect -y. Key commands include:

spawn      # start a process
expect     # wait for a pattern
exp_continue # continue waiting after a match
send       # send input to the process
interact   # hand control back to the user
set        # define variables

Note: expect scripts must start with #!/usr/bin/expect and cannot be run with a Bash interpreter.