5.5 Stream Editing with sed (103.2)

5.5 Stream Editing with sed (103.2)

Quick Summary: The GNU sed (Stream Editor) utility performs automated, non-interactive text transformations on input streams or files. By reading input line-by-line into its internal Pattern Space, applying specified addressing rules and commands (such as substitution s, deletion d, print p, append a, and insert i), and writing results to standard output, sed provides powerful batch editing capabilities essential for Topic 103.2.

1. sed Architecture & Execution Model

Unlike interactive text editors (vi or nano), sed is a stream editor that processes data sequentially without user interaction.

Internal sed Execution Cycle for Each Input Line:
┌─────────────────────────────────────────────────────────────────────────┐
│ Input Stream (File or Stdin)                                           │
└────────────────────────────────────┬────────────────────────────────────┘
                                     │ 1. Read next line & strip trailing \\n
                                     ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ PATTERN SPACE (Primary Working Buffer in RAM)                           │
│ • Current line resides here during processing                           │
│ • sed commands execute sequentially on this buffer                      │
│   (e.g., s/foo/bar/, /error/d, 10p)                                     │
└───────────────────▲────────────────┬───────────────────▲────────────────┘
     h / H (Copy/   │                │ g / G (Copy/      │ x (Exchange)
     Append to Hold)│                │ Append from Hold) │
                    ▼                │                   ▼
┌────────────────────────────────────┼────────────────────────────────────┐
│ HOLD SPACE                         │                                    │
│ (Secondary Persistent Scratchpad)  │                                    │
└────────────────────────────────────┘                                    │
                                     │ 2. Automatic Output (Unless -n set)│
                                     ▼                                    │
┌─────────────────────────────────────────────────────────────────────────┴┐
│ Standard Output (stdout)                                                │
└─────────────────────────────────────────────────────────────────────────┘

The Standard sed Processing Cycle

1. Read: Reads a line from the input stream and strips the trailing newline character.
2. Pattern Space Load: Places the line into the temporary working memory called Pattern Space.
3. Execute: Evaluates all sed instructions in order against the Pattern Space. If an address filter matches, the associated command runs.
4. Output: Unless automatic printing is disabled (via the -n option), sed writes the contents of the Pattern Space to standard output, followed by a newline.
5. Flush & Repeat: Clears the Pattern Space and repeats the cycle for the next line until End-Of-File (EOF).

Essential sed Command-Line Options

⚠️ LPIC-1 Trap: On GNU/Linux systems, sed -i 's/old/new/' file edits the file directly with no backup. If an argument is attached immediately to -i (e.g., sed -i.orig 's/old/new/' file), sed saves the unmodified original as file.orig.

2. Addressing Mechanisms in sed

By default, sed commands apply globally to every line of input. Addresses restrict command execution to specific lines, ranges, or regular expression matches.

Addressing Paradigms

# Deleting empty lines and lines starting with '#' comments:
$ sed -e '/^#/d' -e '/^[[:space:]]*$/d' /etc/nginx/nginx.conf

# Printing only the block between '<VirtualHost>' and '</VirtualHost>':
$ sed -n '/<VirtualHost>/,/</VirtualHost>/p' /etc/httpd/conf/httpd.conf

3. The Substitution Command (s)

The substitution command is the most frequently tested sed operation on LPIC-1. $\text{sed '[address]s/pattern/replacement/flags' [file]}$

Substitution Flags

• g (Global): Replaces all non-overlapping occurrences of the pattern on the line (without g, only the first occurrence on each line is replaced).
• p (Print): Prints the Pattern Space if a successful substitution occurred (almost always paired with -n).
• i / I (Ignore Case): Case-insensitive regular expression matching.
• N (Numeric Instance): Replaces only the N-th occurrence of the pattern on the line (e.g., s/foo/bar/2 replaces only the second foo).
• w <file> (Write): Writes the modified line directly to the specified file.

Custom Delimiters (Preventing Leaning Toothpick Syndrome)

While the forward slash (/) is standard, sed allows any single character to serve as the delimiter for the s command. This is critical when manipulating file paths or URLs:

# Standard slash delimiter requires ugly escaping:
$ sed 's/\\/var\\/www\\/html/\\/srv\\/www\\/public/g' config.txt

# Using '#' or '|' as custom delimiters:
$ sed 's#/var/www/html#/srv/www/public#g' config.txt
$ sed 's|https://example.com|https://secure.example.com|g' links.txt

Backreferences (\\1–\\9) and Matched Pattern (&)

• &: Represents the entire string that matched the search regular expression.
• \\1, \\2, ... \\9: Represents sub-expressions captured inside parentheses (\\( ... \\) in Basic Regular Expressions or ( ... ) with -E / -r).

# Wrapping all numbers in square brackets using '&':
$ echo "Port 8080 and Port 443" | sed 's/[0-9]\\+/[&]/g'
Port [8080] and Port [443]

# Swapping two words using capture groups and backreferences:
$ echo "Doe, John" | sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\\2 \\1/'
John Doe

# Extracting IP addresses from ifconfig output:
$ ip -4 addr show eth0 | sed -n -E 's/.*inet ([0-9.]+).*/\\1/p'
192.168.1.50

4. Other Core sed Commands

# Using 'q' (Quit) as a high-performance alternative to head on multi-gigabyte files:
# Unlike 'head', 'sed 5q' exits immediately after line 5 without reading the remainder of the stream:
$ sed '5q' /var/log/messages

💡 LPIC-1 Exam Fill-in-the-Blank Alert: When asked for the sed option that suppresses the automatic printing of pattern space so that only lines explicitly matching a p command are displayed, the answer is -n (or --quiet / --silent). When asked for the option to edit files directly in place, the answer is -i.