3.2a Pipes, Redirection, and Basic Regular Expressions

Connecting commands and shaping I/O (objective 3.2)

Objective 3.2 is about searching and extracting data. Before the individual tools (grep, sort, cut, and friends), you need the plumbing: pipes, redirection, and a little regular expression vocabulary so patterns mean what you intend.

---

Pipes: |

A pipe connects two programs. The standard output (stdout) of the left command becomes the standard input (stdin) of the right command — no temporary file required.

ps aux | grep nginx
dmesg | grep -i usb
cat access.log | grep " 404 "

Read pipelines left to right: “generate text, then filter or transform it.” You can chain several stages:

grep "ERROR" app.log | sort | uniq -c

That pattern — produce → filter → refine — is the heart of day-to-day Linux text work and a frequent exam theme.

Pipes carry stdout, not stderr, unless you merge streams first (see 2>&1 below).

Pipe versus redirect: grep error app.log | wc -l counts on the fly; grep error app.log > hits.txt saves matches to a file. Pick | for another command, >/>> for a file.

---

Redirection operators

Every process has three standard streams:

Writing and appending: > and >>

echo "first" > notes.txt      # creates or truncates notes.txt
echo "second" >> notes.txt    # adds a second line
ls /etc > listing.txt         # save directory listing

Danger: > truncates immediately. command > important.txt with a typo can wipe important.txt before you notice. Prefer >> when you mean “add to the log.”

Reading input: <

sort < names.txt
wc -l < app.log

< feeds a file into stdin. Many commands also accept filenames as arguments (sort names.txt).

Errors: 2> and merging with 2>&1

ls /exists /missing > out.txt 2> err.txt
# out.txt gets the successful listing; err.txt gets the error for /missing

ls /missing > all.txt 2>&1
# both stdout and stderr land in all.txt

Order tip: write > file 2>&1 (redirect stdout to file, then point stderr at wherever stdout now goes). The reverse order 2>&1 > file first merges stderr to the terminal’s stdout, then only redirects stdout — a classic fill-in trap.

Worked silence-and-pipe: find /var/log -name "*.log" 2>/dev/null | head discards permission errors on stderr so only successful path names flow into head. /dev/null is the “bit bucket.”

Basic regular expressions

Linux Essentials expects awareness of basic regex atoms used with tools like grep. A regular expression is a pattern, not always a literal string.

grep "err.r" app.log
grep "[0-9][0-9][0-9]" codes.txt     # three digits in a row (BRE style)
grep -E "colou?r" words.txt            # color or colour with ERE ?

Shell globs vs regex: In the shell, * means “any string” in filenames (ls *.log). Inside grep’s default basic regular expressions, * repeats the previous piece. Quoting protects your intent:

grep "file.txt" notes          # . means any character — matches fileXtxt too
grep "file\.txt" notes         # escaped dot = literal period

When a pattern has spaces or shell-special characters, quote it so the shell does not expand globs or split words before grep runs:

grep "connection refused" /var/log/syslog

Tiny practice patterns

You do not need advanced Perl-style regex for 010-160. Own ., [ ], *, and when ? is optional-atom (ERE) versus a shell wildcard.

Fill-in traps: wrong merge order (2>&1 > file vs > file 2>&1); using > when the story says append; unquoted * so the shell expands names before grep.

Putting it together

cut -d: -f1 /etc/passwd | sort | uniq > users.txt
cmd_that_warns 2>&1 | grep -i warn

Exam mindset

• | connects stdout → stdin between commands.
• > overwrites; >> appends; < supplies stdin from a file.
• 2> is stderr; 2>&1 merges stderr into stdout (watch operator order).
• Regex . [ ] * (and ? in the right mode) change matching — quote and escape when you mean literals.

With that plumbing solid, the extract tools in the next section become short, readable pipelines instead of one-off memorization.