4.1 Shell Navigation, Environment Variables, Redirection & Text Utilities

Key Takeaways

  • Bash login startup uses /etc/profile and the first readable user profile. Interactive non-login startup uses ~/.bashrc plus any system-specific setup.

  • Absolute paths begin at /. Relative paths resolve from the current directory. In Bash, an unquoted ~ can expand to a home directory.

  • Export variables that child programs need. To diagnose a prompt or command lookup, check PS1, PATH, and whether the shell is interactive.

  • File descriptors 0, 1 and 2 conventionally represent input, output and error. Redirection order matters when one descriptor is copied from another.

  • Use shell-appropriate history and text tools. NUL delimiters preserve filenames containing spaces and newlines when building command arguments.

Last updated: November 2026

4.1 Shell Navigation, Environment Variables, Redirection & Text Utilities

A mastery of GNU/Linux command-line shell mechanics, stream redirection, and text processing pipelines is essential for any enterprise systems administrator and represents a significant portion of Objective 1.5 on the CompTIA Linux+ (XK0-006) examination. Whether triaging multi-gigabyte application logs, automating system configuration through shell scripts, or managing input/output streams across distributed systems, Linux administrators must understand how the shell initializes, manages process environments, and routes data streams through modular utility pipelines.


1. Shell Environment & Configuration Files

Bash startup depends on two independent properties: whether Bash is a login shell and whether it is interactive. A terminal application can be configured to start either kind. In Bash, check shopt -q login_shell for login status and look for i in $- to verify interactivity.

Bash invocationTypical startup behavior
LoginReads /etc/profile, then the first existing readable file among ~/.bash_profile, ~/.bash_login, and ~/.profile, unless startup is suppressed. A selected profile can source other files.
Interactive non-loginReads ~/.bashrc unless suppressed. A system rc file can be loaded through distribution configuration or sourced by another file.
Noninteractive scriptBash can read the file named in BASH_ENV. A noninteractive login invocation or network-daemon invocation can follow additional startup rules. In particular Bash can read ~/.bashrc when it detects a remote shell connection.
LogoutAn interactive login shell can read ~/.bash_logout when it exits.

/etc/profile.d/ is a distribution convention, commonly processed by /etc/profile. Bash itself does not mandate that directory. For a value needed by both login and interactive non-login shells, use a small shared configuration file and source it from the startup files that actually run. Check the parent environment too: a child Bash can inherit an already exported value without reading the file that first set it. Keep startup output and interactive aliases out of automation paths unless intentionally needed.

Navigate using paths

An absolute path such as /var/log/messages begins at the filesystem root. A relative path is resolved from the current directory. In ../logs/app.log, .. reaches the parent, while ./check.sh explicitly names a file in the current directory. An unquoted ~ in cd ~/projects can expand to the home directory. Use pwd to verify where you are and cd - to return to the directory recorded in OLDPWD when it is set.

2. Environment Variables & Scope Management

An assignment sets a shell variable. Mark it with export when commands launched from the shell should receive it in their process environment. An already exported variable stays exported when reassigned. Bash subshells created with parentheses also copy shell state; compare that with running a new program such as bash -c.

Essential System Environment Variables

VariableDefinition and Operational Function
PATHColon-delimited list of directories searched by the shell when an un-pathed command is executed (e.g., /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin).
SHELLCommonly identifies the user’s preferred login shell. It does not determine which shell is executing the current command.
HOMENormally set to the user’s home directory, for example /home/student. A process can inherit or override it.
USER or LOGNAMECommon account-name variables. Use id for the actual process identity.
DISPLAYIdentifies an X display when set for an X client. It can be absent from a text or non-X session.
PWDCurrent working directory path (updated dynamically by the cd builtin).
OLDPWDPrevious working directory path (used by cd - to return to the prior directory).
PS1Primary prompt string defining the visual appearance of the interactive command prompt (e.g., [\u@\h \W]\$ ).
HISTSIZELimits how many entries are kept in Bash’s history list; check the configured value.
HISTFILEPath to the file where command history is written upon shell exit (default ~/.bash_history).
LANGDefault system locale controlling language, character encoding (e.g., en_US.UTF-8), and date formats.
LC_ALLAn overriding locale variable; when set to a nonempty value, it takes precedence over LANG and individual LC_* subcategories (LC_COLLATE, LC_TIME, LC_NUMERIC).

Viewing, Setting, and Unsetting Variables

# Start with an unexported variable for this exercise.
unset APP_ENV
APP_ENV="staging"

# Verify local variable is visible in current shell
echo "$APP_ENV"           # Output: staging
BASH_ENV= bash -c 'echo "$APP_ENV"' # Output: empty line

# Export the variable to the environment (inherited by child processes)
export APP_ENV="staging"
BASH_ENV= bash -c 'echo "$APP_ENV"' # Output: staging

# Alternative export syntax using separate declaration
PORT=8080
export PORT

# View all exported environment variables
env
printenv

# Query a specific environment variable
printenv PATH

# Display all shell variables (local and exported) as well as defined shell functions
set

# Remove a variable from the shell and environment entirely
unset APP_ENV

Key Distinction: Without other operands, env and printenv display the environment they receive. In contrast, set displays all local variables, environment variables, and shell functions.


3. I/O Redirection, Streams & Pipelines

Linux processes use integer file descriptors (FDs) to refer to open files, pipes and other I/O resources. A shell typically starts commands with descriptors 0, 1 and 2 inherited or redirected. They can already point to files, pipes, a terminal, or another destination.

The Standard File Descriptors

File DescriptorNameExample Terminal BindingCommon Linux Alias
0stdin (Standard Input)Keyboard/dev/stdin -> /proc/self/fd/0
1stdout (Standard Output)Terminal Screen/dev/stdout -> /proc/self/fd/1
2stderr (Standard Error)Terminal Screen/dev/stderr -> /proc/self/fd/2

Redirection Operators and Syntax

# Overwrite standard output (FD 1) to a file (creates if absent, truncates if present)
systemctl status nginx > /tmp/nginx_status.txt

# Append standard output (FD 1) to a file without truncating
echo "Deployment successful" >> deployment.log

# Overwrite standard error (FD 2) to a diagnostic file
find /etc -name "*.conf" 2> /tmp/find_errors.log

# Append standard error (FD 2) to a file
myscript.sh 2>> script_errors.log

# Redirect standard input (FD 0) from an existing file
wc -l < request.log

# A quoted delimiter suppresses shell expansion in the here-document body.
cat <<'END_SETTINGS' > lab-settings.txt
mode=test
literal=$HOME
END_SETTINGS

# Bash here-string (<<<): passes an expanded string followed by a newline
tr '[:lower:]' '[:upper:]' <<< "hello linux world"
# Output: HELLO LINUX WORLD

Merging Output Streams & The Black Hole (/dev/null)

Administrators frequently need to redirect both standard output and standard error into a single destination file or suppress unwanted terminal chatter:

# Traditional POSIX stream merging (Order of evaluation matters!)
# 1. Direct stdout (1) to output.log
# 2. Duplicate stderr (2) to wherever stdout (1) is currently pointing
command > output.log 2>&1

# CRITICAL PITFALL: Incorrect ordering
command 2>&1 > output.log
# Explanation: At the moment '2>&1' is evaluated, FD 1 still points to the screen.
# Therefore, stderr is sent to the screen, and then stdout is redirected to output.log!

# Modern Bash shorthand for combining stdout and stderr (overwrite)
command &> output.log

# Modern Bash shorthand for combining stdout and stderr (append)
command &>> output.log

# Discard stdout while appending only stderr to the audit log.
backup_script.sh >/dev/null 2>>backup-errors.log

# Discard standard error while retaining standard output on the console
find / -name "backup.tar.gz" 2> /dev/null

# Discard all output entirely (silent execution for cron jobs and background tasks)
rsync -av /src/ /dst/ > /dev/null 2>&1
# Modern syntax equivalent:
rsync -av /src/ /dst/ &> /dev/null

Pipelines (|) and Stream Multiplexing (tee)

A pipeline connects the standard output of one command to the standard input of the next through a pipe. With myscript.sh 2>&1 | tee audit.log, Bash connects standard output to the pipe first. The explicit redirect then sends standard error to that same pipe. The following examples use the standard output stream:

# Pipeline: Stream stdout of ps into grep stdin
ps aux | grep "[n]ginx"

# Stream Multiplexing with tee: Write stdout to a file while simultaneously echoing to stdout
ls -la /var/log | tee /tmp/directory_listing.txt

# Append output with tee (-a flag)
uptime | tee -a system_uptime.log

4. Text Manipulation & Stream Processing Utilities

The Unix philosophy emphasizes small, focused command-line utilities that perform a single task well and combine seamlessly through pipelines.

Viewing and Slicing Content

  • cat & tac: cat concatenates and displays files sequentially. tac concatenates and displays lines in reverse order (bottom-up), ideal for reviewing chronologically ordered log files from newest to oldest.
  • head: Displays the first lines of a file (head -n 20 /var/log/syslog displays the first 20 lines).
  • tail: Displays the last lines of a file (tail -n 50 /var/log/messages).
    • tail -f <file>: By default GNU tail follows the open descriptor. If the file is renamed, it can continue reading the old file while updates go to a replacement. GNU tail can notice truncation of the file it is following.
    • tail -F <file>: On GNU tail, equivalent to following by name and retrying when the path is unavailable.

Slicing Columns and Merging Files

  • cut: Extracts sections from each line of input based on delimiters:
    # Extract usernames (field 1) and login shells (field 7) from /etc/passwd using ':' delimiter
    cut -d: -f1,7 /etc/passwd
    
  • paste: Merges lines from corresponding files side-by-side:
    paste -d, users.txt departments.txt
    

Sorting and Deduplication

  • sort: Sorts lines of text according to specified rules:
    • -n: Numeric sort (evaluates numerical value rather than alphabetical ASCII ordering; e.g., 2 comes before 10).
    • -r: Reverses the result of comparisons (descending order).
    • -k <field>: Selects a sort key. For exactly field 2, use -k 2,2; -k 2 starts at field 2 and can extend to the end of the line.
    • -t <char>: Specifies the field delimiter (default is whitespace).
    • -u: Keeps one representative from each set of equal keys under the selected sort options.
  • uniq: Reports or omits adjacent repeated lines.

    CRITICAL RULE: uniq only detects duplicate lines that are immediately adjacent. Use sorting first when you must group equal lines that are separated in the input. Preserve the input order if adjacent runs are what you want to measure.

    # Extract IP addresses, sort them, count unique occurrences, and sort by frequency descending
    awk '{print $1}' /var/log/httpd/access_log | sort | uniq -c | sort -rn
    
    # Display ONLY lines that appear more than once (duplicate entries)
    sort names.txt | uniq -d
    
    # Display ONLY unique lines (lines that appear exactly once)
    sort names.txt | uniq -u
    

Counting and Character Translation

  • wc: Word, line, and byte counter:
    • wc -l: Counts newline characters; a final unterminated line does not add a newline to the count.
    • wc -w: Counts total words.
    • wc -c: Counts total bytes.
  • tr: Translates, squeezes, or deletes characters from standard input:
    # Convert all lowercase characters to uppercase using POSIX character classes
    cat input.txt | tr '[:lower:]' '[:upper:]'
    
    # Delete specified characters (-d flag), such as carriage return characters from Windows CRLF files
    tr -d '\r' < windows_script.sh > linux_script.sh
    
    # Squeeze repeated consecutive characters (-s flag), such as multiple spaces into a single space
    echo "System    status    OK" | tr -s ' '
    

Advanced Pattern Searching with grep

The grep (Global Regular Expression Print) utility searches text streams for matching patterns:

FlagDescription and Purpose
-iIgnore case distinctions in both pattern and input.
-vInvert the sense of matching; selects non-matching lines (excludes lines).
-r / -RBoth recurse. GNU -r follows command-line symlinks, while -R follows symlinks encountered during the traversal too.
-nPrefix each line of output with the 1-based line number within its input file.
-cSuppress normal output; print a count of matching lines for each input file.
-lSuppress normal output; print the name of each input file containing a match.
-EInterpret pattern as an Extended Regular Expression (ERE), equivalent to egrep (supports +, ?, |, ()).
-FSearch for fixed strings rather than regular expressions. Without another option such as -x, a substring can match.
# Find all active non-comment, non-blank lines in a configuration file
grep -Ev '^[[:space:]]*(#|$)' /etc/ssh/sshd_config

Stream Editing with sed

sed is a non-interactive stream editor that parses an input stream line-by-line, applies script transformations, and emits the modified stream to stdout.

  • Basic Substitution Syntax: sed 's/search_pattern/replacement/flags' filename
    • s: Substitution command.
    • g: Global flag (replaces all occurrences on a line, not just the first).
  • In-Place Modification (-i):
    # On GNU sed, edit a disposable config and save the previous file with a .bak suffix.
    sed -i.bak 's/^mode=test$/mode=staging/' lab-settings.txt
    
  • Line Deletion (/d):
    # Delete lines matching a pattern
    sed '/DEBUG/d' /var/log/application.log
    
    # Delete a specific range of lines (e.g., delete lines 1 through 5)
    sed '1,5d' input.txt
    

Text Processing & Data Extraction with awk

awk is a complete pattern-scanning and field-processing programming language. By default, awk treats input lines as records and splits fields at runs of blanks, ignoring leading and trailing blanks. Set the separator when the input has a defined delimiter.

  • Special Variables:
    • $0: The entire current line/record.
    • $1, $2, ... $NF: The first, second, through last field on the line.
    • NF: Number of Fields in the current record.
    • NR: Number of Records (the current line number processed).
    • FS: Field Separator (default is whitespace; set via -F).
    • OFS: Output Field Separator (default is space).
# Print users with UID >= 1000 from /etc/passwd
awk -F: '$3 >= 1000 {print "User: " $1 " | Home: " $6 " | Shell: " $7}' /etc/passwd

# Sum the first field in a documented one-row-per-request report.
awk '{sum += $1} END {print "Total bytes:", sum+0}' request-bytes.txt

Safe Pipeline Execution with xargs

xargs builds argument lists from standard input. Its default parsing treats blanks, newlines, quotes and backslashes specially. Use NUL-separated names when files can contain spaces or newlines. This GNU example is read-only:

find ./lab-data -type f -name '*.log' -print0 | xargs -0 -r wc -c --

Each pathname reaches wc as one argument. GNU -r skips execution on empty input, and -- ends option parsing for wc. If a batch contains several files, wc may also print a total for that batch. NUL framing addresses argument parsing; it does not authorize removal of files or prevent races in a shared writable tree.

History, aliases, editors and small tools

Use history in an interactive Bash shell to inspect recent commands. With Bash history expansion enabled, !! expands to the previous history entry, and !number can select a numbered entry. Inspect the expansion or edit the recalled line instead of blindly repeating it, especially if it changes files. An alias such as alias ll='ls -alF' defines a shortcut in the current shell. A command such as source ./lab-env.sh or . ./lab-env.sh runs trusted shell commands in the current shell, so variable changes can persist in that shell.

  • echo is useful for simple messages. Use printf '%s\n' "$value" with a fixed format when printing arbitrary text.

  • Run uname -r to see the running kernel release. Use printf '%s\n' 'scale=2; 7/2' | bc for decimal arithmetic with an explicit scale.

  • Use less or more to read a long text file one screen at a time. A pager is useful when the task is inspection rather than editing.

  • In Vim, use normal mode for movement, i for insertion, Esc to return, and :wq to write and quit. Nano displays common shortcuts, including Ctrl+O to write and Ctrl+X to exit.

  • Bash manual, GNU find and GNU xargs

  • GNU grep, GNU sed and GNU awk

Test Your Knowledge

A new terminal launches Bash interactively with login mode disabled. Its parent did not export APP_ENV, and only ~/.bash_profile sets it. Why is APP_ENV missing, and what change works for this terminal?

A

The shell reads only /etc/profile, so put it there

B

The shell normally reads ~/.bashrc rather than ~/.bash_profile, so set it there or source a shared file from the relevant startup files

C

Bash was noninteractive because it was opened by a terminal

D

Bash disables all environment variables in interactive mode

Test Your Knowledge

During an interactive test, backup_script.sh 2>&1 >/dev/null leaves errors on the terminal. The intended result is to discard stdout and append stderr to /tmp/backup-errors.log. Which command does that?

A

backup_script.sh >/dev/null 2>&1

B

backup_script.sh 2>>/dev/null >/tmp/backup-errors.log

C

backup_script.sh &>>/tmp/backup-errors.log

D

backup_script.sh >/dev/null 2>>/tmp/backup-errors.log

Test Your Knowledge

On a GNU/Linux lab host, which pipeline passes each matching filename intact to wc, even when names contain spaces or newline characters?

A

find ./lab-data -type f -name "*.log" -print0 | xargs -0 -r wc -c --

B

find ./lab-data -type f -name "*.log" | xargs -I {} wc -c "{}"

C

wc -c $(find ./lab-data -type f -name "*.log")

D

ls ./lab-data | tr "\n" " " | xargs wc -c

Sections you finish are checked off in the contents.