10.1 Locate and Interpret System Log Files and Journals

10.1 Locate and Interpret System Log Files and Journals

Quick Answer: On RHEL 10, start with journalctl for systemd unit and boot diagnostics, and know key files under /var/log for classic text trails. Filter by unit (-u), priority (-p), boot (-b), and time (--since / --until). Escalate with sudo. Interpret timestamps, priorities, and unit names, then fix the failing service or config the exam requires.

Why logging is an EX200 skill

Under Operate running systems, Red Hat expects you to locate and interpret system log files and journals. Typical lab pressure looks like:

• A service will not start—find the error and correct it.
• Login or SSH fails after a config change—trace auth messages.
• After reboot, something is degraded—inspect the current boot journal.
• A scheduled job “did nothing”—check cron and unit logs.

You are not graded on printing every log line. You are graded on using logs to diagnose and leave the system correct. Speed comes from knowing where to look and how to filter noise.

Two log worlds on RHEL 10

Modern RHEL funnels most service output into the journal. Many environments also forward selected messages to /var/log/messages and /var/log/secure via rsyslog. For EX200, be fluent in both: journalctl first for units, /var/log when a task or habit points there.

# High-value starting points
sudo journalctl -xe
sudo journalctl -u sshd -b
sudo tail -n 50 /var/log/messages
sudo tail -n 50 /var/log/secure

journalctl essentials

Current boot and previous boots

journalctl                 # all accessible entries (can be huge)
journalctl -b              # current boot only
journalctl -b -1           # previous boot
journalctl -b -2           # two boots ago
journalctl --list-boots    # boot IDs and timestamps

Exam tip: After a reboot that “broke” a change, compare journalctl -b (this boot) with journalctl -b -1 (last boot) for the same unit.

Unit and service focus

journalctl -u sshd.service
journalctl -u sshd                 # .service suffix often optional
journalctl -u NetworkManager -u firewalld
journalctl -u httpd -b --no-pager

systemctl status UNIT already shows a short journal excerpt. When that is not enough, jump straight to journalctl -u UNIT -b -e (jump toward end) or -xe style investigation.

systemctl status sshd -l --no-pager
journalctl -u sshd -b -p err..alert --no-pager

Priority (severity) filters

Priorities from most to least severe (syslog-style):

journalctl -p err                 # err and worse (0–3)
journalctl -p warning             # warning and worse
journalctl -p err -b -u sshd
journalctl -p 3                   # numeric form

When hunting a failed start, -p err (or warning) cuts chatter so the real failure surfaces.

Time windows

journalctl --since "2026-08-05 09:00:00"
journalctl --since "10 min ago"
journalctl --since today
journalctl --since yesterday --until "1 hour ago"
journalctl -u cron --since "2026-08-05" --until "2026-08-06"

Time filters pair well with unit filters when a task says “after you restarted X.”

Follow, reverse, pager control

journalctl -f                     # follow (like tail -f)
journalctl -u sshd -f
journalctl -r                     # reverse (newest first)
journalctl -n 50                  # last 50 lines
journalctl --no-pager             # scriptable / full dump to terminal
journalctl -o verbose             # all fields
journalctl -o json-pretty         # machine-oriented (rare on exam)

Under time pressure prefer --no-pager or -n so you are not stuck inside less.

Field matches (precise filtering)

Journal entries carry fields. Useful examples:

journalctl _SYSTEMD_UNIT=sshd.service
journalctl _COMM=sshd
journalctl _PID=1234
journalctl _UID=0
journalctl PRIORITY=3
journalctl SYSLOG_IDENTIFIER=sudo

Kernel messages:

journalctl -k                     # kernel only (like dmesg via journal)
journalctl -k -b
dmesg --ctime | tail              # still valid; journal -k is integrated

Catalog and explain helpers

journalctl -x                     # add explanation catalogs when available
journalctl -xe                    # jump to end + explanations (classic “what just failed?”)

-xe is the reflex after a failed systemctl start when status is unclear.

Classic paths under /var/log

Know these files by purpose:

ls -la /var/log
sudo less /var/log/secure
sudo grep -i failed /var/log/secure | tail
sudo tail -f /var/log/messages

Rotation: Files may be secure, secure-20260801, or messages-.... Use zgrep on rotated .gz archives if you must search history.

sudo zgrep -i "Failed password" /var/log/secure*

Mapping problems to log sources

SELinux is a full later domain; for this section, recognize that “permission denied” in an app log may be SELinux, not only DAC—confirm with audit logs when context fits.

Correlating systemctl and journals

systemctl --failed
systemctl status firewalld -l --no-pager
journalctl -u firewalld -b --no-pager | tail -n 80

Active: failed lines often include a pointer into the journal. Read the first real error, not only the last “Failed to start” summary. Common patterns:

• ExecStart binary missing or wrong path
• Config syntax error (sshd, nginx, named)
• Dependency/order failure (network not ready)
• Permission / SELinux / port bind conflict (Address already in use)

Reading a log line (interpretation)

A journal-style line typically includes:

1. Timestamp — when it happened (timezone is system local unless you force UTC).
2. Host — hostname.
3. Identifier / unit — which program.
4. Message — human-readable event.
5. Priority — implicit in filtering; verbose mode shows fields.

Interpretation checklist:

1. Is this the current boot?
2. Is the unit the one the task cares about?
3. Is the message an error or harmless noise?
4. Does it name a file, port, or user you can fix?
5. After a fix, does a new log line show success?

sudo systemctl restart sshd
sudo journalctl -u sshd -n 20 --no-pager

Permissions and disk location awareness

• Unprivileged users often see only their own journal slice; system logs need root.
• Journal storage may be volatile (/run/log/journal) or persistent (/var/log/journal)—preserving journals is Section 10.2; here, if journalctl -b -1 is empty on a fresh default, persistence may be off.
• Never “fix” a boot issue by deleting all of /var/log on the exam.

ls /run/log/journal 2>/dev/null
ls /var/log/journal 2>/dev/null

Exam workflows

Workflow A — Service will not start

sudo systemctl start httpd
systemctl is-active httpd
sudo systemctl status httpd -l --no-pager
sudo journalctl -u httpd -b -p err..warning --no-pager
# Fix config or modules named in the error, then:
sudo systemctl start httpd
systemctl is-active httpd

Workflow B — Auth failure after SSH hardening

sudo journalctl -u sshd -b --no-pager | tail -n 50
sudo tail -n 50 /var/log/secure
# Look for: bad ownership on keys, PermitRootLogin, PasswordAuthentication, SELinux context on ~/.ssh

Workflow C — What failed this boot?

systemctl --failed
sudo journalctl -p err -b --no-pager
sudo journalctl -b -p err -o short-iso --no-pager | head -n 100

Workflow D — Follow live while testing

# Terminal 1
sudo journalctl -u sshd -f
# Terminal 2
ssh user@localhost
# Watch failure reason appear immediately

Common traps

1. Scrolling the entire unfiltered journal — always constrain with -b, -u, -p, or --since.
2. Reading only /var/log/messages when the unit only logged to the journal — use journalctl -u.
3. Ignoring “failed” units after reboot — systemctl --failed is a map.
4. Acting on an old boot’s errors — confirm -b current.
5. Missing sudo and concluding “no logs exist.”
6. Treating every WARNING as fatal — read severity and whether the unit is actually active.
7. Forgetting --no-pager and wasting time in less under the clock.

Persistence of diagnosis vs persistence of config

Logs tell you what happened. Enabling a service, fixing a unit file, or correcting sshd_config is what survives reboot. After fixes, re-check logs and systemctl is-active / is-enabled as the task requires.

Section checkpoint

You should open the journal with journalctl, filter by boot, unit, priority, and time, know landmark files under /var/log (especially messages and secure), correlate with systemctl status and --failed, interpret error lines into concrete fixes, and verify success with a short post-fix log sample. That is the EX200 standard for locating and interpreting system logs and journals.