6.3 Process Monitoring: ps, top, free, watch, pgrep, pstree & uptime (103.5)

6.3 Process Monitoring: ps, top, pgrep, pstree, uptime

Quick Summary: In Linux, every running program, background daemon, and worker thread operates as a process tracked by the kernel. Administrators inspect system performance, resource utilization, and application health using a standard diagnostic toolkit: ps provides static process snapshots using UNIX (ps -ef) or BSD (ps aux) options; top provides a dynamic real-time resource monitor; pgrep queries processes by name or command-line regex; pstree diagrams process hierarchy trees rooted at PID 1; and uptime assesses multi-core CPU load averages over 1, 5, and 15-minute intervals.

---

1. Process Lifecycle, Identifiers & Process States

A process is an instance of an executing program in memory with its own allocated virtual address space, file descriptor table, and security context.

Core Process Identifiers

• PID (Process ID): A unique unsigned integer assigned by the kernel to identify an active process. The first user-space process created during boot (systemd or SysV init) always receives PID 1.
• PPID (Parent Process ID): The PID of the process that spawned this child process via the fork() system call. When a parent process terminates before its children, the orphaned child processes are re-parented to PID 1.
• UID / GID (User & Group IDs): The real and effective user/group credentials governing the process's filesystem and syscall permissions.
• TTY: The controlling terminal device (e.g., pts/0, tty1, or ? for background daemons with no controlling terminal).

The Process State Model (STAT / S Column)

In ps and top output, the kernel reports the operational state of every process using single-letter codes:

Process State Modifiers

In BSD-style ps aux output, additional characters append to the primary state code:

• <: High priority (has a negative niceness value, taking more CPU time).
• N: Low priority (has a positive niceness value, "nice" to other processes).
• s: Session leader (the process created the terminal session, e.g., a login shell).
• l: Multi-threaded process (cloned using CLONE_THREAD).
• +: Member of the foreground process group in its controlling terminal.

Example STAT Strings in `ps aux`:
Ss   --> Session leader in interruptible sleep (e.g., bash login shell)
R+   --> Process actively executing in the foreground (e.g., top, grep)
S<l  --> Multi-threaded process with elevated real-time priority (e.g., audio server)
Z    --> Zombie process awaiting parent collection

---

2. Snapshot Inspection with ps (UNIX vs. BSD Styles)

The ps (Process Status) utility dumps a static snapshot of active processes. The tool supports three distinct syntax conventions:

1. UNIX (POSIX) Options: Prefixed with a single dash - (e.g., ps -ef, ps -e).
2. BSD Options: Used without any dashes (e.g., ps aux, ps ax).
3. GNU Long Options: Prefixed with two dashes -- (e.g., ps --sort=-%cpu).

# 1. UNIX Standard Full Listing: ps -ef
$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 08:00 ?        00:00:02 /sbin/init
root       842     1  0 08:01 ?        00:00:00 /usr/sbin/sshd -D
admin     2415   842  0 09:15 pts/0    00:00:00 -bash
admin     3102  2415  0 10:22 pts/0    00:00:00 ps -ef

# 2. BSD User-Oriented Listing: ps aux
$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1 168432 13420 ?        Ss   08:00   0:02 /sbin/init
root       842  0.0  0.0  15812  7104 ?        Ss   08:01   0:00 /usr/sbin/sshd -D
admin     2415  0.0  0.0  10856  5120 pts/0    Ss   09:15   0:00 -bash
admin     3105  0.0  0.0  11492  3480 pts/0    R+   10:22   0:00 ps aux

Comparing Output Columns: UNIX vs. BSD

Advanced ps Querying & Sorting Examples

# Sort processes by memory consumption in descending order (highest first):
$ ps aux --sort=-%mem | head -n 10

# Sort processes by CPU consumption in descending order:
$ ps aux --sort=-%cpu | head -n 10

# Custom output formatting showing only PID, PPID, User, and Command:
$ ps -eo pid,ppid,user,stat,comm

# Filter processes for a specific user:
$ ps -u admin -f

---

3. Dynamic Real-Time Monitoring with top

The top program provides an interactive, real-time diagnostic dashboard that updates every 3.0 seconds by default.

top - 10:30:15 up 2 days,  3:14,  2 users,  load average: 0.15, 0.08, 0.02
Tasks: 185 total,   1 running, 184 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.3 us,  1.0 sy,  0.0 ni, 96.2 id,  0.3 wa,  0.1 hi,  0.1 si,  0.0 st
MiB Mem :   7952.4 total,   2145.1 free,   3210.8 used,   2596.5 buff/cache
MiB Swap:   2048.0 total,   2048.0 free,      0.0 used.   4421.2 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1420 admin     20   0  852412 184210  45120 S   4.5   2.3   1:12.45 node
  842 root      20   0   15812   7104   5980 S   0.0   0.1   0:00.12 sshd

Top Header Metrics Breakdown

1. System Uptime & Load Average (Line 1): Current system clock, elapsed uptime, logged-in user count, and exponential moving load averages over 1, 5, and 15 minutes.
2. Tasks Summary (Line 2): Breakdown of total system processes across running, sleeping, stopped, and zombie states.
3. CPU Utilization Breakdown (Line 3):
   • us (User): Time spent running un-niced user-space processes.
   • sy (System): Time spent running kernel-space system routines and syscalls.
   • ni (Nice): Time spent running niced (low-priority) user processes.
   • id (Idle): Percentage of time CPU is idle and unutilized.
   • wa (I/O Wait): Time CPU spent waiting for disk/storage I/O completion. High wa indicates disk bottlenecks.
   • hi / si (Hardware / Software IRQ): Time servicing hardware and software interrupts.
   • st (Steal Time): CPU cycles stolen by the hypervisor in virtualized environments.
4. Memory Lines (Lines 4 & 5): Physical RAM and Swap breakdown showing Total, Free, Used, Buffers/Cache, and avail Mem (memory available for starting new applications without swapping).

Interactive Keystrokes in top

---

4. Targeted Process Querying with pgrep

The pgrep utility looks through the currently running processes and lists the process IDs (PIDs) matching specified selection criteria (e.g., regex patterns, usernames):

# List PIDs of all processes named 'nginx'
$ pgrep nginx
1042
1043
1044

Key pgrep Flags for LPIC-1

# Find processes matching 'python' and print full arguments:
$ pgrep -a python
4120 /usr/bin/python3 /opt/app/worker.py --threads=4

# Count how many worker processes are running for user www-data:
$ pgrep -u www-data -c
8

---

5. Visualizing Process Trees with pstree & System Load with uptime

pstree

pstree displays the running processes as a tree structure, visually showing parent-child inheritance hierarchies rooted at PID 1 (systemd or init):

$ pstree -p
systemd(1)─┬─cron(820)
           ├─sshd(842)───sshd(2410)───bash(2415)───pstree(3201)
           ├─systemd-journal(412)
           └─systemd-udevd(450)

Essential pstree Flags

• -p: Displays Process IDs (PIDs) in parentheses next to each process name.
• -u: Shows user transitions when a child process runs under a different UID than its parent.
• -a: Displays command-line arguments for each process.
• -h: Highlights the current process and its ancestors.

uptime and System Load Averages

The uptime utility outputs a single line summarizing system availability:

$ uptime
 10:45:02 up 14 days,  2:30,  3 users,  load average: 1.25, 0.85, 0.40

Interpreting System Load Average

The three numbers represent the average number of processes in a Runnable (R) or Uninterruptible Disk Sleep (D) state over the past 1, 5, and 15 minutes.

• On a single-core system, a load average of 1.0 means the CPU is at 100% capacity.
• On a 4-core system, a load average of 4.0 represents 100% saturation. A load average of 1.25 indicates that the 4-core system is running at roughly 1.25 / 4 = 31.25% utilization.

---

6. Memory Snapshots with free

free is listed in the 103.5 Terms and Utilities alongside ps, top and uptime, and it answers the one question ps cannot: how much memory does the system as a whole have left? It reads /proc/meminfo and prints a two-line (or three-line, with swap) summary.

$ free -h
               total        used        free      shared  buff/cache   available
Mem:            7.8Gi       3.1Gi       2.1Gi       210Mi       2.5Gi       4.3Gi
Swap:           2.0Gi          0B       2.0Gi

Exam Trap — "free" Looks Alarmingly Low: A healthy long-running Linux server shows very little in the free column, because the kernel uses every otherwise-idle page for the disk cache. That is correct behaviour, not a leak. Read available, not free, to judge memory pressure. Genuine exhaustion shows up as growing Swap: used plus rising si/so columns in vmstat.

---

7. Repeating a Command with watch

watch also appears in the 103.5 Terms and Utilities. It re-runs a command on a fixed interval and redraws its output in place, giving you a live view of any tool that only produces a snapshot — free, ps, df, lsblk, swapon, or a file in /proc.

# Default interval is 2 seconds
$ watch free -h

# Refresh every 5 seconds
$ watch -n 5 'ps -eo pid,comm,%cpu --sort=-%cpu | head -10'

# Highlight what changed since the previous refresh
$ watch -d -n 1 'cat /proc/loadavg'

# Exit automatically as soon as the command's output changes
$ watch -g lsblk

Exam Rule: Quote any pipeline you hand to watch. Without quotes, the shell applies the pipe to watch itself — watch ps | head pipes watch's output into head instead of repeating the whole pipeline. Press Ctrl+C to stop; watch runs until interrupted unless -g or -e is given.