4.1 Process Management & System Logging

Process Management & System Logging

In modern multi-tasking operating systems, CPU execution time, system memory, and peripheral hardware are constantly shared among hundreds of concurrently running programs and background services. An IT support specialist or systems administrator must understand how operating systems instantiate, schedule, prioritize, and terminate execution units. When applications hang, memory leaks occur, or servers crash, IT professionals rely on process management tools and centralized system event logs to isolate root causes and restore normal operations.

---

1. Process vs. Thread Fundamentals & Architecture

To manage program execution, operating systems abstract hardware execution into Processes and Threads.

+-----------------------------------------------------------------------------+
|                        PROCESS VS. THREAD MEMORY MODEL                      |
|                                                                             |
|   PROCESS (PID 2450) - Isolated Virtual Address Space                       |
|   +---------------------------------------------------------------------+   |
|   |  Text Segment (Compiled Machine Code Instructions)                  |   |
|   |  Data Segment (Global & Static Variables)                           |   |
|   |  Heap Segment (Dynamically Allocated Memory: malloc / new)          |   |
|   |  File Descriptor Table (Open Files, Sockets, Pipes)                 |   |
|   +---------------------------------------------------------------------+   |
|   |  CONCURRENT EXECUTION THREADS (Shared Heap, Code, and Global Data)  |   |
|   |                                                                     |   |
|   |  +-------------------+  +-------------------+  +-----------------+  |   |
|   |  | THREAD 1 (TID 1)  |  | THREAD 2 (TID 2)  |  | THREAD 3 (TID 3)|  |   |
|   |  | - Stack (Locals)  |  | - Stack (Locals)  |  | - Stack (Locals)|  |   |
|   |  | - Program Counter |  | - Program Counter |  | - PC / Registers|  |   |
|   |  | - CPU Registers   |  | - CPU Registers   |  | - CPU Registers |  |   |
|   |  +-------------------+  +-------------------+  +-----------------+  |   |
|   +---------------------------------------------------------------------+   |
+-----------------------------------------------------------------------------+

The Process: Isolated Execution Container

A Process is an active instance of a computer program loaded into system memory for execution. Each process is allocated an isolated, protected Virtual Address Space by the operating system kernel and Memory Management Unit (MMU):

• Process Control Block (PCB): The kernel data structure tracking process state, Process ID (PID), Parent Process ID (PPID), CPU register contents, memory page tables, open file descriptors, and security tokens.
• Memory Isolation: A process cannot read or write to the memory space of another process without explicit Inter-Process Communication (IPC) mechanisms (such as shared memory, pipes, or Unix domain sockets). If process A crashes, process B remains completely unaffected.
• Heavyweight Overhead: Creating a new process requires cloning or creating new memory mappings, page tables, and file handles, resulting in measurable CPU and RAM context-switching overhead.

The Thread: Lightweight Unit of Execution

A Thread (often termed a lightweight process) is the smallest schedulable unit of CPU execution within a parent process:

• Shared Resources: All threads within a single parent process share the same virtual address space, heap memory, global variables, and open file descriptors.
• Private Context: Each thread maintains its own Thread Control Block (TCB), dedicated stack (for local variables and function calls), CPU registers, and Program Counter (PC).
• Multithreading Efficiency: Threads allow a program to perform multiple tasks concurrently (such as a web browser rendering a webpage on one thread while streaming audio on another) with minimal context-switching overhead.
• Risk of Shared Memory: Because threads share memory, an unhandled fatal error or segmentation fault in any single thread will terminate the entire parent process and all its sister threads.

The Process Hierarchy & System Init (PID 1)

When a Linux operating system boots, the kernel initializes hardware, mounts the root filesystem, and launches the very first user-space process assigned Process ID 1 (PID 1):

• In modern enterprise Linux distributions (RHEL, Ubuntu, Debian, CentOS, Rocky Linux), systemd acts as PID 1.
• In legacy Unix systems, this was the init daemon (SysVinit).
• PID 1 serves as the direct or indirect ancestor of every subsequent process spawned on the system. It is responsible for bringing the system to its designated target state, managing system daemons, and adopting/reaping orphaned processes.

---

2. Process Lifecycle & Execution States

During its lifespan, a process transitions through various execution states governed by the kernel's CPU scheduler.

+-----------------------------------------------------------------------------+
|                        PROCESS LIFECYCLE STATE MACHINE                      |
|                                                                             |
|       [NEW / FORK]                                                          |
|            |                                                                |
|            v                                                                |
|     +------------+      Kernel Schedules CPU Slice      +-------------+     |
|     | READY (R)  | -----------------------------------> | RUNNING (R) |     |
|     +------------+ <----------------------------------- +-------------+     |
|           ^              CPU Time Slice Expired (Preempt)      |   |        |
|           |                                                    |   |        |
|           | I/O Complete / Signal Received                     |   |        |
|           |                                                    |   |        |
|     +------------+                                             |   |        |
|     | SLEEPING   | <-------------------------------------------+   |        |
|     | - S (Intr) |       Awaiting I/O, Event, or Sleep Timer       |        |
|     | - D (Unin) |                                                 |        |
|     +------------+                                                 |        |
|                                                                    |        |
|     +------------+       SIGSTOP / SIGTSTP (Ctrl+Z)                |        |
|     | STOPPED (T)| <-----------------------------------------------+        |
|     +------------+ ------------------------------------------------+        |
|                                  SIGCONT                                    |
|                                                                             |
|     +------------+       Process Terminates (exit())               |        |
|     | ZOMBIE (Z) | <-----------------------------------------------+        |
|     +------------+                                                          |
|            |                                                                |
|            | Parent calls wait() to read exit status                        |
|            v                                                                |
|     [REAPED / DEAD]                                                         |
+-----------------------------------------------------------------------------+

Standard Process States in Linux

1. Running / Runnable (R): The process is currently executing machine instructions on a CPU core or sitting in the kernel run-queue waiting for an immediate CPU time slice.
2. Interruptible Sleep (S): The process is suspended, waiting for an event, user input, timer expiration, or I/O availability (such as keyboard keystrokes or network packet arrivals). It wakes up immediately upon receiving a POSIX signal.
3. Uninterruptible Sleep (D - Disk/Device Sleep): The process is blocked waiting directly on synchronous hardware I/O (typically waiting for a response from an NVMe drive, SAN storage, or NFS network share). Crucially, a process in state D cannot be killed even by SIGKILL (9) until the hardware subsystem completes the I/O or returns a hardware timeout.
4. Stopped / Traced (T): The process has been suspended by a job control signal (such as SIGSTOP or pressing Ctrl+Z in a terminal) or is being actively inspected by a debugger (gdb, strace). Execution resumes only upon receiving SIGCONT.
5. Zombie / Defunct (Z): The process has completed execution and called exit(), releasing its memory, open files, and CPU allocations. However, its entry remains in the kernel's Process Table because its parent process has not yet executed the wait() or waitpid() system call to read the child's exit status code.

Dealing with Zombie and Orphan Processes

• Zombie Processes: Because zombies consume zero RAM and zero CPU, a few zombies are harmless. However, if a buggy parent process continuously spawns and abandons children without reaping them, the kernel Process Table can become exhausted (reaching pid_max), preventing any new processes from spawning. You cannot kill a zombie with kill -9 <PID> because it is already dead. To eliminate a zombie, you must kill or restart the parent process (PPID), causing PID 1 (systemd) to adopt the zombie and immediately reap it.
• Orphan Processes: When a parent process terminates before its child finishes execution, the child becomes an orphan. The Linux kernel automatically re-parents orphaned processes to systemd (PID 1), which monitors the orphan and reaps its exit code upon termination.

---

3. Linux Process Management Commands & Prioritization

Systems administrators use a suite of command-line tools to monitor system health, inspect resource consumption, manage background jobs, and adjust CPU execution priorities.

+-----------------------------------------------------------------------------+
|                        LINUX PROCESS MONITORING COMMANDS                    |
|                                                                             |
|   ps aux (BSD Style)                         ps -ef (Standard POSIX Style)  |
|   - a: All users' processes                  - -e: Every process on system  |
|   - u: User-oriented format (CPU/MEM %)      - -f: Full format listing      |
|   - x: Include processes without a TTY       - Shows: UID, PID, PPID, C,    |
|   - Shows: USER, PID, %CPU, %MEM, VSZ,               STIME, TTY, TIME, CMD  |
|            RSS, TTY, STAT, START, TIME, CMD                                 |
+-----------------------------------------------------------------------------+

ps (Process Status)

• ps aux: Displays a complete snapshot of all running processes across all users with resource usage percentages. (VSZ = Virtual Memory Size in KB; RSS = Resident Set Size / actual physical RAM in KB; STAT = Process State e.g., Ss, R+, Z).
• ps -ef: Displays a hierarchical POSIX view including Parent Process IDs (PPID), useful for identifying parent-child relationships.
• ps -u alice: Displays all processes owned by the user alice.
• ps aux --sort=-%mem | head -n 10: Displays the top 10 memory-consuming processes.

Interactive Real-Time Monitors: top & htop

• top: The standard interactive terminal monitor displaying real-time CPU load averages (1, 5, 15 minutes), memory utilization, swap usage, and a dynamic process list.
  • k: Prompts for a PID and signal number to kill a process directly.
  • r: Prompts for a PID to renice (re-prioritize) a process.
  • M: Sorts process table by physical Memory usage.
  • P: Sorts process table by CPU usage.
  • q: Quits top.
• htop: An enhanced, full-color interactive process viewer featuring per-core CPU meters, mouse click support, visual tree views (F5), and quick filtering (F4) without requiring memorization of PIDs.

Background Jobs & Shell Job Control

When executing long-running tasks in an interactive shell, administrators use job control operators:

• & (Background Execution): Appending & runs a command immediately in the background, freeing the terminal prompt (tar -czf backup.tar.gz /var/www/ &).
• jobs -l: Lists all active background jobs in the current shell session with job numbers and PIDs.
• Ctrl+Z: Sends SIGTSTP to pause the active foreground command and place it in the background in a Stopped state.
• bg %1: Resumes stopped job number 1 in the background.
• fg %1: Brings background job number 1 into the foreground.
• nohup command &: Runs a command immune to hangups (SIGHUP), allowing the process to continue running even after the user logs out or closes the SSH terminal.

Process Priority: nice and renice

Linux implements dynamic priority scheduling via Niceness values ranging from -20 (highest priority / least nice to others) to 19 (lowest priority / most nice to others). The default niceness for standard user processes is 0.

• Launching a process with custom priority: nice -n 10 tar -czf backup.tar.gz /data
• Modifying priority of an already running process: sudo renice -n -5 -p 4210

---

4. POSIX Signals & Termination Mechanics

Operating systems communicate control events to processes via asynchronous notifications called POSIX Signals. Administrators send signals using the kill, killall, and pkill utilities.

+-----------------------------------------------------------------------------+
|                          POSIX TERMINATION SIGNALS                          |
|                                                                             |
|   [ADMINISTRATOR / KERNEL]                                                  |
|              |                                                              |
|              +---> SIGTERM (15) ---> [PROCESS] Caught -> Flushes buffers,   |
|              |                                           closes DB handles, |
|              |                                           deletes temp files |
|              |                                           -> Exits cleanly.  |
|              |                                                              |
|              +---> SIGKILL (9)  ---> [KERNEL] Bypasses process completely.  |
|                                      Kernel immediately frees RAM & handles.|
|                                      Process CANNOT intercept or clean up.  |
+-----------------------------------------------------------------------------+

Core Linux Signals Reference

Linux Signal Commands

• kill -15 3421 (or kill 3421): Sends SIGTERM to PID 3421.
• kill -9 3421: Forcefully terminates PID 3421 via SIGKILL.
• killall nginx: Sends SIGTERM to all processes named nginx.
• killall -9 httpd: Forcefully terminates all Apache httpd worker processes.
• pkill -u alice -9: Kills all processes owned by user alice.
• pkill -f "python3 worker.py": Matches full command-line strings using regex.

---

5. Linux Services & Daemons (systemd Architecture)

A Daemon is a background process that runs continuously without a controlling terminal, waiting to handle network requests, scheduled tasks, or hardware events (e.g., sshd, nginx, cron, systemd-resolved).

In modern Linux distributions, daemons are organized as Units managed by the systemd init and service manager.

+-----------------------------------------------------------------------------+
|                        SYSTEMD UNIT FILE ARCHITECTURE                       |
|                                                                             |
|   File: /etc/systemd/system/myapp.service                                   |
|   +---------------------------------------------------------------------+   |
|   | [Unit]                                                              |   |
|   | Description=Enterprise Node.js API Service                          |   |
|   | After=network.target mysql.service                                  |   |
|   |                                                                     |   |
|   | [Service]                                                           |   |
|   | Type=simple                                                         |   |
|   | User=www-data                                                       |   |
|   | WorkingDirectory=/opt/myapp                                         |   |
|   | ExecStart=/usr/bin/node /opt/myapp/server.js                        |   |
|   | Restart=on-failure                                                  |   |
|   | RestartSec=5s                                                       |   |
|   |                                                                     |   |
|   | [Install]                                                           |   |
|   | WantedBy=multi-user.target                                          |   |
|   +---------------------------------------------------------------------+   |
+-----------------------------------------------------------------------------+

systemctl Service Management Operations

---

6. Windows Process & Service Management

Windows operating systems provide both graphical diagnostics and command-line PowerShell cmdlets for controlling active processes and background Windows Services.

Graphical Diagnostic Utilities

• Task Manager (taskmgr.exe):
  • Processes Tab: Real-time grouped display of Apps, Background Processes, and Windows Processes with CPU, RAM, Disk, and Network utilization.
  • Performance Tab: Dynamic graphs of CPU clock frequency, memory commit charges, disk read/write response latency, Ethernet/Wi-Fi throughput, and GPU load.
  • Startup Apps Tab: Manages startup programs impact score and enables/disables programs from launching upon interactive user login.
  • Details Tab: Direct process listing displaying PIDs, status, username contexts, CPU architecture (32-bit vs 64-bit), and memory Working Set.
  • Services Tab: Displays installed Windows Services with PIDs and operational status (Running, Stopped).
• Resource Monitor (resmon.exe): Provides granular breakdown of Disk I/O per file, Network connections per TCP port, and Memory hardware faults.
• Performance Monitor (perfmon.msc): Enterprise performance tracking tool used to record historical Data Collector Sets across hundreds of performance counters.
• Sysinternals Process Explorer (procexp.exe): Microsoft's advanced administrative tool showing real-time hierarchical process trees (parent-child relationships), loaded DLLs, open kernel handles, and VirusTotal malware hash scanning.

Windows Services Architecture & PowerShell Management

Windows Services run in isolated session contexts (Session 0) under dedicated service accounts (NT AUTHORITY\SYSTEM, LOCAL SERVICE, NETWORK SERVICE).

---

7. System Logging & Diagnostics (Linux & Windows)

System logging provides the definitive audit trail required to troubleshoot intermittent errors, hardware failures, security intrusions, and unexpected system reboots.

+-----------------------------------------------------------------------------+
|                        SYSTEM LOGGING ARCHITECTURES                         |
|                                                                             |
|   [LINUX LOGGING ARCHITECTURE]                                              |
|   Kernel Ring Buffer ---> dmesg                                             |
|   Applications / Services ---> systemd-journald ---> /run/log / /var/log/journal
|                           ---> rsyslogd        ---> /var/log/syslog / auth.log
|                                                                             |
|   [WINDOWS LOGGING ARCHITECTURE]                                            |
|   OS Subsystems / Apps ---> Windows Event Log Service ---> .evtx Files      |
|                             - Application.evtx                              |
|                             - Security.evtx                                 |
|                             - System.evtx                                   |
|   Queried via: Event Viewer (eventvwr.msc) or Get-WinEvent / Get-EventLog   |
+-----------------------------------------------------------------------------+

Linux Logging: /var/log & journalctl

Linux systems store human-readable ASCII logs under /var/log alongside binary systemd journal logs.

• Core Files in /var/log:
  • /var/log/syslog (Ubuntu/Debian) or /var/log/messages (RHEL/CentOS): General system messages, service activity, and general application notices.
  • /var/log/auth.log (Ubuntu/Debian) or /var/log/secure (RHEL/CentOS): User authentication events, sudo elevations, SSH logins, and failed password attempts.
  • /var/log/kern.log: Linux kernel messages, hardware driver alerts, and firewall (iptables/nftables) drop logs.
  • /var/log/boot.log: System startup initialization messages.
• dmesg (Driver Message): Displays messages from the kernel ring buffer, crucial for diagnosing newly connected USB devices, storage controller errors, and RAM/CPU hardware faults (dmesg -T adds human-readable timestamps; dmesg | grep -i error).
• journalctl Querying: systemd-journald captures structured, binary logs indexed for high-speed retrieval:
  • journalctl -u nginx.service: Displays logs specifically for the Nginx unit.
  • journalctl -f: Continuously follows and streams new log entries in real time (equivalent to tail -f).
  • journalctl -xe: Opens journal at the end of the log with extended diagnostic explanations (-x) and jumps to bottom (-e).
  • journalctl -b: Filters logs generated during the current boot cycle (journalctl -b -1 shows previous boot cycle).
  • journalctl -p err: Filters logs by priority level (emerg, alert, crit, err, warning, notice, info, debug).
  • journalctl --since "30 min ago": Time-bounded log extraction.

Windows Event Viewer (eventvwr.msc)

Windows records structured XML events into binary .evtx files stored in C:\Windows\System32\winevt\Logs\.

Standard Windows Event Channels

1. Application: Logs generated by installed applications and third-party software (e.g., SQL Server, Outlook, custom enterprise apps).
2. Security: Logs security audits, user logon/logoff events, privilege use, file access permissions, and account management policies. Governed by Local Audit Policy.
3. System: Logs generated by core Windows operating system components, device drivers, hardware errors, and Windows Services failures.
4. Setup: Logs recorded during operating system installations, service pack deployments, and Windows Updates.

Event Severity Levels

• Information: Normal operational milestones (e.g., a service started successfully).
• Warning: A potential issue that does not immediately halt execution but may cause future failure (e.g., low disk space warning).
• Error: Significant problem causing loss of functionality in a service or application.
• Critical: Severe failure causing an operating system crash, blue screen (BSOD), or hardware failure.
• Verbose: Detailed low-level technical progress notes used during active software debugging.

Critical Windows Event IDs for IT Support

Querying Logs with PowerShell Get-WinEvent

PowerShell provides rapid event log filtering across local and remote systems:

# Query the latest 10 failed logon attempts from the Security log:
Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4625} -MaxEvents 10 | Format-Table TimeCreated, Message -Wrap

# Query all Error and Critical events from the System log in the last 24 hours:
Get-WinEvent -FilterHashtable @{
    LogName   = 'System'
    Level     = 1, 2  # 1 = Critical, 2 = Error
    StartTime = (Get-Date).AddDays(-1)
} | Select-Object TimeCreated, Id, ProviderName, Message