2.2 IoT & Operational Technology (OT/SCADA) Forensic Fundamentals

2.2 IoT & Operational Technology (OT/SCADA) Forensic Fundamentals

The expansion of interconnected smart devices and the convergence of enterprise Information Technology (IT) with Operational Technology (OT) have radically reshaped the digital forensics landscape. Investigators are frequently called to investigate incidents involving Internet of Things (IoT) edge appliances, smart building controls, and critical infrastructure facilities—including electrical power grids, municipal water treatment plants, and automated manufacturing pipelines.

Unlike standardized workstation or server environments, IoT and OT systems are characterized by extreme architectural diversity, proprietary operating systems, real-time operating systems (RTOS), fragile network stacks, and strict physical safety requirements. A forensic misstep in an OT environment can lead not only to data spoliation, but to catastrophic physical damage, environmental contamination, or loss of human life.

---

IoT Architectural Layers & Evidence Sources

To conduct an organized IoT investigation, forensic analysts rely on the standardized three-tier IoT architectural framework:

+---------------------------------------------------------------------------------+
|                            IOT ARCHITECTURAL LAYERS                             |
+---------------------------------------------------------------------------------+
|  Application Layer                                                              |
|  - Cloud APIs, mobile companion apps, database historians, web dashboards       |
|  - Evidence: Server access logs, mobile app databases (SQLite), user telemetry  |
+---------------------------------------------------------------------------------+
|  Network / Transmission Layer                                                   |
|  - Communication protocols: Zigbee, Z-Wave, BLE, LoRaWAN, 6LoWPAN, Wi-Fi       |
|  - Evidence: Wireshark PCAPs, gateway routing tables, 802.15.4 frame dumps      |
+---------------------------------------------------------------------------------+
|  Perception / Sensing Layer                                                     |
|  - Physical transducers, actuators, sensors, edge microcontrollers (ARM, ESP32) |
|  - Evidence: Flash memory dumps, EEPROM, SRAM state, hardware test points       |
+---------------------------------------------------------------------------------+

1. Perception (Sensing) Layer

Consists of physical sensors (temperature, pressure, acoustic, motion), actuators (solenoids, relays, valves), and edge microcontrollers (e.g., ARM Cortex-M, ESP32, AVR, PIC). Evidence at this layer exists in hardware memory components—volatile SRAM, internal flash, and external SPI/I2C EEPROMs.

2. Network (Transmission) Layer

Facilitates communication between perception devices and centralized controllers. Edge technologies frequently employ low-power wireless personal area networks (WPAN):

• Zigbee (IEEE 802.15.4): 2.4 GHz mesh networking utilizing coordinators, routers, and end devices.
• Z-Wave: Proprietary sub-GHz (908.42 MHz US / 868.42 MHz EU) mesh network protocol optimized for home automation.
• Bluetooth Low Energy (BLE): Operates over 2.4 GHz ISM band utilizing Generic Attribute Profile (GATT) services and characteristics.
• LoRaWAN: Low-power wide-area network protocol designed for long-range telemetry (up to 15 km) over sub-GHz radio frequencies.
• 6LoWPAN: Transports IPv6 packets over low-power IEEE 802.15.4 wireless links via header compression.

3. Application Layer

Integrates device data with enterprise backends, mobile companion applications (iOS/Android), and cloud infrastructure brokers (Amazon AWS IoT Core, Microsoft Azure IoT Hub). Application protocols include MQTT (Message Queuing Telemetry Transport, running over TCP port 1883 or 8883 with TLS) and CoAP (Constrained Application Protocol, running over UDP port 5683 or 5684 with DTLS).

---

Edge Device Hardware Acquisition Constraints

Acquiring digital evidence from embedded IoT hardware presents significant physical and electrical challenges:

1. Absence of Standard Storage Interfaces: Embedded devices do not feature SATA, NVMe, or USB mass storage ports. Memory is surface-mounted onto the printed circuit board (PCB).
2. Volatile Memory Volatility: Microcontroller SRAM is exceedingly small (often 64 KB to 512 KB) and flushes instantaneously upon power loss. Many microcontrollers lack a battery-backed Real-Time Clock (RTC), causing system timestamps to reset to the Unix epoch (1970-01-01) or firmware compile date on reboot.
3. Anti-Tamper & Zeroization: High-security devices incorporate enclosure micro-switches or light sensors that trigger cryptographic key erasure (zeroization) if the chassis is opened.
4. Proprietary Flash Translation Layers (FTL): Direct extraction of raw NAND flash chips captures out-of-band (OOB) spare area data, bad block markings, and wear-leveling artifacts that require custom reverse engineering to assemble into a readable file system.

---

Physical Flash Memory Extraction Methodologies

When logical access (SSH, telnet, web portal) is disabled or compromised, investigators must turn to hardware-level acquisition methods. These techniques are categorized by their invasiveness and electrical complexity:

1. UART (Universal Asynchronous Receiver-Transmitter)

UART is a two-wire asynchronous serial protocol used for hardware debugging. Most IoT PCBs expose UART test pads:

• Pinout Identification: Using a digital multimeter (DMM) or logic analyzer, identify GND (continuity to ground plane), VCC (constant 3.3V or 1.8V), TX (fluctuating voltage during boot), and RX.
• Baud Rate Detection: Common baud rates include 115200, 57600, 38400, and 9600 (8 data bits, no parity, 1 stop bit: 8N1).
• Bootloader Interception: Connect via a serial terminal (screen /dev/ttyUSB0 115200). By pressing an interrupt key (such as Space or Enter) during initial power-on, the analyst halts the U-Boot bootloader, gaining access to commands like printenv (display environment variables), md (memory display), or dumping flash over a TFTP network link.

2. JTAG (Joint Test Action Group - IEEE 1149.1)

JTAG provides hardware boundary-scan capabilities, allowing an investigator to halt the processor, control the CPU registers, and read or write system memory and external flash chips without running code on the target OS.

+-----------------------------------------------------------------------------+
|                        JTAG TEST ACCESS PORT (TAP)                          |
+-----------------------------------------------------------------------------+
|  Signals:                                                                   |
|  - TCK (Test Clock): Synchronizes test logic operations                     |
|  - TMS (Test Mode Select): Drives the 16-state TAP controller state machine |
|  - TDI (Test Data In): Serial input data to instruction/data registers      |
|  - TDO (Test Data Out): Serial output data shifted out of registers         |
|  - TRST (Test Reset): Optional active-low asynchronous reset pin            |
+-----------------------------------------------------------------------------+

When unlabeled test pads exist on a board, forensicators use a JTAGulator—an open-source hardware tool that automates pinout discovery by driving test vectors across all possible pin combinations to identify the TAP controller state machine response.

3. In-System Programming (ISP)

ISP allows direct extraction of eMMC or eMCP flash chips without removing them from the PCB. The investigator solders microscopic "fly-wires" (enamelled copper wire) directly to exposed PCB test points connecting to the memory chip's data lines:

• DAT0 (Data line 0)
• CMD (Command line)
• CLK (Clock line)
• VCC / VCCQ (Core and I/O voltages, typically 1.8V / 3.3V)
• GND (Ground)

The wires connect to an eMMC reader (such as an EasyJTAG Plus or Medusa Pro). The reader powers only the memory bus while leaving the main host CPU unpowered, performing a full bit-stream physical dump of the eMMC user partitions (User Data, Boot 1, Boot 2, RPMB).

4. Chip-off Forensics

Chip-off is the method of last resort. The target IC (such as a 153-ball BGA eMMC chip) is physically desoldered from the board:

1. Apply specialized no-clean liquid flux around the chip perimeter.
2. Heat the PCB using an infrared preheater (150°C) and hot-air rework nozzle (230°C–250°C) until lead-free solder balls liquefy.
3. Lift the chip vertically using vacuum tweezers to prevent shearing solder pads.
4. Clean residual solder from the chip underside using a soldering iron, solder wick, and isopropyl alcohol (IPA).
5. Place the chip into a matching hardware socket adapter (e.g., UP-828 or Xeltek SuperPro) to extract a raw physical binary image.

---

OWASP IoT Top 10 Vulnerabilities

Forensic investigators must understand common attack vectors exploited by adversaries targeting connected edge devices. The OWASP IoT Top 10 provides the baseline vulnerability framework:

1. Weak, Guessable, or Hardcoded Passwords: Publicly known default credentials (e.g., admin:admin, root:vizcvio in Mirai botnet variants).
2. Insecure Network Services: Unneeded, unencrypted listening ports (telnet on port 23, cleartext HTTP, debug ports).
3. Insecure Ecosystem Interfaces: Vulnerabilities in web admin consoles, cloud backend APIs, or mobile apps without mutual authentication.
4. Lack of Secure Update Mechanisms: Firmware updates lacking digital cryptographic signatures, sent over unencrypted HTTP channels.
5. Use of Insecure or Outdated Components: Outdated Linux kernels, vulnerable BusyBox binaries, or unpatched third-party libraries.
6. Insufficient Privacy Protection: User PII stored unencrypted on edge devices or transmitted without TLS.
7. Insecure Data Transfer and Storage: Lack of encryption at rest on SPI flash or cleartext transmission over the local network.
8. Lack of Device Management: Inability to inventory, monitor, or decommission compromised appliances.
9. Insecure Default Settings: Permissive factory default access permissions.
10. Lack of Physical Hardening: Readily accessible UART, JTAG, or USB debug ports enabling physical extraction of credentials and keys.

---

Industrial Control Systems (ICS) / SCADA Forensics

Operational Technology (OT) encompasses hardware and software that monitors and controls physical processes—including Supervisory Control and Data Acquisition (SCADA) systems, Distributed Control Systems (DCS), Programmable Logic Controllers (PLCs), Remote Terminal Units (RTUs), and Intelligent Electronic Devices (IEDs).

The Purdue Enterprise Reference Architecture (Purdue Model)

Forensic analysis of industrial incidents is structured according to the Purdue Model (ISA-95 / IEC 62443 standard):

+---------------------------------------------------------------------------------+
|                       PURDUE MODEL & FORENSIC ARTIFACTS                         |
+---------------------------------------------------------------------------------+
| Level 5: Enterprise WAN / External Cloud                                        |
| - Remote vendor access, external telemetry, corporate interconnects             |
+---------------------------------------------------------------------------------+
| Level 4: Enterprise Business Network                                            |
| - Active Directory, ERP, corporate email                                        |
| - Evidence: Windows Event Logs, NetFlow, firewall logs, email headers          |
+=================================================================================+
| Level 3.5: Industrial DMZ (IDMZ)                                                |
| - Jump boxes, data diodes, reverse proxies, patch servers                       |
| - Evidence: Jump host RDP logs, IDMZ firewall session tables                     |
+=================================================================================+
| Level 3: Site Operations & Supervisory Control                                  |
| - Process Historian, HMI servers, Engineering Workstations (EWS)               |
| - Evidence: Historian tag logs, EWS project files, ladder logic source files    |
+---------------------------------------------------------------------------------+
| Level 2: Area Supervisory Control                                               |
| - Local HMIs, SCADA supervisory servers, alarm monitors                         |
| - Evidence: HMI operator action logs, alarm event logs, screen touch archives   |
+---------------------------------------------------------------------------------+
| Level 1: Basic Control                                                          |
| - Programmable Logic Controllers (PLCs), RTUs, IEDs, Safety Controllers (SIS)   |
| - Evidence: PLC firmware image, compiled ladder logic (OB1 blocks), RAM state   |
+---------------------------------------------------------------------------------+
| Level 0: Physical Process                                                       |
| - Sensors, motors, pumps, control valves, centrifuges, conveyor belts           |
| - Evidence: Physical damage patterns, valve positions, calibration drift        |
+---------------------------------------------------------------------------------+

IT vs. OT Forensic Priorities: The Inverted Triad

In standard enterprise IT investigations, the classic security priority is Confidentiality > Integrity > Availability (CIA). In Operational Technology and Critical Infrastructure, this priority is strictly inverted:

$\text{OT Priority:} \quad \mathbf{Safety} \;\ge\; \mathbf{Availability} \;>\; \mathbf{Integrity} \;>\; \mathbf{Confidentiality}$

• Safety & Human Life: A system restart or power interruption can cause a petrochemical pipeline rupture, boiler explosion, or chemical release.
• Availability (Uptime): Industrial processes run 24/7/365. Many plants cannot tolerate network latency jitter exceeding a few milliseconds, let alone downtime.
• Integrity: Ensuring process setpoints, sensor feedback, and control logic execute without tampering.
• Confidentiality: Historically secondary, though proprietary batch formulas and design diagrams remain sensitive.

---

Industrial Network Protocols & Forensic Analysis

Legacy industrial communication protocols were engineered decades ago for closed, serial-connected environments. They inherently lack authentication, encryption, and message replay protection.

1. Modbus TCP Dissection

Modbus TCP encapsulates traditional Modbus RTU frames within a 7-byte MBAP (Modbus Application Protocol) header:

+-------------------------------------------------------------------------+
|                           MODBUS TCP FRAME FORMAT                       |
+-------------------+-------------------+-----------------+---------------+ 
| Transaction ID    | Protocol ID (0x0) | Length (2 bytes)| Unit ID (1 B) |
| (2 bytes)         | (2 bytes)         |                 |               |
+-------------------+-------------------+-----------------+---------------+ 
| Function Code     | Data Payload                                        |
| (1 byte)          | (Variable length)                                   |
+-------------------+-----------------------------------------------------+

Key Function Codes in Forensics:

• 0x01: Read Coils (Discrete outputs/relays)
• 0x03: Read Holding Registers (Analog setpoints)
• 0x05: Write Single Coil (Force relay ON/OFF)
• 0x06: Write Single Register (Alter setpoint)
• 0x10 (16): Write Multiple Registers (Batch setpoint modification)

In Wireshark, an investigator filters for unauthorized setpoint writes using:

modbus.func_code == 0x05 || modbus.func_code == 0x06 || modbus.func_code == 0x10

2. Siemens S7comm & Critical Infrastructure Attacks

Siemens S7comm operates over ISO-on-TCP (port 102, RFC 1006). Adversaries (such as the operators behind Stuxnet or Industroyer) abuse S7comm to upload modified Organization Blocks (OB1—the main cyclic logic execution block) or issue CPU stop commands.

In Wireshark, filtering for PLC control commands reveals:

s7comm.param.func == 0x28  # PLC Control (Start/Stop CPU)
s7comm.param.func == 0x1a  # Request Download (Writing malicious ladder logic)

---

Safety & Operational Constraints in Critical Infrastructure Forensics

Forensic investigators entering an OT/SCADA facility must adhere to strict operational guidelines:

[!CAUTION] Never Run Active Port Scanners in an OT Environment: Running active discovery tools like nmap, nessus, or automated vulnerability scanners against Level 1 or Level 2 OT networks can immediately crash legacy embedded PLC network stacks. Older PLCs utilize fragile, non-RFC-compliant TCP/IP micro-stacks that fault when receiving unexpected SYN packets, malformed flags, or high connection volumes, causing the physical plant to trip.

Safe Evidence Collection Protocols for OT Systems

1. Passive Network Tapping: Evidence collection must rely strictly on hardware Test Access Points (TAPs) or monitored SPAN ports mirroring traffic to a dedicated passive capture appliance. Fiber taps are preferred because they physically prevent electrical signal reflection or back-propagation.
2. Historian & HMI Extraction: Collect data from Level 2 and Level 3 Windows-based servers (Historians, SCADA supervisory servers, EWS) using standard DFIR disk imaging tools (FTK Imager, KAPE) while coordinating with plant operators.
3. Safety Instrumented Systems (SIS): The SIS (operating at Safety Integrity Levels SIL 1 through SIL 4) operates independently from standard PLCs to ensure emergency shutdown if safe operating parameters are breached. Investigators must never attach hardware probes or modify connections on SIS controllers while the process is active.
4. Outage Window Acquisition: If physical memory extraction (JTAG, UART, Chip-off) or PLC firmware dumping is required, it must be scheduled during planned maintenance shutdowns with the written sign-off of the Plant Operations Director.