16.1 Cognitive Engineering, Human-Machine Interface, Displays, and Controls

In high-throughput industrial facilities, automated manufacturing plants, and complex chemical processing units, the human operator functions as the ultimate supervisory controller, decision maker, and safety barrier. Cognitive ergonomics addresses the mental capabilities, cognitive limitations, and psychological workload of workers interacting with machines, computerized interfaces, and automated control loops. Designing systems that align with human neurocognitive architecture significantly reduces error rates, lowers mental fatigue, and prevents catastrophic industrial accidents.

---

1. The Human Information Processing Model

The standard engineering model of human cognition conceptualizes the human operator as an information processing channel that filters, transforms, stores, and responds to sensory inputs:

Human Information Processing Architecture
┌─────────────────┐     ┌────────────────────────┐     ┌───────────────────────┐
│ Sensory Inputs  │ ──> │ Sensory Registers      │ ──> │ Selective Attention   │
│ (Visual, Audio) │     │ Visual: 0.5 s, Audio: 3s│     │ Filter / Bottleneck   │
└─────────────────┘     └────────────────────────┘     └───────────┬───────────┘
                                                                   │
                                                                   v
┌─────────────────┐     ┌────────────────────────┐     ┌───────────────────────┐
│ Motor Execution │ <── │ Decision Making        │ <── │ Working Memory (WM)   │
│ Muscles/Controls│     │ & Response Selection   │ <── │ (7 ± 2 items, 15-20 s) │
└─────────────────┘     └───────────┬────────────┘     └───────────▲───────────┘
                                    │                              │ Retrieval
                                    │ Encoding / Feedback          │ & Storage
                                    v                              v
                        ┌──────────────────────────────────────────────────────┐
                        │ Long-Term Memory (LTM)                               │
                        │ Semantic, Episodic, Procedural Mental Models         │
                        └──────────────────────────────────────────────────────┘

Stages of Cognitive Processing

1. Sensory Receptors and Registers: Environmental physical stimuli are converted into neurological signals. Iconic memory (visual) persists for approximately $0.5\text{ seconds}$; echoic memory (auditory) persists for $2\text{ to }4\text{ seconds}$. This buffer permits brief sensory pattern recognition.
2. Perception and Selective Attention: Attention acts as a capacity-limited filter. Stimuli are parsed using top-down expectations (mental models) and bottom-up sensory salience (brightness, loudness, flashing indicators).
3. Working Memory (Short-Term Memory): The conscious workspace where active calculation, synthesis, and reasoning occur. As formalized by George Miller (1956), working memory capacity is limited to $7 \pm 2$ chunks of unorganized information (modern cognitive research shows $4 \pm 1$ chunks for active complex manipulation). Information in working memory decays within $10\text{ to }20\text{ seconds}$ unless refreshed through active rehearsal.
4. Long-Term Memory (LTM):
   • Semantic Memory: Context-free factual knowledge, system rules, engineering principles.
   • Episodic Memory: Temporally tagged personal operational experiences and historical shift events.
   • Procedural Memory: Motor and cognitive skills executed automatically without conscious verbalization (e.g., typing, emergency shutdown valve actuation sequence).
5. Decision Making and Heuristics: Cognitive shortcuts applied under time pressure. Prone to systematic cognitive biases:
   • Confirmation Bias: Seeking out information that confirms a preconceived diagnosis while ignoring conflicting sensor readouts.
   • Availability Heuristic: Assessing probability based on how easily similar past emergencies come to mind.
   • Anchoring: Fixating on initial readings or early estimates despite subsequent divergent data.
6. Response Selection and Execution: Selecting a motor action plan and transmitting neuro-muscular commands to hands, feet, or vocal cords.

---

2. Rasmussen's Skills, Rules, and Knowledge (SRK) Framework

Jens Rasmussen categorized human performance in engineering and supervisory tasks into three distinct cognitive tiers:

Rasmussen's SRK Cognitive Ladder
[Novel System State / Crisis]      ──> Knowledge-Based (Analytical Goals & Mental Models)
                                              ▲
                                              │ Escalation under ambiguity
[Familiar Pattern Recognized]      ──> Rule-Based (IF-THEN Procedures & Checklists)
                                              ▲
                                              │ Disturbance / Interruption
[Routine Automated Operation]      ──> Skill-Based (Subconscious Sensorimotor Coordination)

---

3. Quantitative Response Time Models

Hick-Hyman Law for Choice Reaction Time

When an operator must choose a discrete response from $n$ possible alternatives, the choice reaction time ($RT$) increases logarithmically with the amount of information (entropy) transmitted:

$RT = a + b \cdot H_T$

For $n$ equiprobable stimuli ($p_i = 1/n$):

$H_T = \log_2(n) \implies RT = a + b \cdot \log_2(n)$

where:

• $RT$ = Total reaction time (seconds or milliseconds)
• $a$ = Simple reaction time (baseline sensory detection and motor nerve conduction delay, typically $0.15 - 0.25\text{ s}$)
• $b$ = Processing speed per bit of decision uncertainty (slope coefficient, typically $0.15 - 0.25\text{ s/bit}$)
• $H_T$ = Information content transmitted, measured in bits: $H_T = \sum_{i=1}^n p_i \log_2(1/p_i)$

Fitts' Law for Movement Time (Motor Target Acquisition)

Fitts' Law models the time required to rapidly move a limb or pointing device (e.g., stylus, mouse cursor, finger) a distance $D$ to a target of width $W$:

$MT = a + b \cdot \log_2\left( \frac{2D}{W} \right) = a + b \cdot ID$

where:

• $MT$ = Movement time (seconds)
• $D$ = Distance (amplitude) from starting point to the center of the target
• $W$ = Width of the target along the dimension of motion
• $ID = \log_2(2D/W)$ = Index of Difficulty, expressed in bits
• $a, b$ = Empirically fitted regression constants specific to the body limb and control device

Engineering Design Takeaway: To minimize operator task completion time on touchscreen interfaces and control boards, critical buttons must have large target widths ($W$) and be placed at short distances ($D$) from neutral resting positions.

---

4. Cognitive Workload and Measurement: NASA-TLX

Mental workload reflects the proportion of an operator's limited cognitive processing capacity demanded by a set of concurrent tasks. When workload exceeds available mental bandwidth, error rates spike exponentially.

NASA Task Load Index (NASA-TLX)

The NASA-TLX is the gold-standard subjective multidimensional workload assessment instrument. It evaluates six independent subscales on a continuous $0 - 100$ scale:

1. Mental Demand (MD): How much mental and perceptual activity was required (e.g., thinking, calculating, deciding)?
2. Physical Demand (PD): How much physical activity was required (e.g., pushing, pulling, turning)?
3. Temporal Demand (TD): How much time pressure did you feel due to the pace or task rate?
4. Performance (OP): How successful do you think you were in accomplishing the goals set by the experimenter?
5. Effort (EF): How hard did you have to work (mentally and physically) to accomplish your level of performance?
6. Frustration Level (FR): How insecure, discouraged, irritated, stressed, and annoyed versus secure and relaxed did you feel?

Two-Stage NASA-TLX Scoring

1. Weighting Stage: The operator performs 15 pairwise comparisons among the 6 subscales. The weight $w_i$ for subscale $i$ equals the number of times it was chosen as the more significant source of workload ($0 \le w_i \le 5$), such that: $\sum_{i=1}^6 w_i = 15$
2. Rating Stage: The operator assigns a numerical score $R_i \in [0, 100]$ to each subscale along a 20-interval bipolar visual analog scale.
3. Composite Weighted Score Calculation: $\text{Score}_{\text{TLX}} = \frac{\sum_{i=1}^6 w_i \cdot R_i}{15}$
4. Raw TLX (RTLX): When time constraints prevent pairwise comparisons, the unweighted arithmetic mean is used: $\text{RTLX} = \frac{1}{6}\sum_{i=1}^6 R_i$.

Objective and Physiological Workload Metrics

• Heart Rate Variability (HRV): Spectral analysis of R-R intervals reveals that elevated cognitive workload suppresses parasympathetic nervous activity, significantly decreasing the spectral power peak around $0.1\text{ Hz}$.
• Pupillometry: Pupil diameter expands in direct proportion to working memory load and mathematical processing demand.
• Secondary Task Performance: Introducing a secondary task (e.g., time estimation, auditory tone discrimination) measures residual spare capacity. Degradation of secondary task performance serves as an early indicator of cognitive overload before primary task breakdown occurs.

---

5. Human Error Taxonomy & Reason's Swiss Cheese Model

Reason's Swiss Cheese Model of System Accidents

James Reason conceptualized high-hazard industrial systems as defended by multiple layers of engineering and administrative barriers (the slices of cheese). Each barrier contains inherent, shifting weaknesses and gaps (the holes):

• Latent Conditions: Systemic organizational flaws embedded deep within management decisions, poor ergonomic layouts, defective software logic, inadequate staffing, and deferred maintenance. They lie dormant for months or years.
• Active Failures: Unsafe acts (slips, lapses, mistakes, violations) committed by frontline operators at the operational "sharp end" in direct contact with the machinery.

An accident occurs only when the holes across all defensive layers momentarily align, opening a catastrophic trajectory of opportunity.

Reason's Swiss Cheese Model
Organizational  Unsafe          Defective       Operator        Loss Event
Decisions       Supervision     Protections     Active Failure  / Hazard
   │                │               │               │              ▲
   ▼                ▼               ▼               ▼              │
 ┌───┐            ┌───┐           ┌───┐           ┌───┐            │
 │ O │            │   │           │   │           │   │            │
 │   │            │ O │           │   │           │   │            │
 │   │ ══════════>│   │══════════>│ O │══════════>│ O │════════════╪═══ Trajectory
 │   │            │   │           │   │           │   │            │   of Accident
 └───┘            └───┘           └───┘           └───┘            │
 Latent          Latent          Latent          Active            │
 Condition       Condition       Condition       Failure           │

Detailed Taxonomy of Human Unsafe Acts

Classification of Human Error (Reason / Rasmussen)
Unsafe Acts
 ├── 1. Unintended Actions (Execution Failures)
 │    ├── Slips: Action not performed as intended due to physical execution breakdown
 │    │          (e.g., finger slips to adjacent switch, turning dial wrong way)
 │    └── Lapses: Memory failure or omission of an intended step
 │               (e.g., forgetting to re-arm an interlock, skipping a checklist item)
 └── 2. Intended Actions (Planning Failures & Deviations)
      ├── Mistakes: The plan is fundamentally deficient; action proceeds as planned but fails
      │    ├── Rule-Based: Misapplication of a good rule, or faithful execution of a bad rule
      │    └── Knowledge-Based: Misinterpreting a novel anomaly due to inaccurate mental model
      └── Violations: Deliberate departure from established rules and protocols
           ├── Routine Violations: Common shortcuts tolerated by workplace culture
           ├── Exceptional Violations: Ad-hoc deviations during unprecedented crises
           └── Optimizing Violations: Deviations performed for personal convenience or speed

---

6. Human-Machine Interface (HMI) Design Principles

The Principles of Compatibility

Compatibility describes the degree to which system interfaces match human perceptual and behavioral expectations. High compatibility accelerates learning, shortens reaction times, and drastically suppresses error rates.

Movement Compatibility Rules

• Warrick's Principle: The indicator pointer moves in the same direction as the side of the control knob closest to the display scale.
• Scale-Side Principle: If the control is located on one side of a linear scale, moving the control edge adjacent to the scale causes the pointer to track that edge's direction.
• Population Stereotypes:
  • Rotary knobs: Clockwise rotation $\implies$ increase in value, volume, or flow.
  • Toggle switches: In North America, flipping toggle UP $\implies$ ON/START, flipping toggle DOWN $\implies$ OFF/STOP. (Note: In the UK and parts of Europe, down traditionally means on).
  • Valves: Counterclockwise $\implies$ OPEN; Clockwise $\implies$ CLOSED ("righty-tighty, lefty-loosey").

---

7. Visual and Auditory Displays

Visual Display Selection

Check Reading and Gestalt Grouping

In complex control rooms with banks of analog meters, gauges should be physically aligned and zeroed such that under normal, safe operational conditions, all pointers orient in the identical direction (typically 12 o'clock or 9 o'clock). An abnormal condition instantly disrupts visual symmetry, allowing the human operator's preattentive visual processing to detect anomalies in milliseconds without reading individual values.

Check Reading Alignment (Normal vs Anomaly)
  Normal Operation: Symmetry Intact         Abnormal Condition: Symmetry Broken
    ┌─┐     ┌─┐     ┌─┐     ┌─┐              ┌─┐     ┌─┐     ┌─┐     ┌─┐
    │▲│     │▲│     │▲│     │▲│              │▲│     │►│     │▲│     │▲│
    └─┘     └─┘     └─┘     └─┘              └─┘     └─┘     └─┘     └─┘
     1       2       3       4                1       2       3       4
   All needles point to 12 o'clock           Meter 2 needle deflects right:
   Instant perceptual verification           Immediate preattentive visual alert

Head-Up Displays (HUDs)

HUDs project optical symbology onto a transparent combiner glass directly in the operator's line of sight:

• Optical Collimation: Light rays are focused to optical infinity ($> 6\text{ m}$), eliminating the $0.5\text{ s}$ accommodation and vergence delay required when shifting focus between external scenes and internal instrument panels.
• Conformal Symbology: Artificial horizon lines and target vectors overlay $1:1$ on real-world objects.
• Human Factors Risks: Display clutter and cognitive tunneling (attentional capture where operators fixate on digital symbology and fail to perceive unexpected physical obstacles in the visual environment).

Auditory Displays and Alarms

Auditory signals are omnidirectional and demand immediate attention, making them ideal for high-urgency alarms:

• Signal-to-Noise Ratio (SNR): Auditory alarms must be set $15\text{ to }25\text{ dBA}$ above ambient background noise to guarantee 100% detection.
• Ceiling Threshold: Sound levels should never exceed $100 - 110\text{ dBA}$ to prevent startle reflexes, panic responses, or acoustic trauma.
• Auditory Masking: Ambient industrial noise with frequencies matching the alarm tone masks the signal; alarms must incorporate multiple distinct harmonic frequency components ($500 - 3,000\text{ Hz}$).
• Alarm Fatigue: Excessive nuisance alarms cause operators to consciously silence, ignore, or bypass safety warning horns.

---

8. Control Design Principles and Mechanics

Control-Display Ratio (C/D Ratio)

The Control-Display (C/D) Ratio quantifies the relationship between the physical displacement of an input control and the resulting movement of the indicator on the display:

$C/D = \frac{\text{Displacement of Control}}{\text{Displacement of Display}} = \frac{\Delta C}{\Delta D}$

For a rotary knob of radius $R$ rotating through angle $\theta$ (in radians) controlling a pointer moving a linear distance $\Delta D$:

$\Delta C = R \cdot \theta = R \cdot \left( \frac{\theta^\circ \cdot \pi}{180^\circ} \right) \implies C/D = \frac{R \cdot \theta}{\Delta D}$

Control-Display Trade-Off
 High C/D Ratio (e.g., C/D = 3.0)        Low C/D Ratio (e.g., C/D = 0.2)
  - Low Interface Sensitivity             - High Interface Sensitivity
  - Large control motion produces         - Small control motion produces
    small display movement                  large display movement
  - Optimized for: Precision Fine Tuning  - Optimized for: Coarse, Rapid Slewing

Forms of Control Resistance

Providing physical resistance in control hardware yields essential kinesthetic and proprioceptive feedback to the operator:

1. Elastic (Spring) Resistance: Force is directly proportional to displacement ($F = k \cdot x$). Provides continuous muscle tension feedback regarding control deflection; automatically returns the control to a neutral center position.
2. Viscous Damping Resistance: Force is directly proportional to movement velocity ($F = c \cdot v$). Absorbs rapid erratic twitches, damps physiological tremors, and prevents gross overshooting during tracking.
3. Inertial Resistance: Force is proportional to acceleration ($F = m \cdot a$). Resists sudden changes in movement speed, promoting smooth continuous cranking.
4. Coulomb (Static/Frictional) Resistance: Constant resistive force opposing initial movement. Prevents accidental actuation caused by light brush or structural vibration, but excessive friction introduces stick-slip jerkiness.

Preventing Accidental Control Activation

To prevent inadvertent actuation of critical actuators (e.g., emergency stops, chemical dump valves, explosive pyrotechnic initiators):

• Recessing: Mounting pushbuttons flush or sunken beneath the control panel surface.
• Shielding and Covers: Spring-loaded flip covers, physical barriers, or mechanical collars that require two distinct actions.
• Location and Orientation: Isolating critical controls outside the normal sweep envelope of routine operational tasks.
• High Actuation Force: Sizing spring detents so that incidental brushing cannot depress the switch.
• Two-Hand Controls: Requiring concurrent, continuous depression of two separate buttons spaced $\ge 50\text{ cm}$ apart (e.g., stamping presses to keep hands out of pinch points).
• Keylocks and Mechanical Interlocks: Requiring key insertion or sequential release levers (dead-man switches).

---

9. Step-by-Step Worked Engineering Calculations

Worked Example 16.1.1: Hick-Hyman and Fitts' Law Evaluation

Problem: An industrial human-machine interface engineer is redesigning a chemical transfer station panel. The current control panel presents an operator with $8$ equiprobable routing valves. From empirical baseline testing, the operator's simple sensory-motor reaction delay is $a = 0.20\text{ s}$, and the cognitive processing slope is $b = 0.16\text{ s/bit}$. Once the decision is made, the operator moves their hand from a central keypad across a distance $D = 32\text{ cm}$ to strike a rectangular valve button of width $W = 4\text{ cm}$. The Fitts' Law coefficients for hand movement are $a_F = 0.10\text{ s}$ and $b_F = 0.14\text{ s/bit}$.

1. Calculate the transmitted cognitive information $H_T$ and the choice reaction time $RT$.
2. Calculate the Index of Difficulty ($ID$) and movement time ($MT$) for button acquisition.
3. Calculate the total response time ($TRT = RT + MT$).

Solution:

Step 1: Compute Transmitted Information and Choice Reaction Time Since all $n = 8$ choices are equiprobable: $H_T = \log_2(n) = \log_2(8) = 3.0\text{ bits}$

Applying the Hick-Hyman Law: $RT = a + b \cdot H_T = 0.20\text{ s} + (0.16\text{ s/bit})(3.0\text{ bits}) = 0.20 + 0.48 = 0.68\text{ s}$

Step 2: Compute Fitts' Law Index of Difficulty and Movement Time $ID = \log_2\left( \frac{2D}{W} \right) = \log_2\left( \frac{2 \times 32\text{ cm}}{4\text{ cm}} \right) = \log_2\left( \frac{64}{4} \right) = \log_2(16) = 4.0\text{ bits}$

Applying Fitts' Law: $MT = a_F + b_F \cdot ID = 0.10\text{ s} + (0.14\text{ s/bit})(4.0\text{ bits}) = 0.10 + 0.56 = 0.66\text{ s}$

Step 3: Calculate Total Operator Response Time $TRT = RT + MT = 0.68\text{ s} + 0.66\text{ s} = 1.34\text{ s}$

---

Worked Example 16.1.2: NASA-TLX Composite Mental Workload Derivation

Problem: A control room operator completes a supervisory monitoring simulation during a power grid emergency. Following the task, the operator performs the 15 pairwise comparisons among the six NASA-TLX dimensions to establish subscale weights, and then rates each subscale on the 0 to 100 visual analog scale. Calculate the overall composite weighted NASA-TLX workload score.

Solution:

Step 1: Verify Weight Summation $\sum_{i=1}^6 w_i = 5 + 0 + 4 + 2 + 3 + 1 = 15 \quad \checkmark$

Step 2: Calculate Weighted Product for Each Subscale

• $\text{MD}: 5 \times 85 = 425$
• $\text{PD}: 0 \times 20 = 0$
• $\text{TD}: 4 \times 90 = 360$
• $\text{OP}: 2 \times 40 = 80$
• $\text{EF}: 3 \times 75 = 225$
• $\text{FR}: 1 \times 60 = 60$

Step 3: Calculate Total Sum and Weighted Average $\sum_{i=1}^6 w_i R_i = 425 + 0 + 360 + 80 + 225 + 60 = 1,150$

$\text{Score}_{\text{TLX}} = \frac{1,150}{15} = 76.67$

Conclusion: The operator's composite weighted mental workload score is $76.7$ out of $100$, indicating severe operational cognitive saturation driven predominantly by mental and temporal demand.

---

Worked Example 16.1.3: Control-Display Ratio and Linear Travel

Problem: A precision machining operator uses a rotary control knob of diameter $d = 6.0\text{ cm}$ (radius $R = 3.0\text{ cm}$) to position a laser crosshair along a digital inspection microscope screen. The interface requires a Control-Display (C/D) ratio of $2.5$ for micro-positioning.

1. If the operator rotates the knob through an angle of $45^\circ$, how far does the crosshair travel on the display?
2. What knob rotation angle $\theta$ is required to slew the crosshair across a display distance of $12.0\text{ cm}$?

Solution:

Step 1: Calculate Display Movement for $45^\circ$ Rotation Convert angular rotation to radians: $\theta = 45^\circ \times \frac{\pi}{180^\circ} = \frac{\pi}{4} \approx 0.7854\text{ rad}$. Compute the physical circumferential displacement of the control surface: $\Delta C = R \cdot \theta = (3.0\text{ cm})(0.7854) = 2.356\text{ cm}$

Using the definition of C/D ratio: $C/D = \frac{\Delta C}{\Delta D} \implies \Delta D = \frac{\Delta C}{C/D} = \frac{2.356\text{ cm}}{2.5} = 0.942\text{ cm} = 9.42\text{ mm}$

Step 2: Calculate Required Knob Rotation for a $12.0\text{ cm}$ Display Travel $\Delta C = (C/D) \times \Delta D = 2.5 \times 12.0\text{ cm} = 30.0\text{ cm}$

Determine the required angular displacement: $\theta = \frac{\Delta C}{R} = \frac{30.0\text{ cm}}{3.0\text{ cm}} = 10.0\text{ radians}$ $\theta^\circ = 10.0\text{ rad} \times \frac{180^\circ}{\pi} \approx 572.96^\circ \approx 1.59\text{ full rotations}$

---

10. NCEES Reference Handbook Tips & Exam Traps

• Slip vs. Mistake Distinction: On the FE exam, pay close attention to operator intention. If the operator formulated the correct plan but inadvertently hit the wrong button or moved a lever too far, it is a slip (execution failure). If the operator intentionally hit the wrong button because they misdiagnosed the system state or misapplied a procedure, it is a mistake (planning failure).
• C/D Ratio vs. Sensitivity Trap: Sensitivity is the reciprocal of the C/D ratio ($\text{Sensitivity} = 1 / [C/D]$). A high C/D ratio corresponds to low sensitivity (coarse control movement yields small display travel, ideal for fine precision). A low C/D ratio corresponds to high sensitivity (small control movement yields large display travel, ideal for rapid slewing).
• Logarithm Bases in Human Performance Models: Hick-Hyman and Fitts' Law use base-2 logarithms ($\log_2$), representing information in bits. Using natural log ($\ln$) or common log ($\log_{10}$) without converting will yield incorrect results. Remember: $\log_2(x) = \frac{\ln(x)}{\ln(2)} = \frac{\log_{10}(x)}{\log_{10}(2)}$.
• Auditory Alarm Signal-to-Noise Requirement: An auditory alarm must be $15\text{ to }25\text{ dBA}$ ABOVE ambient noise, not an absolute value of $15 - 25\text{ dBA}$. If background factory floor noise is $82\text{ dBA}$, the alarm must be designed between $82 + 15 = 97\text{ dBA}$ and $82 + 25 = 107\text{ dBA}$.