12.3 Analyzing POE Data, Isolating Environmental Effects & Mixed Outcomes

Analyzing POE Data, Isolating Environmental Effects & Mixed Outcomes

Core Principle: In laboratory science, researchers isolate independent variables by holding all external factors constant. In healthcare architecture, post-occupancy evaluations take place in messy, volatile socio-technical systems. A hospital is an open environment where physical architecture constantly intersects with clinical protocols, staffing ratios, technology rollouts, administrative policies, and community disease burdens. Analyzing POE data demands sophisticated methodological techniques to disentangle environmental effects from operational confounders, alongside a scientific framework to diagnose mixed, null, or contradictory outcomes.

Novice design teams frequently commit the error of naive attribution: observing a 20% drop in patient falls after opening a new hospital tower and immediately proclaiming that the architectural layout cured the problem. Conversely, if infection rates rise post-occupancy, leadership may hastily declare the design a failure. In rigorous Evidence-Based Design, correlation is never assumed to be causation. The EDAC-certified professional must possess the analytical rigor to isolate architectural contributions from confounding operational noise.

---

The Anatomy of Healthcare Confounders

When evaluating post-occupancy performance against Step 6 baselines, researchers encounter four pervasive categories of confounding variables that can mimic, mask, or distort environmental effects:

┌────────────────────────────────────────────────────────────────────────┐
│               FOUR CATEGORIES OF HEALTHCARE POE CONFOUNDERS            │
├───────────────────────────────────┬────────────────────────────────────┤
│ 1. OPERATIONAL & STAFFING         │ 2. CLINICAL & PROTOCOL BUNDLES     │
│    • Nurse-to-patient staffing    │    • New fall prevention programs  │
│      ratio changes (e.g., 1:4->1:5)│      (Purposeful Hourly Rounding) │
│    • Shift length (8-hr to 12-hr) │    • Central line insertion bundles│
│    • High agency/traveler nurse % │    • Barcode scanning mandates     │
│    • Leadership / CNO turnover    │    • Antimicrobial stewardship     │
├───────────────────────────────────┼────────────────────────────────────┤
│ 3. TECHNOLOGICAL & SYSTEMS        │ 4. PATIENT POPULATION & EPIDEMIC   │
│    • Concurrent EHR transitions   │    • Shifts in Case Mix Index (CMI)│
│      (e.g., Epic/Cerner go-live)  │    • Patient age & comorbidity shifts│
│    • Smart-bed sensor rollouts    │    • Seasonal viral surges         │
│    • Automated dispensing units   │      (Influenza, RSV, COVID-19)    │
│    • VoIP clinical smartphone push│    • Summer trauma census surges   │
└───────────────────────────────────┴────────────────────────────────────┘

1. Operational and Staffing Confounders

Human capital dynamics exert enormous influence over clinical outcomes. If a health system relocates to a new inpatient tower with 100% single rooms but simultaneously cuts nurse staffing ratios from 1:4 to 1:6 due to budgetary pressures, call-bell response times will lengthen and fall rates may spike. The degradation in safety is not an architectural failure of single rooms; it is an operational failure of nurse staffing capacity. Similarly, relying heavily on temporary travel nurses unfamiliar with unit policies can artificially inflate medication error rates.

2. Clinical Protocols and Quality Bundles

Hospitals constantly roll out clinical quality improvement initiatives. If an ICU transitions to a new decentralized design and simultaneously implements a chlorhexidine gluconate (CHG) patient-bathing protocol and an ultrasound-guided central line insertion bundle, central line-associated bloodstream infections (CLABSI) will plummet. Attributing this clinical triumph solely to the new architectural layout represents severe attribution error. The research design must isolate what proportion of infection reduction derived from single-room isolation versus clinical asepsis bundles.

3. Technological and Information System Cutover

Capital hospital projects frequently bundle architectural construction with massive health IT migrations. Transitioning to a new Electronic Health Record (EHR) system, rolling out smart beds with integrated scale sensors, or issuing clinical VoIP smartphones during the post-occupancy window drastically alters staff documentation time, communication patterns, and alarm acoustics. Researchers must account for technological implementation curves when evaluating nurse travel and bedside care time.

4. Patient Acuity and Epidemiological Shifts

A hospital's Case Mix Index (CMI) quantifies the clinical complexity and resource intensity of its inpatient population. If an acute care hospital closes an affiliated regional subacute facility, the average CMI of the remaining inpatient units may rise from 1.45 to 1.85. Higher acuity patients have significantly greater baseline risks of delirium, unassisted falls, and secondary infections. Comparing raw post-occupancy fall rates against baseline without risk-adjusting for CMI and age distributions severely skews findings.

---

Quasi-Experimental Methodologies for Isolating Environmental Effects

Because healthcare executives cannot ethically randomize sick patients into poorly designed versus well-designed hospital rooms, EBD researchers rely on quasi-experimental research designs. Two methodologies serve as the gold standard for neutralizing confounders:

┌────────────────────────────────────────────────────────────────────────┐
│                     INTERRUPTED TIME SERIES (ITS)                      │
├────────────────────────────────────────────────────────────────────────┤
│  Outcome Rate (e.g., Falls/1,000 Patient Days)                         │
│       │                                                                │
│       │  Pre-Occupancy Baseline Trend                                  │
│       │  \                                                             │
│       │   \                                                            │
│       │    \   Level Change (Immediate Drop)                           │
│       │     ▼   │                                                      │
│       │         ▼  Post-Occupancy Slope Change                         │
│       │            \                                                   │
│       │             \                                                  │
│       │              \─────────────────────────────────────────────    │
│       │                  Move-In Date                                  │
│       └───────────────────────┼────────────────────────────────────    │
│        Month: -12  -6   -1    0   +1   +6   +12  +18  +24              │
└────────────────────────────────────────────────────────────────────────┘

1. Interrupted Time Series (ITS) Analysis

Interrupted Time Series (ITS) analysis is one of the strongest quasi-experimental designs available for evaluating healthcare environments. Instead of comparing a single pre-move data point against a single post-move data point, ITS tracks outcome metrics across dozens of continuous, equidistant time intervals (e.g., 24 continuous months pre-move and 24 continuous months post-move).

Statistical segmented regression models quantify two distinct parameters:

• The Level Change (Immediate Step Change): The immediate, vertical shift in the outcome rate observed at the exact moment of facility occupancy.
• The Slope Change (Trajectory Change): The alteration in the longitudinal trend (rate of change over time) following occupancy compared to the pre-move baseline slope.

ITS controls for pre-existing secular trends. If hospital fall rates were already declining by 0.5% per month due to an ongoing nurse-rounding campaign before the move, ITS separates that continuous background trend from the acute step-change or accelerated slope caused by the new physical architecture.

2. Matched Non-Equivalent Control Units (Difference-in-Differences)

When a health system renovates a specific unit (e.g., renovating the 4th-floor cardiac stepdown unit into acuity-adaptable rooms while leaving the 5th-floor neuro stepdown unit in its legacy multi-bed configuration), researchers can deploy a matched control group design utilizing Difference-in-Differences (DiD) estimation.

┌────────────────────────────────────────────────────────────────────────┐
│             MATCHED CONTROL UNIT / DIFFERENCE-IN-DIFFERENCES           │
├────────────────────────────────────────────────────────────────────────┤
│  Target Outcome Metric                                                 │
│       │                                                                │
│       │                         Unrenovated Control Unit (Same Hospital)│
│       │                         ------------------------------------   │
│       │                        / (Reflects hospital-wide protocol      │
│       │                       /   changes, seasonal flu, CMI shifts)   │
│       │  Renovated Unit      /                                         │
│       │  -------------------/   Difference-in-Differences              │
│       │                    \    [Estimated Design Effect]              │
│       │                     \                                          │
│       │                      \─────────────────────────────────────    │
│       │                        Post-Occupancy Intervention Unit        │
│       └───────────────────────┼────────────────────────────────────    │
│                              Move-In Date                              │
└────────────────────────────────────────────────────────────────────────┘

By comparing the change in the intervention unit with the change in the control unit, researchers account for hospital-wide influences—seasonal respiratory surges, system-wide EHR upgrades, and institutional policy shifts—that affect both units similarly. If the two units followed similar trends before the change, a difference that emerges afterward is a stronger estimate of the built environment's contribution, although unit-specific changes can still confound it.

---

Dissecting Mixed, Null, and Contradictory POE Outcomes

In real-world healthcare research, empirical data rarely confirms every design hypothesis cleanly. Post-occupancy evaluations routinely produce mixed outcomes (some hypotheses supported, others refuted), null outcomes (no statistically significant change), or contradictory outcomes (the outcome moved in the opposite direction of the hypothesis).

Far from representing project failures, mixed outcomes provide valuable learning. The hypothetical exemplars below show why specific design typologies can produce complex, conflicting results:

Exemplar 1 (Hypothetical): Decentralized Nursing Stations (The Travel vs. Isolation Paradox)

• The Step 5 Hypothesis: If decentralized nurse charting alcoves are installed immediately adjacent to patient rooms, nurse walking distance will decrease and bedside care time will increase, because travel latency between central stations and patient beds is eliminated.
• The Step 8 Quantitative Outcome: RTLS tracking data confirmed the hypothesis: nurse walking distance decreased by 26% (from 8.2 km to 6.1 km per shift), and bedside clinical time increased by 18%.
• The Unintended Contradictory Outcome: Validated staff surveys revealed a 35% surge in registered nurse perceptions of social isolation, a 28% drop in teamwork cohesion scores, and increased communication breakdowns during critical patient decompensation events.
• The Mechanistic Diagnosis: While decentralized alcoves optimized physical locomotion, they dismantled the informal socio-spatial hub of the centralized nurses' station. Junior nurses lost passive auditory surveillance and spontaneous peer mentoring from senior colleagues. Clinicians felt isolated in decentralized corridors, unable to locate peer backup quickly when emergencies erupted.

Exemplar 2 (Hypothetical): Single-Patient Inpatient Rooms (The Infection vs. Fall Paradox)

• The Step 5 Hypothesis: If all semi-private rooms are replaced with 100% single-patient private rooms, Hospital-Acquired Infections (HAIs) and inpatient fall rates will decrease, because private rooms prevent environmental cross-contamination and provide dedicated space for family support.
• The Step 8 Quantitative Outcome: NHSN data validated the infection hypothesis: contact-transmitted pathogens (C. diff and MRSA) dropped by 38%. However, NDNQI fall data revealed a 22% increase in patient falls with injury.
• The Unintended Contradictory Outcome: In semi-private rooms, roommates or visiting family members frequently functioned as secondary surveillance, pressing the call light when a delirious patient attempted to climb out of bed. In single rooms behind closed acoustic doors, patient unassisted bed exits went unnoticed. Furthermore, if the architectural layout positioned the patient toilet on the far wall opposite the bed without continuous handrail guidance, unassisted nocturnal transfers resulted in severe falls.

Exemplar 3 (Hypothetical): Acuity-Adaptable Bed Units (The Universal Room Operational Mismatch)

• The Step 5 Hypothesis: If universal acuity-adaptable rooms are built to eliminate intra-hospital patient transfers, medication reconciliation errors and length of stay will decrease, because patient handoffs between ICU and stepdown teams are eliminated.
• The Step 8 Quantitative Outcome: Length of stay showed no statistically significant change (null result), and nurse turnover spiked by 15%.
• The Mechanistic Diagnosis: The physical space was built flawlessly, but the hospital failed to execute the required clinical organizational restructuring. Medical-surgical nurses lacked intensive care competency training, while ICU nurses resisted caring for lower-acuity stepdown patients. Cross-training programs stalled, forcing the hospital to move clinical teams to the patient—creating scheduling chaos and administrative friction that negated architectural flexibility.

---

Secondary Qualitative Diagnostics: The Three Failure Typologies

When a post-occupancy evaluation reveals that an EBD hypothesis was not supported, researchers must not abandon the inquiry. Quantitative metrics identify the statistical discrepancy, but secondary qualitative diagnostics (staff shadowing, behavioral mapping, and root-cause focus groups) are required to categorize why the hypothesis failed.

A useful way to classify why a hypothesized outcome was not achieved (this guide's framework) is to sort failures into three typologies:

┌────────────────────────────────────────────────────────────────────────┐
│                     THE THREE EBD FAILURE TYPOLOGIES                   │
├───────────────────────────────────┬────────────────────────────────────┤
│ 1. IMPLEMENTATION FAILURE         │ 2. OPERATIONAL / WORKAROUND        │
│    • Physical design compromised  │    • Space built correctly, but    │
│    • Value Engineering cuts       │      human workflow bypassed it    │
│    • Contractor substitutions     │    • Staff workarounds negate      │
│    • Acoustic/lighting spec drift │      the intended design feature   │
├───────────────────────────────────┴────────────────────────────────────┤
│ 3. THEORETICAL / CONCEPTUAL FAILURE                                    │
│    • The underlying mechanistic hypothesis was fundamentally flawed    │
│    • Spatial intervention cannot solve an organizational culture issue │
└────────────────────────────────────────────────────────────────────────┘

---

Environmental Workarounds: The Canary in the Coal Mine

During Step 8 qualitative walkthroughs, researchers frequently observe environmental workarounds—informal, unauthorized modifications made to the physical space by occupants trying to circumvent architectural barriers.

In Evidence-Based Design, workarounds are not viewed as staff insubordination; they are the premier diagnostic indicator of an unresolved spatial-operational mismatch:

• Taped Sensor LEDs and Wall Alarms: Clinical staff placing adhesive tape or gauze over flashing call-bell dome lights or wall-mounted acoustic monitor LEDs, signaling extreme sensory alarm fatigue.
• Propped Fire and Corridor Doors: Wedging trash cans or linen carts into heavy cross-corridor doors, signaling that automated magnetic hold-opens are malfunctioning or that opening heavy doors impedes rapid clinical emergency response.
• Ad-Hoc Paper Signage ("Post-It Note Architecture"): Handwritten laminated notes taped to walls ("DO NOT STORE CHAIRS HERE", "CLEAN UTILITY ONLY", "PUSH CODE BUTTON TWICE"), signaling severe wayfinding breakdowns, ambiguous spatial zoning, or non-intuitive technology interfaces.
• Corridor Cart Clusters: Pushing mobile medication carts (WOWs) and supply towers out of patient alcoves into central hallway clusters, signaling that decentralized nurses are seeking peer social interaction and visual contact.