12.1 Evaluating Interventions with Data

12.1 Evaluating Interventions with Data

Quick Answer: Domain 3A asks case managers to systematically collect timely and accurate data to evaluate interventions. Choose measures tied to the care-plan or program goal, compare results to a baseline, confirm that planned services were actually delivered, and use iterative improvement (for example, PDSA) to continue, refine, or stop the intervention.

Domain 3—Process Improvement and Evaluation—accounts for only about 7 of 90 scored core items (~8%) on the ACM exam, yet it connects directly to how case managers prove value, prevent harm, and improve transitions. Blueprint item 3A centers on evaluating interventions with data. On the exam, expect scenarios about selecting trustworthy metrics, interpreting before/after results, and closing the evaluation loop after discharge—not about running a full biostatistics department.

Why Evaluation Is Part of Case Management

Case management is a cyclical process: assess → plan → coordinate/intervene → evaluate → revise. Without evaluation, the plan is only a set of orders and hopes. Evaluation answers three practical questions:

1. Did the intervention happen as intended (fidelity / implementation)?
2. Did outcomes move in the desired direction (effectiveness)?
3. What should change for this patient—or for the next cohort (improvement)?

ACMA Standards of Practice and the ACM content outline treat evaluation as a professional obligation, not an optional “nice to have” for quality staff alone. Front-line case managers contribute patient-level outcome review; program leaders aggregate those signals into unit and system improvement.

Timely and Accurate Data (Domain 3A Core Requirement)

The handbook outline emphasizes that data used to evaluate interventions should be timely and accurate. In practice:

Timely does not mean “real-time for everything.” It means the feedback cycle matches the risk. A high-risk heart-failure discharge needs post-discharge follow-up within days; a quarterly program scorecard may be adequate for system-level denial trends—if leadership still acts on the findings.

Accurate includes operational definitions. “Readmission” for a local CM dashboard must specify: all-cause vs same-cause, 7-day vs 30-day, planned vs unplanned, same hospital vs any hospital. CMS’s Hospital Readmissions Reduction Program (HRRP) uses risk-adjusted, all-cause unplanned 30-day readmissions for defined conditions/procedures—useful context for hospital programs, but your local evaluation measure still needs a written definition.

Example: After launching a heart-failure transition bundle, the case manager compares the unit’s 30-day all-cause unplanned readmission rate for HF discharges in the three months before the bundle with the three months after. Same definition, same data source, documented sample size—this is Domain 3A evaluation.

Outcome Measures Case Managers Actually Use

Choose measures that reflect care coordination and transition effectiveness:

Utilization and efficiency

• Length of stay (LOS) and observed-to-expected (O:E) LOS versus geometric mean LOS (GMLOS)
• Avoidable days / delays (e.g., waiting on auth, placement, diagnostics)
• Observation-to-inpatient conversion patterns when clinically relevant
• Denial rates and overturn rates (links to payer collaboration, Domain 2L)

Transition and quality outcomes

• 30-day readmission rate (often the clearest population signal for discharge-planning effectiveness)
• ED revisits after discharge
• Follow-up appointment completion within the planned window (e.g., 7 days for many high-risk medical discharges)
• Medication reconciliation completion and post-discharge medication access (filled prescriptions)

Patient-centered / plan-of-care measures

• Goal attainment (SMART goals met / partially met / not met)
• Functional status change (ADL/IADL, mobility)
• Patient-reported understanding (teach-back success) and experience items related to discharge information
• Caregiver readiness and burden when the home plan depends on informal care

Structure vs process vs outcome (Donabedian framing)

Strong evaluation pairs a process measure (did we deliver the bundle?) with an outcome measure (did readmissions fall?) and watches for balancing harms.

Evaluating at Two Levels

Patient-level evaluation

Before closing or handing off a case, review:

• Were goals met or revised with the patient/family?
• Were ordered post-acute services actually initiated (home health first visit, SNF bed secured, DME delivered)?
• Any new barriers (transportation, cost, caregiver collapse) requiring plan revision?

Ordering a service is not the same as implementing it. Finding that a home infusion never started is a critical evaluation insight—not a “successful” plan on paper.

Program / population-level evaluation

Aggregate data reveal patterns a single chart cannot: weekend discharges with higher readmissions, units with chronic pharmacy reconciliation delays, or SNFs with frequent early bounce-backs. Population findings feed Domain 3C risk work and Domain 2M throughput improvement.

PDSA Awareness for Case Managers

Plan-Do-Study-Act (PDSA) is a lightweight iterative cycle used across healthcare quality improvement. ACM candidates are not scored as Lean Black Belts, but they should recognize how PDSA supports Domain 3A:

1. Plan — Define the problem with data, set a measurable aim, pick one change to test, predict what will happen, and choose measures.
2. Do — Run a small pilot (one unit, one week, one patient cohort).
3. Study — Compare measured results to the prediction and baseline; note surprises.
4. Act — Adopt and spread, adapt and retest, or abandon; then cycle again.

Common PDSA mistakes that appear in exam distractors: changing several interventions at once so attribution is impossible; skipping measurement; declaring success from anecdotes; or jumping to hospital-wide rollout before a small test.

Example: Plan: reduce weekend HF readmissions by scheduling Monday follow-up calls for Friday/Saturday discharges. Do: pilot on one medicine unit for four weekends. Study: compare 7- and 30-day returns vs prior weekends. Act: if calls were completed and returns fell, spread to other units; if calls were not completed, fix staffing before expanding.

Closing the Loop and Documenting

Evaluation is incomplete until results drive a decision and documentation supports continuity:

• Continue effective interventions and standardize them.
• Modify when partial benefit or fidelity gaps appear.
• Abandon ineffective or harmful approaches.
• Document outcomes in the record so the team shares one source of truth for effectiveness and next steps.

Case managers also use evaluation findings to demonstrate program value to leadership—pairing readmission, LOS, avoidable-day, and denial metrics is far stronger than counting emails or contacts alone.

ACM Exam Focus

Expect Domain 3A items that reward: timely accurate data; outcome measures tied to the intervention (especially readmissions/LOS/avoidable days); before/after or goal-based evaluation; confirmation of service implementation; and PDSA-style structured improvement—not blame, marketing, or irrelevant operational trivia.