21.2 ROI, Course Improvement and Training Reports

Jack Phillips' Level 5: Return on Investment (ROI)

Dr. Jack Phillips expanded Kirkpatrick's four-level model by adding Level 5: Return on Investment. Phillips' methodology translates isolated Level 4 results into financial terms and balances them against fully loaded program costs.

Core Formulas

$$\text{Benefit-Cost Ratio (BCR)} = \frac{\text{Total Monetary Benefits}}{\text{Total Program Costs}}$$ $$\text{Net Return on Investment (ROI \%)} = \frac{\text{Net Program Monetary Benefits}}{\text{Total Program Costs}} \times 100 = \frac{\text{Total Benefits} - \text{Total Costs}}{\text{Total Costs}} \times 100\%$$

• A $\text{BCR} > 1.0$ indicates that the economic benefits exceed the investment.
• An $\text{ROI} > 0\%$ indicates a positive financial return over and above recouping the initial investment.

Comprehensive Industrial Case Study: Chemical Plant Material Handling

This hypothetical example assumes 500 workers and 1,000,000 hours annually. The $80,000 intervention budget covers the stated training costs only; it does not include an equipment redesign. Every benefit below is a modeling assumption, not a measured causal effect.

1. Fully Loaded Training Program Costs

• Curriculum instructional design and external consultant fees: $12,000
• Trainer delivery hours (12 sessions x 4 hours at $75/hr): $3,600
• Trainee participant wages during instructional hours (500 workers x 4 hours at $30/hr): $60,000
• Training materials, job aids, and practice props: $4,400
• Total Loaded Program Cost: $80,000

Assumed benefits for the exercise

The example assumes 15 prior cases and five later cases at the same hours, with an assumed average direct cost of $12,000 per case. The modeled difference is 10 cases or $120,000. An assumed attribution factor of 80% yields $96,000; that percentage needs supporting evidence in a real evaluation. Assume separately itemized, nonoverlapping indirect savings of $115,200 for this exercise, giving $211,200 total modeled benefits. Do not infer a universal indirect-cost multiplier or double count costs.

Financial calculations from the assumed inputs

• Net modeled benefit: $211,200 minus $80,000 equals $131,200.
• BCR: $211,200 divided by $80,000 equals 2.64.
• Net ROI: $131,200 divided by $80,000, multiplied by 100, equals 164%.

These are modeled values for the exercise, not observed causal savings. A real analysis needs evidence for attribution, cost boundaries and benefits.

Continuous Improvement under ANSI/ASSP Z490.1

Use evaluation as a continuing improvement process: identify the gap, decide the change, implement it, then check whether it improves the intended outcome. Consensus guidance supports program evaluation; this guide does not attribute an unread clause or universal review interval to Z490.1.

1. Plan: Establish training needs assessments, performance learning objectives, and valid assessment criteria.
2. Do: Deliver instruction using qualified trainers, appropriate media, and high-fidelity practice.
3. Check: Collect and analyze multi-level evaluation data (Levels 1 through 5), audit compliance records, and calculate safety rates.
4. Act: Implement corrective curricular adjustments, retrain instructors, update job aids, and address workplace transfer barriers.

Review triggers

Review at defined organizational intervals and when procedures, hazards, requirements, assessment results or workplace feedback indicate a need. Assign ownership and preserve the evidence supporting each revision. The appropriate interval depends on risk and the program.

Executive Safety Reporting

To maintain executive support, CIT professionals must synthesize multi-level evaluation findings into executive summary dashboards. Reports should articulate strategic alignment, highlight leading indicators (behavioral audit compliance rates), demonstrate lagging metric reductions (TRIR and DART improvements), and validate fiscal stewardship via BCR and ROI figures.

Preparing a useful training report

A training report should lead with the decision needed: retain the course, revise an activity, add workplace support or investigate insufficient evidence. State the objective, population, course version, delivery dates and sources of evidence. Summarize reaction, learning, behavior and results separately enough that readers can see what each finding establishes.

Distinguish observed data, assumptions and recommendations. In the hypothetical ROI calculation, modeled benefits of $211,200 and costs of $80,000 yield a BCR of 2.64 and net ROI of 164%. The calculation is correct for those inputs, but the attribution and indirect savings are assumptions. Report that limitation prominently rather than calling the result measured cash return.

Include an action table with the finding, evidence, proposed change, owner, date and follow-up measure. For example, an observation pattern may justify revising specimen-selection practice, while unavailable references require an operational change. Assigning both to "more training" would conceal the different causes.

After implementation, check the planned outcome and document whether the change worked. A revised file, a completed meeting or an approving signature is evidence of activity, not necessarily improvement. Preserve the report and version relationship under the record policy so future reviewers can trace the decision.

Checking cost and benefit boundaries

The exercise's rates are assumed fully loaded inputs for the stated categories. In a real evaluation, check whether the reported cost includes development, delivery, learner time, administration, maintenance and required support. If a separate engineering intervention is part of the initiative, its cost and effect need an explicit boundary rather than being omitted while all savings are credited to training.

Benefits also need a documented basis. The exercise's indirect savings are an assumed amount, not a published ratio. A real report should show the nonoverlapping items supporting that estimate, their period and source. Avoid counting both an avoided repair invoice and the same invoice within a broader incident-cost estimate.

Sensitivity analysis can demonstrate uncertainty. If attribution is poorly established, report alternative modeled values or decline to claim a measured ROI. A mathematically correct percentage is only as sound as its inputs. Keep intangible outcomes such as improved communication distinct from monetized benefits unless a defensible valuation is available.

A training report can recommend an important improvement without a positive financial estimate. Required training and essential hazard controls do not depend solely on financial return. State the relevant obligation, learning evidence and operational need, then describe what the organization should do and what will be checked afterward. The report's purpose is a supported decision, not the highest possible ROI number.

Key takeaways

• Net ROI subtracts program cost before dividing by cost; BCR uses total benefits.
• Modeled attribution and indirect savings must be labeled as assumptions.
• Training reports link findings to decisions, owners and follow-up evidence.