16.1 Evaluating Performance Against SLAs and Indicators

16.1 Evaluating Performance Against SLAs and Indicators

Quick Answer: Domain 4 task A.5 is evaluate performance—for example against SLAs, goal/performance indicators, and systems effectiveness. Chapter 11 taught you to read an SLA. This task is whether the service, the goal, and the system actually performed. A vendor credit is not restored care.

Management and Leadership is 25% of CPHIMS—about one scored item in four on the 100-scored-item, two-hour form. Chapter 15 put HIT inside the organizational plan. This chapter asks whether last quarter’s promises held. Chapter 13 already taught you to read tickets, errors, and network monitors as one operations picture (tasks C.5 and C.6). A.5 is the management evaluation: compare evidence to a written service level or indicator, name what was excluded, and decide whether the process or system was effective—not merely up.

CPHIMS practice questionsPractice questions with detailed explanations

Why evaluation is not the contract and not the dashboard

Task A.5 sits after the documents exist. The MSA, SOW, and SLA were signed (chapter 11). The IT plan named departmental objectives (section 15.4). Operations is paging the help desk (chapter 13). Evaluation is the compare step:

1. What was the written promise or goal?
2. How is it measured, over what window, by whom?
3. What does the evidence show, including residuals and exclusions?
4. Did the service, the indicator, and the system do the intended job?
5. What changes: operations, contract remedy, process, design, or the metric itself?

If you only collect charts, you are monitoring. If you only send a credit request, you are administering a clause. A.5 is judging performance and effectiveness so leaders can keep, kill, or reshape the service.

HIMSS does not publish a CPHIMS-official SLA catalog, KPI library, or composite “systems effectiveness” score. Use the contract the organization signed and the indicators the organization governs. Inventing a HIMSS index that averages uptime with harm is not professional practice.

Three lenses the exam will not let you mash together

Keep the labels honest:

• An SLA is how well an ongoing service must perform, plus measurement method, exclusions, and remedies. Internal SLAs (help desk, interface bureau, imaging archive) count. Vendor SLAs count. A slogan on a sales slide does not.
• A goal or performance indicator is a written target with an owner and grain. “Improve the EHR” is not one. “Median new-patient wait from 38 days to 21, ambulatory operations owner, completed new-patient visits” is one.
• Systems effectiveness is outcome-shaped: did barcode medication administration actually close the loop, or did nurses type around it? Did the claims edits stop the denials the business case named?

Chapter 15 already banned substituting uptime for an access aim. A.5 applies the same honesty to service contracts and system purpose.

How to evaluate an SLA without getting played

Read the clause the way chapter 11 taught, then score the period:

1. Quote the metric and the method. User-transaction success is not the same as a vendor ICMP ping. “Messages delivered to the interface engine” is not “ADT bound to the enterprise MPI.”
2. Name the window. Calendar month, rolling 30 days, or business hours only. A 4-hour Tuesday outage inside a 99.9% monthly target can still “meet” the SLA and still fail the hospital.
3. Apply exclusions last, in writing. Planned maintenance, third-party networks, “factors beyond our control,” and unlimited change freezes are where green numbers are born. Excluded time is not secret success.
4. Separate credit from restoration. Exclusive-remedy clauses often cap the conversation at a small invoice credit and a five-day claim window. The credit is a contract fact. It is not evidence that emergency diversion, canceled clinics, or lost results were acceptable performance.
5. Score internal SLAs the same way. If the service desk promised 80% first-contact resolution and you only report average speed of answer, you evaluated a different service.

A practical A.5 SLA packet: target, method, inclusions, exclusions, actual, variance, user-visible impact, remedy used, and the residual the credit does not fix.

Goal indicators and systems effectiveness

Evaluate indicators the way you measured organizational goals in section 15.1:

• Quote the numbered target and owner.
• Use organizational grain (discharge, attributed member, completed encounter)—not portal logins unless the plan defined logins as the measure.
• Publish residual honesty (unmatched identities, missing encounters, workarounds).
• Time the window after the process changed. Go-live is a milestone, not effectiveness.

Evaluate systems effectiveness by the job in the business case or design, not by install completeness:

• Medication-use system: scan rates and override reasons and known workarounds, not “module live.”
• Revenue cycle: first-pass yield and denial reasons the edits were supposed to catch, not “claims file sent.”
• Identity: match residual and false-merge rate, not “EMPI installed.”
• Imaging or lab: results available in the chart in the promised window, not “interface green.”

If nurses built a paper parallel process, the system is not effective even if the SLA is green. If the only people who can complete the workflow are three super-users, effectiveness is not enterprise-wide.

Distinctions the exam will punish

• Write the SLA (chapter 11, Domain 3) versus evaluate performance against it (A.5).
• Met the SLA versus service was usable when clinicians needed it.
• Indicator movement versus activity (tickets closed, emails sent, training hours).
• Systems installed versus systems effective.
• Vendor credit versus restored operations and residual harm or delay.
• One green monitor versus the combined picture chapter 13 already required.

Scenarios and exam traps

Scenario. The hosting SLA is 99.95% monthly availability measured by vendor pings. The month scores 99.96%. CPOE timed out for 90 minutes during morning rounds because application sessions failed while the ICMP check stayed green. Report the SLA as met by its method, and report systems effectiveness as failed for ordering. Do not let the ping colonize the effectiveness sentence.

Scenario. A four-hour imaging-archive outage diverts two stroke workups. The vendor tenders a $400 service credit under an exclusive-remedy clause and a five-day claim window. File the credit on time. Then evaluate performance: the service was not effective for emergency imaging, the residual is diverted care, and the next decision is operational workaround plus contract or architecture change—not “the SLA handled it.”

Scenario. The interface SLA counts “messages delivered to the engine.” Delivery is 99.8%. Eight percent of ADT events never bind to the enterprise MPI. Identity matching is a systems-effectiveness miss and likely a goal-indicator miss for any registry that assumes a unique person. Delivered-but-unmatched is not success.

Scenario. The departmental indicator is 80% first-contact resolution on the service desk. Leadership wants a single chart of tickets closed. Tickets closed is volume. Resolution on first contact is the written service indicator. Evaluate the indicator you published, or change the indicator in governance—do not silently swap.

Watch these traps:

1. Treating a met SLA as proof the system was effective.
2. Measuring a vendor ping, a load-balancer check, or “file sent” when the clause or the user job was a transaction, a match, or a result in the chart.
3. Calling a credit memo restored performance.
4. Substituting activity (tickets, training hours, go-live dates) for goal indicators.
5. Hiding exclusions so a red month looks green.
6. Inventing a HIMSS-official CPHIMS effectiveness index.

/practice/cphimsPractice questions with detailed explanations