14.1 Testing Methodology and Internal Controls

14.1 Testing Methodology and Internal Controls

Quick Answer: Task D.1 is a formal testing methodology—unit, integrated, stress, and acceptance—with owners, environments, and exit criteria. Task D.2 is internal control during the test: security audits, versioning, and change control so confidentiality, integrity, and availability still hold. A green click is not a method.

Information and Systems Management is 30% of CPHIMS. After selection and implementation (Chapter 13), the outline asks whether you can prove the solution before you trust it with patients. Testing is not a help-desk courtesy and it is not “the vendor already certified it.” It is how you refuse to promote a dose-range check, an ADT feed, or a revenue interface that has never been exercised as work-as-done.

CPHIMS practice questionsPractice questions with detailed explanations

Why a formal methodology, not a demo weekend

Healthcare systems fail at the seams. A single order set can look perfect while the pharmacy verify queue, the eMAR barcode, and the allergy interrupt disagree. A report can render in a lab tenant and time out when every clinic opens at 07:00. Operations can “like the screens” and still refuse the downtime path. CPHIMS stems hide the missing test inside a political sentence: “unit testing passed, go live Friday,” “UAT is optional because we are agile,” “just use production so the data look real.”

A formal testing methodology is a written plan that names:

1. Scope — which requirements, interfaces, and safety functions are in this cycle.
2. Types — unit, integrated, stress, acceptance, plus regression after a fix.
3. Environments — development, test/QA, sometimes a production-like staging bed; not an unlabeled laptop.
4. Data — synthetic or de-identified preferred; if a production extract is used, the same CIA controls apply.
5. Traceability — each must-have requirement maps to at least one test case and a recorded result.
6. Entry and exit criteria — what must be true to start, and what “pass” means.
7. Owners — who executes, who signs, who can stop the clock.
8. Defect rules — severity, retest, and what is a go-live blocker versus a punch-list item.

If you cannot point to that plan, you are hoping. Hope is not D.1.

The four types HIMSS names

Keep the four official examples distinct. Adjacent ideas (smoke, regression, usability, security test) support them; they do not replace them.

Unit testing belongs to the builder: the analyst, engineer, or vendor who changed one object. It is necessary and it is insufficient. Promoting on unit results alone is the most common CPHIMS trap in this task.

Integrated testing is the clinical and operational path. It needs the real partners in the chain—pharmacy verify, laboratory accession, HIM MPI, device middleware—not only the EHR tenant. Identity, units of measure, and acknowledgments are in scope (Chapter 12.1). A green engine queue is not an integrated test.

Stress (load / volume / performance) testing asks what happens at census, concurrent users, and message rate, not at a quiet Saturday. Medication barcode scanning at 09:00, radiology retrieve during a trauma, and eligibility checks at clinic open are healthcare stress cases. Stress also includes negative load: what happens when the interface is backlogged, the MPI is slow, or a site is on downtime viewers. Capacity theater (“we bought more CPU”) is not a measured test.

Acceptance testing is the sponsor’s and users’ formal statement that the implementation matches requirements and contracted scope. It is scripted against the approved design, not a scavenger hunt. Signatures matter because acceptance is how you later run task D.3 (contract and design validation). Informal “looks good” Slack messages are not acceptance.

Regression is the control you add whenever a defect fix or a late build lands: re-run the safety and path cases you already passed so a unit repair does not break the integrated chain. Regression is not a fifth official HIMSS example, but it is how you keep D.1 honest after D.2 change control lets a new version in.

Environments and test data

Promote through development → test (and staging if you have it) → production. Testers do not “try it in prod because the data are better” unless the organization has a tightly controlled production-validation window with a backout, and even then you do not use real patients as a sandbox.

Test data rules CPHIMS expects you to defend:

• Prefer synthetic or de-identified sets that still exercise edge cases (newborns, unknown sex, overlays, look-alike names).
• A production clone is ePHI. It needs a BAA if a vendor hosts it, access logs, encryption, and a destruction date. “It’s only test” is not a Privacy Rule exception.
• Mask or tokenize identifiers when the test purpose does not need them.
• Never leave a test extract on a shared drive, a contractor laptop, or an unsanctioned cloud folder.

Internal controls while the test is running (D.2)

Task D.2 is explicit: implement and monitor internal controls that protect resources and keep availability, confidentiality, and integrity during testing. HIMSS names the examples: security audits, versioning control, and change control.

Security audits during test ask who can reach the test bed, whether that access is unique and role-based, whether break-the-glass or admin IDs are logged, and whether a contractor’s “temporary” VPN still exists after the cycle. Audit the test environment the way you would audit a small clinic: joiner-mover-leaver, no shared “tester” login, and a review of exports. A test tenant that any vendor engineer can browse without a ticket is a confidentiality failure even if no patient has been treated in it yet.

Versioning control labels what was tested. Record the EHR version, the local build package, the interface map revision, the device firmware, and the test-script version. If two analysts fix the same order set overnight and UAT runs against an unlabeled mix, you cannot say what passed. Versioning is an integrity control: the artifact under test is the artifact you will promote.

Change control during test freezes silent edits. Mid-cycle “just one mapping change” without a ticket, a retest plan, and a new version label invalidates the cycle. The change-advisory or project change log is not bureaucracy; it is how you know the acceptance signature still attaches to the same system. Emergency fixes go through an expedited path with a documented regression set—not a hallway rebuild.

CIA during testing, in one line each:

• Confidentiality — least-privilege testers, no stray PHI copies, vendor access time-boxed.
• Integrity — labeled versions, no untracked edits, defects retested against the same script.
• Availability — load tests scheduled so they do not take down production; test jobs do not share the production interface engine without a window.

A security audit that only reads the production SIEM and ignores the test cloud has not met D.2.

Scenarios and exam traps

Scenario. Pharmacy unit-tested a new verify queue and declared the medication go-live ready. Nursing never walked a barcode from order to eMAR, and the interface map was edited twice during the week with no version label. Integrated testing is missing, and D.2 versioning/change control failed. Do not protect the date.

Scenario. Leadership wants “real data” so testers copy last month’s entire inpatient census to a vendor sandbox that has an NDA but no BAA and no access log. That is a PHI disclosure dressed up as quality assurance. Use a de-identified or synthetic set, or a BAA-covered, access-controlled clone with a destruction date.

Scenario. A load test is scheduled against the production interface engine on a Tuesday morning “because staging is stale.” Availability of care systems is a D.2 duty. Stress the production-like bed, or take a declared window with a backout—not a surprise flood during clinic open.

Scenario. Super-users spend an hour clicking favorite screens and send a “looks great” email. That is not acceptance. Acceptance uses scripts traced to requirements and a named operational signature.

Watch these traps:

1. Treating a passed unit test as integrated, stress, or acceptance.
2. Calling ONC certification or a vendor Connectathon a local test.
3. Testing in production to “save time.”
4. Copying identifiable charts into an uncontrolled test bed.
5. Editing the build mid-UAT without change control and a new version.
6. Using the go-live date as the exit criterion.

D.1 names the test. D.2 keeps the test honest. Section 14.2 then asks whether the thing you tested is also the thing you bought and whether anyone is better off.