Software, Firmware, and Spreadsheet Validation

Validation of Automated Calibration Software and Firmware

Automation can improve throughput and reduce manual transcription, but can also replicate a configuration or algorithm error across many instruments. Validate intended functionality and relevant data transfer, failure handling, and numerical behavior before release.

COTS Software vs. Custom Automated Test Scripts

1. Commercial Off-The-Shelf (COTS) Software: Review intended use, configuration, interfaces, and evidence of performance. General-purpose commercial software used within its designed range may be considered sufficiently validated under ISO/IEC 17025, but laboratory-specific formulas, scripts, configurations, and data paths still need appropriate checks. Installation and operational qualification can document those checks; ISO/IEC 17025 does not universally prescribe those named protocols or source-code inspection.
2. Custom Automated Scripts (Python, LabVIEW VIs, Keysight VEE, C#): In-house developed test automation requires documented validation for the intended use and controlled changes under ISO/IEC 17025 Clause 7.11:
   • Requirement Specification: Formal document defining the exact test points, mathematical algorithms, instrument bus commands, and pass/fail limits.
   • Algorithm Verification: Comparing software mathematical outputs against known reference datasets or manual hand calculations (e.g., verifying Steinhart-Hart equation conversions for thermistors or ITS-90 polynomial derivations for PRTs).
   • Boundary and Stress Testing: Testing the script under extreme edge cases: injecting maximum full-scale values, negative inputs, out-of-range signals, zero division, and instrument communication timeout disconnects.
   • Numerical Precision and Rounding: Ensuring software avoids premature rounding errors. Adequate numerical precision must be maintained throughout all intermediate calculations, applying rounding only to the final reported result and uncertainty.

Software Change Control and Version Management

Uncontrolled editing of test scripts is a major source of audit nonconformances. Laboratories must implement strict software configuration management:

Checksums can help identify a validated software version, but SHA-256 hashing and checking every run are not prescribed universal ISO requirements. Use an effective change-control and release process with documented validation, authorization, access controls, backup, and recovery. Record firmware configurations when they can affect the method.

---

Validation and Control of Calculation Spreadsheets

Spreadsheets (e.g., Microsoft Excel) are ubiquitous in calibration laboratories for uncertainty budgets, curve fitting, and data reduction. Under ISO/IEC 17025 Clause 7.11, spreadsheets are classified as software and are subject to mandatory validation and control.

Practical spreadsheet controls

1. Formula and Access Protection: Use effective permissions and release controls to prevent unauthorized or accidental changes to formulas, coefficients, limits, and macros. Distinguish authorized development from routine data entry, and revalidate relevant changes. Locked cells are one useful implementation.
2. Independent Calculation Benchmark: Compare outputs with independent calculations or suitable reference results for representative and boundary inputs. Retain the test cases, expected and observed results, conclusions, and approval evidence. A side-by-side worksheet is one possible record format, not a universal prescribed form.
3. Formula Error Trapping: Test the sheet against illegal entries: entering text into numeric cells, negative values into absolute pressure formulas, or zero into division terms, verifying that the spreadsheet flags errors rather than producing corrupted numbers.
4. Version Control and Revision Header: Every deployed spreadsheet can use a controlled header detailing:
   • Unique Document ID and Revision Level (e.g., CAL-SHT-042 Rev C)
   • Author Name and Date of Creation
   • Technical Manager Approval Signature and Date
   • Summary of Changes from previous revisions
   • SHA-256 File Hash or Read-Only Network Deployment

---

Common Operational Traps & CCT Exam Pitfalls

An editable formula without appropriate change controls creates an integrity risk. Assess the actual controls, unauthorized changes, and impact; finding classification follows the audit program. Cell locking is one useful safeguard, not the only permitted implementation.

A release test for automated measurements

An automated script must do more than calculate a nominal pass case. In a hypothetical voltage routine, independently check readings just below, at, and just above the specified acceptance limit. Verify negative inputs, units, range changes, overload responses, stale readings, and communication timeouts. An absent reading must be identified as an error rather than converted to zero and passed. Confirm that the device has actually settled and that an instrument command selects the intended function and range.

Use a known dataset to compare raw observations, corrections, specification bounds, uncertainty, and conformity statements with an independent calculation. Preserve enough precision to make decisions at the limit and then check final reporting. Verify that the output identifies the correct asset, method revision, date, units, coverage factor, and scope. Test recovery after interruption and ensure incomplete work cannot become a released certificate.

After a change to the tolerance formula, recheck affected ranges and previously verified boundary cases. After an instrument replacement or firmware update, assess command responses, timing, data formats, and any metrological change before resuming use. Archive the approved version and validation evidence with controlled access and backups. These tests evaluate actual failure modes; a checksum identifies a version but cannot establish that its algorithm is correct.