Conformity Decisions and False-Accept Risk

When a customer submits an instrument to a calibration laboratory, the primary commercial expectation is often a simple binary verdict: "Pass" (In-Tolerance) or "Fail" (Out-of-Tolerance). However, because every measurement result is inherently accompanied by measurement uncertainty, declaring conformity is fundamentally a probabilistic decision under conditions of imperfect knowledge. A measured reading that lies just inside a tolerance limit might represent a physically non-conforming instrument whose true value lies outside the boundary; conversely, a measured reading just outside the limit might represent a perfectly conforming instrument.

To manage this uncertainty and prevent commercial and technical conflicts, modern metrological standards establish formalized decision rules and guard-banding methodologies.

---

Conformity Assessment and ISO/IEC 17025:2017 Clause 7.8.6

Clause 7.8.6 of ISO/IEC 17025:2017 establishes specific accreditation requirements whenever a calibration laboratory provides a statement of conformity to a specification or standard:

1. Mandatory Documentation of the Rule: The laboratory must document the exact decision rule employed, taking into account the level of risk (such as false accept and false reject probabilities) and the measurement uncertainty.
2. Prior Customer Agreement: The decision rule must be communicated to and agreed upon with the customer before work, unless inherent in the requested specification or standard (typically during the contract review phase under Clause 7.1).
3. Transparent Reporting on the Certificate: The calibration certificate must clearly identify:
   • The specific statement of conformity (e.g., Pass, Fail, Indeterminate).
   • Which specification, standard, or clause was evaluated.
   • The decision rule applied (including any guard-banding formulas or reference standards such as ILAC-G8:09/2019 or ASME B89.7.3.1).

Per ILAC-G8:09/2019 (Guidelines on Decision Rules and Statements of Conformity), a decision rule is defined as:

"A rule that describes how measurement uncertainty is accounted for when stating conformity with a specified requirement."

---

Consumer's Risk vs. Producer's Risk: The Statistical Dilemma

In statistical hypothesis testing and calibration conformity assessment, decision errors are classified into two classical categories:

Consumer's Risk (Type II Error / $\beta$ / False Acceptance)

Consumer's Risk, quantified as the Probability of False Acceptance (PFA), is the probability that an instrument or product is accepted as conforming ("Pass") when its true property value actually violates specification limits.

• Physical Cause: The true value is outside the tolerance band, but due to measurement error and uncertainty, the instrument indicates a value inside the acceptance zone.
• Practical Impact: Defective, out-of-tolerance test equipment is deployed into active service. In aerospace, military, medical device, and nuclear industries, false acceptance can lead to undetected product failures, flight control malfunctions, medical misdiagnoses, and catastrophic safety hazards.
• Specific Requirements: Some governing systems, including ANSI/NCSL Z540.3, impose risk requirements within their scope. Consult the actual applicable document and risk methodology. Do not infer a universal 2% guarantee from a 4:1 TUR or apply one system’s requirement to every ISO/IEC 17025 calibration.

Producer's Risk (Type I Error / $\alpha$ / False Rejection)

Producer's Risk, quantified as the Probability of False Rejection (PFR), is the probability that an instrument or product is rejected as non-conforming ("Fail") when its true property value is actually within specification limits.

• Physical Cause: The true value is in tolerance, but measurement uncertainty pushes the observed reading outside the acceptance limit.
• Practical Impact: Conforming equipment is needlessly condemned, scrapped, adjusted, or sent for expensive rework. In manufacturing, excessive Producer's Risk creates severe economic waste and false out-of-tolerance investigations.

---

Simple Acceptance and the Shared Risk Rule

The most widespread historical decision rule is Simple Acceptance (often termed the Shared Risk Rule under ILAC-G8 and ASME B89.7.3.1).

Operational Definition:

Under simple acceptance, the Acceptance Limits ($A_L$) are set exactly equal to the Tolerance Limits ($TL$):

$$A_{L,\text{upper}} = \text{USL}, \quad A_{L,\text{lower}} = \text{LSL}$$

• If the measured value falls inside the tolerance limits ($y \in [\text{LSL}, \text{USL}]$), the instrument is declared Pass (In-Tolerance).
• If the measured value falls outside the tolerance limits ($y < \text{LSL}$ or $y > \text{USL}$), the instrument is declared Fail (Out-of-Tolerance).

The Shared Risk Assumption:

Simple acceptance can involve shared risk, but the applicable rule must be communicated and agreed with the customer unless it is inherent in the requested standard or specification. Do not infer customer agreement merely from performing the calibration.

The 50% False Acceptance Trap at the Boundary:

Simple acceptance can be an agreed choice, with its risk understood. A high TUR reduces the width of the uncertain boundary region but does not eliminate risk at the boundary. However, consider what happens when a measured value lies directly on the specification limit ($y = \text{USL}$):

• Assuming the measurement uncertainty follows a symmetric Gaussian (normal) probability density function centered at the measured value $y$, the area under the curve is split equally:
• Approximately half the symmetric distribution lies beyond this boundary, with the opposite boundary assumed sufficiently remote.
• Therefore, declaring a "Pass" when $y = \text{USL}$ carries an specific false-accept risk of approximately 50% under that model!

Lower TUR widens the region where uncertainty can affect the decision. Whether the resulting risk is acceptable depends on the agreed rule, model, and consequences; a ratio alone does not set a universal risk verdict.