6.8 Measurement Systems Across the Organization, and Metrology

Measurement systems exist in every function

A measurement system is everything that produces a number: the instrument, the procedure, the person, the software, the environment, and the definitions. That description fits a coordinate measuring machine and it equally fits a credit risk scoring process. The Body of Knowledge names seven areas explicitly.

Two general rules follow.

Where the gauge is a person, the MSA is attribute agreement analysis. A quality-monitoring team scoring recorded calls is a measurement system whose repeatability is whether the same scorer gives the same call the same score twice, whose reproducibility is whether different scorers agree, and whose accuracy is agreement against an expert-established standard. Most service organizations have never measured any of the three.

Where the gauge is a database query, the MSA is definitional. If "on-time delivery" is computed by one query in logistics and a different query in customer service, the two functions have two different measurement systems and their disagreement will be attributed to the process rather than to the measurement.

A worked transactional example

A claims operation classifies each claim as "simple" or "complex" to route it. Six adjusters each classify the same 50 claims twice, and their classifications are compared with an expert panel's standard.

• Repeatability (within-adjuster agreement) is 88%.
• Reproducibility (between-adjuster agreement) is 71%.
• Accuracy against the standard is 66%.

The routing rule is unreliable, so roughly a third of claims are routed incorrectly, and any analysis that compares simple with complex claims is contaminated. The corrective action is definitional -- rewrite the classification rule with explicit decision criteria for the ambiguous cases -- not statistical.

Metrology

Metrology is the science of measurement and, in a quality system, the discipline that keeps measurement devices fit for purpose over time. The Body of Knowledge names three elements.

Calibration systems

Calibration compares an instrument against a reference of known accuracy and, where necessary, adjusts it. A calibration system specifies, for every measuring device in use:

• A unique identifier and a current status label showing the due date.
• The reference standard used and its own calibration status.
• The calibration interval and how it was determined.
• The acceptance criteria, usually expressed as a maximum permissible error.
• The environmental conditions required, where they matter.
• Records of every calibration, with as-found and as-left readings.

The as-found reading is the one that matters for risk. It records the instrument's condition before adjustment and is therefore the evidence of whether product measured since the last calibration was measured correctly.

Traceability to reference standards

Traceability is an unbroken chain of documented comparisons, each with a stated uncertainty, linking a working instrument to a recognized national or international standard.

International standard (SI definition)
        |
National metrology institute (e.g. NIST, NPL, PTB)
        |
Accredited calibration laboratory (ISO/IEC 17025)
        |
Company reference / master standard
        |
Working instrument on the shop floor

Every link must be documented, and uncertainty accumulates down the chain. The conventional guidance is that the reference should be substantially more accurate than the instrument it calibrates -- ratios of 4:1 or 10:1 in accuracy are common requirements -- so that the calibration uncertainty is negligible relative to the tolerance being verified.

A certificate that merely asserts "calibrated" without naming the reference standard, its calibration status, and the measurement uncertainty is not evidence of traceability.

Control and integrity of devices and standards

The system must also protect the devices themselves:

• Identification and status: every device labelled, in-service devices distinguishable from those awaiting calibration or withdrawn.
• Handling, storage, and transport: protection from shock, contamination, and environmental extremes; masters stored separately and used only for calibration.
• Adjustment safeguards: seals or access controls preventing adjustments that would invalidate the calibration.
• Damage and suspect-condition procedure: a dropped or suspect gauge is immediately withdrawn and recalibrated before reuse.
• Software and firmware control: version-controlled and validated, because a measurement algorithm is part of the measurement system.

Setting and adjusting calibration intervals

Intervals should be evidence-based, derived from observed drift history rather than convention. Inputs include the manufacturer's recommendation, criticality of the characteristic measured, usage rate and environment, and, most importantly, the record of as-found conditions.

The adjustment rule is straightforward: if devices routinely arrive at calibration well within tolerance, the interval may be extended; if out-of-tolerance conditions occur, it must be shortened.

Out-of-tolerance conditions

When an instrument is found out of tolerance at calibration, the calibration record alone is not sufficient. A documented impact assessment must follow:

1. Determine the period at risk: everything measured since the last acceptable calibration.
2. Quantify the error and compare it with the tolerance of the characteristics measured.
3. Identify the affected product, batches, or decisions.
4. Decide on containment: re-measure, review, notify the customer, or accept with justification.
5. Record the rationale and correct the underlying cause, which frequently means shortening the interval.

This is also why the Control phase includes measurement system reanalysis: as a process improves and its variation shrinks, a measurement system that was adequate at the baseline may consume an unacceptable share of the reduced tolerance.