7.1 Measurement System Fundamentals: Precision, Accuracy & Bias

Measurement System Fundamentals: Precision, Accuracy & Bias

Quick Answer: Measurement System Analysis (MSA) evaluates data collection tools, fixtures, and operators to ensure observations reflect true process behavior rather than measurement noise. In the Six Sigma Measure phase, total observed variation is partitioned into process variation and measurement error ($\sigma^2_{total} = \sigma^2_{process} + \sigma^2_{measurement}$). Metrology assesses accuracy (systematic error: bias, linearity, and stability) and precision (random error: repeatability and reproducibility). Under the 10-to-1 Resolution Rule, an instrument's smallest readable graduation must not exceed 10% of the specification tolerance or process spread. Independent CSSYB study guide by OpenExamPrep.

Purpose of MSA in the Measure Phase

In Lean Six Sigma, data informs every root-cause hypothesis and capability calculation. However, raw process data never reflects pure process performance alone. Every observation combines true part-to-part variation with error introduced by instruments, fixtures, environment, and human handling.

Total observed variance ($\sigma^2_{total}$) is the mathematical sum of true process variance ($\sigma^2_{process}$) and measurement system variance ($\sigma^2_{measurement}$):

$$\sigma^2_{total} = \sigma^2_{process} + \sigma^2_{measurement}$$

Validating the measurement system is a mandatory gate in the DMAIC Measure phase. If measurement noise is excessive, two operational risks arise:

1. Producer's Risk (Type I Error / False Alarm): Conforming product is mistakenly rejected because measurement noise pushes recorded values outside specification limits, causing wasteful scrap and unwarranted machine adjustments.
2. Consumer's Risk (Type II Error / Missed Defect): Nonconforming product is mistakenly accepted and released because measurement error pulls out-of-spec dimensions inside tolerance boundaries.

A capable measurement system ensures that observed variability reflects actual process realities rather than testing noise.

---

Precision vs. Accuracy: The Metrological Distinction

Precision and accuracy represent fundamentally different types of measurement error that must be evaluated separately:

• Accuracy (Location / Systematic Error): The closeness of agreement between the average of observed measurements and an accepted, certified reference master value. Accuracy errors are systematic offsets caused by uncalibrated zero points, physical wear, or uncorrected thermal expansion.
• Precision (Spread / Random Error): The closeness of agreement among repeated measurements on the same part under identical conditions. Precision reflects random variability caused by mechanical play, electrical noise, or minor operator seating differences, completely independent of the true reference value.

---

Core MSA Properties: Bias, Linearity, and Stability

Measurement system accuracy is evaluated across three primary dimensions:

1. Bias (Accuracy at a Single Point)

Bias is the quantitative difference between the observed average of measurements and a certified reference master value:

$$\text{Bias} = \bar{X}_{observed} - X_{reference}$$

To assess bias, a certified master standard (such as a calibrated gauge block) is measured 10 to 15 times under production conditions. A one-sample $t$-test determines if the sample mean ($\bar{X}$) differs significantly from the reference value ($X_{reference}$) at $\alpha = 0.05$. Significant bias requires physical calibration or digital zero-offset adjustment.

2. Linearity (Accuracy Across the Operating Range)

Linearity evaluates whether bias remains constant across the instrument's entire operating range. A gauge may have zero bias when measuring small components but develop substantial bias when measuring larger parts.

To assess linearity, practitioners measure 3 to 5 reference masters covering 10% to 90% of the gauge's scale. Observed bias is plotted against reference values, and a linear regression line is fitted:

$$\text{Bias} = b_0 + b_1 \times (\text{Reference Value})$$

If the regression slope ($b_1$) is statistically indistinguishable from zero, linearity is acceptable. A significant slope indicates that the instrument requires multi-point span recalibration or software correction.

3. Stability / Drift (Accuracy Over Extended Time)

Stability represents the total variation in measurements obtained with a measurement system on the same master part over an extended timeframe.

Over weeks or months, gauge accuracy can degrade due to mechanical wear, sensor aging, or ambient temperature cycling. Stability is monitored by measuring a reference master at scheduled intervals (such as daily or once per shift) and plotting the results on an Individuals and Moving Range ($I-MR$) or $\bar{X}-R$ control chart. Any out-of-control signals indicate that the system has drifted and requires maintenance or recalibration.

---

Resolution and Discrimination: The 10-to-1 Rule

Resolution (also called discrimination or readability) is the smallest detectable unit or graduation that a measuring device can distinguish and display.

When an instrument lacks adequate resolution, observations cluster into coarse, identical numerical steps. This discretization masks true variation, produces stepped histograms, and inflates standard deviation calculations.

The 10-to-1 Rule (Rule of Ten / Gage Maker's Rule)

To ensure sufficient discrimination, the instrument's smallest increment of measurement must be at most one-tenth (10%) of the process tolerance width (USL - LSL) or one-tenth of six-sigma process variation ($6\sigma_{process}$):

$$\text{Maximum Allowable Resolution} \le \frac{\text{Tolerance Width}}{10} = \frac{\text{USL} - \text{LSL}}{10}$$

For example, if a machined pin has an engineering tolerance of $20.00 \pm 0.05\text{ mm}$ (tolerance width $= 0.10\text{ mm}$), the gauge resolution must be no coarser than $0.10 / 10 = 0.01\text{ mm}$ (or finer, such as $0.001\text{ mm}$). A tool with $0.05\text{ mm}$ resolution fails the 10-to-1 Rule and cannot reliably track process performance.

---

Calibration Systems and Metrological Traceability

Calibration is the formal comparison of an instrument against an uncompromised standard of known accuracy to detect, report, and eliminate systematic error.

Six Sigma programs require metrological traceability—an unbroken, documented chain of calibrations linking shop-floor tools back to primary standards maintained by organizations such as the National Institute of Standards and Technology (NIST) or ISO/IEC 17025 accredited metrology laboratories.

Core calibration requirements include:

• Scheduled Intervals: Periodic checks based on usage frequency and historical drift rates.
• Calibration Stickers: Physical labels displaying serial number, calibration date, expiration date, and technician initials.
• Environmental Controls: Standard metrology room conditions ($20^\circ\text{C} \pm 1^\circ\text{C}$ or $68^\circ\text{F}$) to eliminate thermal expansion distortions.
• Quarantine Procedures: Immediate isolation of any gauge that fails calibration, followed by retrospective risk evaluation of production lots measured since the last valid check.