Interpreting IM&TE Specification Terms

Metrology Subject Focus: Calibration technicians must interpret, calculate, and apply equipment specifications to evaluate whether Inspection, Measuring, and Test Equipment (IM&TE) meets manufacturer performance limits and satisfies process tolerance requirements.

In calibration science and quality engineering, an IM&TE specification represents the formal operational boundaries within which an instrument manufacturer warrants an instrument will perform, or within which an in-house metrology system governs measurement capability. Interpreting these specifications requires understanding the underlying physical mechanisms—such as quantization limits, resistive divider thermal drift, amplifier offsets, and mechanical friction—that dictate how measurement errors scale across an instrument's dynamic range.

A rigorous understanding of specification terms enables calibration technicians to calculate the Maximum Permissible Error (MPE) for any test point, determine compliance during calibration verification, and establish whether an instrument possesses adequate accuracy to support specific manufacturing or testing tolerances.

---

Core Specification Formats

Instrument manufacturers describe measurement accuracy using various mathematical expressions. Each format reflects different internal physical error sources.

Percent of Full Scale (%FS)

Under a Percent of Full Scale (%FS) specification, the maximum permissible absolute error is fixed across the entire operating span of the selected range:

$$\text{MPE}_{\text{abs}} = \pm \left(\frac{\%\text{FS}}{100}\right) \times \text{Full Scale Value}$$

Because the absolute error bound remains constant regardless of the displayed value, the relative error (error expressed as a percentage of the measured value) increases sharply as the indicated value decreases:

$$\text{Relative Error} = \frac{\text{MPE}_{\text{abs}}}{\text{Indicated Value}} \times 100\% = \frac{\%\text{FS} \times \text{Full Scale Value}}{\text{Indicated Value}}$$

The Low-Scale Divergence Phenomenon

Consider a pressure gauge with a full scale of $100\text{ psi}$ and an accuracy specification of $\pm 1\%\text{ FS}$:

• The absolute error bound is constant: $\pm (0.01 \times 100\text{ psi}) = \pm 1.0\text{ psi}$.
• At an indicated pressure of $100\text{ psi}$, the relative error is $\pm 1.0\%$.
• At an indicated pressure of $50\text{ psi}$, the relative error is $\frac{\pm 1.0\text{ psi}}{50\text{ psi}} \times 100\% = \pm 2.0\%$.
• At an indicated pressure of $10\text{ psi}$, the relative error escalates to $\frac{\pm 1.0\text{ psi}}{10\text{ psi}} \times 100\% = \pm 10.0\%$.
• At an indicated pressure of $2\text{ psi}$, the relative error reaches $\frac{\pm 1.0\text{ psi}}{2\text{ psi}} \times 100\% = \pm 50.0\%$.

Evaluate the actual absolute bound at the selected reading and range. A low reading is not automatically invalid: fixed range, floor, and count terms may dominate, so compare their combined effect with the application requirement. Use the valid range giving adequate resolution and uncertainty without exceeding ratings.

Percent of Range (% of Range)

Interpret percent of range using the manufacturer’s definition. Some specifications use span (URL minus LRL); others use a selected nominal range magnitude. The following span equations and examples assume that the datasheet explicitly defines range as span. Do not apply that convention blindly to a bipolar or suppressed scale.

$$\text{Range Span} = \text{URL} - \text{LRL}$$ $$\text{MPE}_{\text{abs}} = \pm \left(\frac{\%\text{Range}}{100}\right) \times (\text{URL} - \text{LRL})$$

For an instrument whose scale begins at zero ($0\text{ to }100\text{ V}$), range and full scale are numerically identical. However, for bipolar, compound, or suppressed scales, the distinction is significant:

• Compound Pressure Gauge ($-14.7\text{ psig to }+60.0\text{ psig}$): The full scale might be stated as $60\text{ psig}$, but the range span is $60.0 - (-14.7) = 74.7\text{ psi}$.
• Environmental Chamber Sensor ($-50.0^\circ\text{C to }+150.0^\circ\text{C}$): The range span is $150.0 - (-50.0) = 200.0^\circ\text{C}$. A $\pm 0.2\%\text{ of range}$ specification yields $\text{MPE}_{\text{abs}} = \pm (0.002 \times 200.0^\circ\text{C}) = \pm 0.4^\circ\text{C}$ across the entire thermal span.

Percent of Reading (%rdg or % of Indicated Value)

A Percent of Reading specification states that the permissible error scales directly with the magnitude of the measured quantity:

$$\text{MPE}_{\text{abs}} = \pm \left(\frac{\%\text{rdg}}{100}\right) \times |\text{Indicated Value}|$$

Under this format, the relative error remains constant across the entire operating range, while the absolute error contracts proportionally as the indicated value decreases. This characteristic is typical of high-end electronic standards, ratio transformers, and optical encoders where error mechanisms are dominated by amplifier gain stability and resistive divider ratios rather than fixed noise floors.

Number of Counts and Least Significant Digits (LSD)

Digital multimeters (DMMs), frequency counters, and digital indicators include a quantization floor term expressed as a number of counts or digits of the least significant digit (LSD). This term accounts for:

1. Analog-to-digital converter (ADC) quantization uncertainty ($\pm 0.5\text{ count}$ rounding).
2. Internal comparator noise and digital jitter.
3. Input amplifier baseline DC offset voltage.

The absolute value of the count specification depends directly on the display resolution of the active range:

$$\text{MPE}_{\text{counts}} = \pm (\text{Number of Counts}) \times (\text{Scale Resolution})$$

For example, if a hypothetical DMM on its $10.00000\text{ V}$ range has a resolution of $0.00001\text{ V}$ ($10\ \mu\text{V}$), an accuracy component specified as $\pm 3\text{ counts}$ represents an absolute error floor of:

$$\pm (3 \times 10\ \mu\text{V}) = \pm 30\ \mu\text{V}$$

If the technician switches the DMM to the $100.0000\text{ V}$ range (where resolution is $0.0001\text{ V} = 100\ \mu\text{V}$), those same $\pm 3\text{ counts}$ now represent $\pm 300\ \mu\text{V}$.

Floor Terms and Combined Specifications

Precision electronic IM&TE specifications rarely consist of a single term. They combine a proportional term (tracking input magnitude) with a fixed floor term (accounting for residual noise, thermal EMFs, and leakage):

$$\text{MPE}_{\text{abs}} = \pm (\%\text{rdg} \text{ term} + \text{Floor term})$$

Common combinations include:

• $\pm (\%\text{ of reading} + \%\text{ of range})$
• $\pm (\%\text{ of reading} + \%\text{ of full scale})$
• $\pm (\%\text{ of reading} + \text{number of counts})$
• $\pm (\text{ppm of reading} + \text{ppm of range})$, where $1\text{ ppm} = 0.0001\% = 1 \times 10^{-6}$.

Applying an output specification

A sourcing instrument and a measuring instrument can use similar specification formats but describe different operations. A source’s bound applies to the quantity delivered at its specified terminals and load. A meter’s bound applies to its indication under the specified input and settings. Verify whether the denominator is the source setting, displayed reading, selected range, or fixed nominal span. The same numeric ppm value can therefore produce different absolute bounds.

For a hypothetical source setting of 2 V on a 10 V range, a 20 ppm setting term contributes 40 microvolts. A 5 ppm range term contributes another 50 microvolts. Their specified total bound is 90 microvolts before any other applicable terms. Changing to a higher range can increase the floor without changing the setting term. Also check load current, lead drop, remote-sense configuration, settling time, and compliance limits. A source that cannot sustain its specified output into the actual load cannot be judged suitable from the unloaded accuracy table alone.