Geometry, Thermal Expansion, and Mass

Geometric Principles, Alignment & Thermal Expansion

Metrological accuracy depends as much on geometry and environment as on instrument quality.

Ernst Abbe's Comparator Principle

Formulated in 1890 by German physicist Ernst Abbe, this principle is the foundational law of dimensional tool design:

"The measuring instrument must always be constructed so that the measuring distance is a straight-line continuation of the graduated scale used as the reference standard."

When the reference scale and measuring axis are collinear (as in an outside micrometer), any angular tilting or wobble (pitch or yaw by angle $\theta$) of the moving member causes only a negligible second-order error:

$$e_{\text{second-order}} = L (1 - \cos\theta) \approx L \frac{\theta^2}{2}$$

When an instrument violates Abbe's principle (as in calipers, height gages, and coordinate measuring arms), the measurement axis is separated from the reference scale by an offset distance $d$. Any angular tilt $\theta$ of the slider jaw generates a catastrophic first-order Abbe error:

$$e_{\text{Abbe}} = d \cdot \tan\theta \approx d \cdot \theta$$

For example, on a caliper measuring a part at the tip of $50\text{ mm}$ jaws ($d = 50\text{ mm}$), an imperceptible jaw play of just $0.05^\circ$ ($0.00087\text{ rad}$) yields an error of:

$$e_{\text{Abbe}} = 50\text{ mm} \times 0.000873 = 0.044\text{ mm} \quad (44\ \mu\text{m})$$

Cosine Error

Cosine error occurs when an instrument's measuring probe or axis of displacement is misaligned with the intended line of measurement by an angle $\theta$.

• In lever-type dial test indicators, the stylus must be aligned parallel to the measured surface ($0^\circ$). If the stylus is inclined at an angle $\theta$ relative to the surface plane, the indicated dial deflection overstates the true perpendicular movement:

$$L_{\text{true}} = L_{\text{indicated}} \cdot \cos\theta$$

• In bore gages and coordinate measurements, if the measuring instrument traverses across an oblique angle $\theta$, the indicated length overstates the true orthogonal distance:

$$L_{\text{indicated}} = \frac{L_{\text{true}}}{\cos\theta}$$

Thermal Expansion and the $20^\circ\text{C}$ Standard Reference Temperature

Per ISO 1:2022, the universal standard reference temperature for geometrical product specifications and dimensional verification is exactly $20^\circ\text{C}$ ($68^\circ\text{F}$).

When a workpiece ($w$) and a measurement standard ($s$) deviate from $20^\circ\text{C}$, the net differential thermal expansion error $\Delta L$ is modeled by:

$$\Delta L = L_0 \left[ \alpha_w (T_w - 20^\circ\text{C}) - \alpha_s (T_s - 20^\circ\text{C}) \right]$$

Where:

• $L_0$ = nominal length at $20^\circ\text{C}$
• $\alpha_w, \alpha_s$ = linear thermal expansion coefficients of workpiece and standard
• $T_w, T_s$ = actual temperatures of workpiece and standard

Thermal stabilization is established by the procedure and uncertainty requirement. Verify equilibrium and gradients using temperature information; a fixed “one hour per inch” rule is not a universal guarantee. Material, mass, packaging, air flow, and the initial temperature difference all affect equilibration.

---

Mass Metrology and Buoyancy Correction

Mass metrology distinguishes true mass from conventional mass. Conventional mass is the mass of a reference of density 8000 kg/m³ that balances the object at 20°C in air of density 1.2 kg/m³. It is a defined convention, not simply any balance indication obtained in ambient air. Use the certificate’s quantity and reference density consistently.

Electronic Analytical Balances

Modern analytical balances utilize Electromagnetic Force Compensation (EMFC). The weight pan rests on a mechanical flexure linkage connected to a coil suspended inside a permanent magnet's field. When a sample is loaded, an optical position detector senses deflection; a servo amplifier increases direct current through the coil to generate an opposing Lorentz force ($F = B \cdot I \cdot L$) returning the mechanism to optical null. The current required is proportional to the downward gravitational force.

Common balance tests: method-specific setup and limits

1. Repeatability (Precision): Determined by loading and unloading a test mass (typically 50% to 100% of capacity) a prescribed number of times (ten in this example). The sample standard deviation $s$ quantifies random repeatability error.
2. Linearity (Accuracy across Span): Evaluated at the required test points, for example these five levels from zero to full capacity (e.g., 0%, 25%, 50%, 75%, 100%). Identifies multi-breakpoint span non-linearities.
3. Eccentricity (Corner Load Error): Verifies that off-center loading does not distort results. A test load (usually $1/3$ capacity) is placed sequentially at the center, front-left, rear-left, rear-right, and front-right quadrants of the pan. Maximum allowable difference must not exceed the instrument specification.

Precision Standards and Weight Classifications

Material and construction requirements depend on nominal value and class. Not every OIML weight must be monolithic stainless steel.

Air Buoyancy Correction

When weighing in air, the fluid buoyant upward force equals the weight of displaced air (Archimedes' Principle). If the sample density $\rho$ differs from the reference weight density $\rho_{r}$, an apparent mass error is introduced.

The fundamental Air Buoyancy Correction Equation is:

$$m = m_r \cdot \frac{1 - \frac{\rho_a}{\rho_r}}{1 - \frac{\rho_a}{\rho}}$$

Where:

• $m$ = true mass of the object
• $m_r$ = apparent mass indication relative to the stated reference (reference weight balance reading)
• $\rho_a$ = air density at ambient conditions (nominally $1.2\text{ kg/m}^3$ at sea level, $20^\circ\text{C}$, 50% RH)
• $\rho_{r}$ is the actual or assigned reference density used in the model; 8000 kg/m³ is the conventional reference density, not the exact density of every stainless-steel weight. Here $m_r$ is the apparent reading referred to the true calibrated mass of that reference. Convert conventional certificate values appropriately before applying a general true-mass model.
• $\rho$ = true density of the object being weighed

Practical Buoyancy Impact

When calibrating water volumes for pipette verification ($\rho_{\text{water}} \approx 1000\text{ kg/m}^3$):

$$\frac{m}{m_r} = \frac{1 - 1.2 / 8000}{1 - 1.2 / 1000} = \frac{1 - 0.00015}{1 - 0.00120} = \frac{0.99985}{0.99880} \approx 1.001051$$

Failing to apply the buoyancy correction when weighing water introduces a systematic negative bias of $0.105\%$ ($1050\text{ ppm}$)—a severe error in micro-pipette calibration.