7.7 Point Estimates, Confidence, Prediction, and Tolerance Intervals

Point estimates: bias and efficiency

A point estimate is a single value computed from a sample to estimate a parameter. Two properties determine how good an estimator is.

Bias. An estimator $\hat{\theta}$ is unbiased when $E[\hat{\theta}] = \theta$ -- on average across repeated samples, it hits the parameter.

• $\bar{x}$ is an unbiased estimator of $\mu$.
• $s^2$ with the $n - 1$ divisor is an unbiased estimator of $\sigma^2$.
• $s$ is not an unbiased estimator of $\sigma$, because the square root is a non-linear function. The correction factor $c_4$ exists for exactly this reason: $E[s] = c_4\sigma$, so $\hat{\sigma} = s / c_4$ is unbiased. This is why control chart constants use $\bar{s}/c_4$.

Efficiency. Among unbiased estimators, the efficient one has the smallest variance. For a normal distribution the sample mean is more efficient than the sample median; for heavy-tailed data the median is more robust though less efficient under normality. Efficiency is why $\bar{x}$ is the default estimator of centre in Six Sigma work and why the median appears when outliers dominate.

Three intervals, three questions

For the same data and confidence level, ordering is always:

$$\text{Confidence interval} \subset \text{Prediction interval} \subset \text{Tolerance interval}$$

The confidence interval is narrowest because estimating an average is easier than predicting an individual, and predicting one individual is easier than bracketing 99% of them.

Confidence interval for the mean

With $\sigma$ unknown, which is the normal case:

$$\bar{x} \pm t_{\alpha/2, \, n-1}\frac{s}{\sqrt{n}}$$

Worked example. $n = 25$, $\bar{x} = 50.2$, $s = 2.4$, 95% confidence. $t_{0.025,24} = 2.064$:

$$50.2 \pm 2.064 \times \frac{2.4}{5} = 50.2 \pm 0.991 = (49.21,\ 51.19)$$

Interpretation, stated carefully: if this sampling procedure were repeated many times, 95% of the intervals constructed would contain the true mean. It is not correct to say there is a 95% probability that $\mu$ lies in this particular interval -- $\mu$ is fixed, and this interval either contains it or does not.

Confidence interval for a proportion

$$\hat{p} \pm z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

valid when $n\hat{p}$ and $n(1-\hat{p})$ both exceed about 5. For small samples or extreme proportions, use an exact (Clopper-Pearson) or Wilson interval instead; the normal approximation can otherwise produce a lower bound below zero.

Confidence interval for a variance

$$\left(\frac{(n-1)s^2}{\chi^2_{\alpha/2,\,n-1}},\ \frac{(n-1)s^2}{\chi^2_{1-\alpha/2,\,n-1}}\right)$$

Note that this interval is asymmetric around $s^2$, because the chi-square distribution is skewed.

Prediction interval

Bounds a single future observation from the same process:

$$\bar{x} \pm t_{\alpha/2,\,n-1}\, s\sqrt{1 + \frac{1}{n}}$$

The $1$ under the root is the variation of the future individual itself; the $1/n$ is the uncertainty in the estimated mean. Using the same data as above:

$$50.2 \pm 2.064 \times 2.4 \times \sqrt{1 + 0.04} = 50.2 \pm 2.064 \times 2.4 \times 1.0198 = 50.2 \pm 5.05 = (45.15,\ 55.25)$$

Five times wider than the confidence interval on the same data. Crucially, as $n$ grows the $1/n$ term vanishes but the $1$ does not, so the prediction interval converges to roughly $\mu \pm 1.96\sigma$ and never collapses. No amount of data makes an individual outcome predictable to a point.

Tolerance interval

Bounds a stated proportion of the population with a stated confidence:

$$\bar{x} \pm k \cdot s$$

where $k$ depends on $n$, the proportion $P$ to be covered, and the confidence $\gamma$. Tolerance intervals are the only one of the three that carries two percentages, and both must be quoted: "a 95%/99% tolerance interval" means 95% confidence that at least 99% of the population lies inside.

Representative two-sided $k$ factors for 95% confidence and 99% coverage:

With $n = 25$, $\bar{x} = 50.2$, $s = 2.4$: $50.2 \pm 3.46 \times 2.4 = 50.2 \pm 8.30 = (41.90, 58.50)$.

Note that $k$ approaches 2.576 -- the standard normal value for 99% coverage -- but never falls below it. The excess above 2.576 is the price of estimating $\mu$ and $\sigma$ from a finite sample.

Choosing the right interval

The most consequential error is using a confidence interval to set a specification limit. A confidence interval on the mean of a well-sampled process is very narrow, so specifications derived from it would reject most of the output. Specification and acceptance limits concerned with individual units require tolerance or prediction intervals.

Tolerance intervals also connect directly to capability. A two-sided 95%/99.73% tolerance interval is essentially a statistically-bounded version of the natural process limits, $\bar{x} \pm 3\sigma$, with the sampling uncertainty made explicit.