7.1 Sampling Distributions, Central Limit Theorem, and Confidence Intervals

Statistical inference provides the mathematical bridge between empirical sample data and population-level decision-making in industrial and systems engineering. Whether establishing process capability baselines, validating cycle time reductions following lean Kaizen events, or verifying supplier material compliance, industrial engineers rarely possess full census data for an entire production population. Instead, decisions must be inferred from finite random samples. The NCEES FE Reference Handbook formalizes these estimation techniques under its Engineering Probability and Statistics section. Mastering sampling distributions, the Central Limit Theorem, and confidence interval construction is essential for both foundational statistical inference and advanced quality engineering on the FE examination.

---

1. Population Parameters vs. Sample Statistics

Statistical inference distinguishes sharply between the fixed, underlying characteristics of a population and the random quantities computed from observed samples.

Unbiased Estimators and Bessel's Correction

An estimator $\hat{\theta}$ is said to be unbiased for a parameter $\theta$ if its mathematical expectation equals the true parameter value: $E[\hat{\theta}] = \theta$.

1. Sample Mean Unbiasedness: $E[\bar{X}] = \mu$.
2. Sample Variance Unbiasedness: If we were to divide by $n$ rather than $n - 1$, the expected value would be $E[S_n^2] = \frac{n-1}{n}\sigma^2 < \sigma^2$, systematically underestimating true population variability. Dividing by $n - 1$ is known as Bessel's correction, which adjusts for the fact that sample deviations are measured relative to the sample mean $\bar{x}$ (which itself minimizes squared deviations) rather than the true parameter $\mu$: $$E[s^2] = E\left[\frac{1}{n - 1}\sum_{i=1}^n (x_i - \bar{x})^2\right] = \sigma^2$$

---

2. Central Limit Theorem (CLT) and the Sampling Distribution of the Mean

Mathematical Formulation

Let $X_1, X_2, \dots, X_n$ be an independent and identically distributed (i.i.d.) random sample drawn from any arbitrary population distribution having finite mean $\mu$ and finite variance $\sigma^2$. As the sample size $n$ increases, the sampling distribution of the sample mean $\bar{X}$ converges in distribution to a normal distribution:

$$\bar{X} \sim N\left(\mu, \, \frac{\sigma^2}{n}\right)$$

The standardized sample mean statistic converges to the standard normal distribution $Z \sim N(0, 1)$:

$$Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \xrightarrow{d} N(0, 1)$$

Standard Error of the Mean

The standard deviation of the sampling distribution of $\bar{X}$ is designated as the Standard Error of the Mean ($\text{SE}$):

$$\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$$

Key Engineering Insight: The Inverse Square Root Law Because the standard error scales inversely with the square root of sample size ($\sqrt{n}$), cutting estimation uncertainty in half requires quadrupling the sample size ($4n$). Reducing error by a factor of 10 requires a 100-fold increase in sampling effort.

Applicability and the Rule of Thumb ($n \ge 30$)

• Normally Distributed Populations: If the parent population is itself normally distributed, $\bar{X}$ is exactly normally distributed for any sample size ($n \ge 1$).
• Non-Normal Populations: If the parent population is skewed, uniform, or bimodal, the distribution of $\bar{X}$ becomes increasingly bell-shaped as $n$ grows. By convention in engineering practice and the NCEES reference literature, a sample size of $n \ge 30$ is considered sufficiently large for the sampling distribution of $\bar{X}$ to be approximated by a normal distribution.

---

3. Student's t-Distribution

In practical manufacturing and industrial environments, the true population standard deviation $\sigma$ is almost never known. When the sample standard deviation $s$ is substituted in place of $\sigma$, the standardized test statistic is:

$$T = \frac{\bar{X} - \mu}{s / \sqrt{n}}$$

Because $s$ is itself a random variable subject to sampling fluctuations, $T$ does not follow a standard normal distribution. Instead, when sampling from a normally distributed population, $T$ follows Student's $t$-distribution with $\nu = n - 1$ degrees of freedom.

                  Standard Normal Z (σ known)
                           │
                        ┌──┴──┐         Student's t (ν = 4, heavier tails)
                        │     │                  │
                     ┌──┘     └──┐            ┌──┴──┐
                   ┌─┘           └─┐        ┌─┘     └─┐
              ─────┴───────────────┴────────┴─────────┴─────

Core Properties of Student's t-Distribution

1. Symmetry: Symmetric, continuous, unimodal bell curve centered at zero ($E[T] = 0$ for $\nu > 1$).
2. Heavier Tails: The variance of the $t$-distribution is $\text{Var}(T) = \frac{\nu}{\nu - 2}$ for $\nu > 2$. Because $\frac{\nu}{\nu - 2} > 1$, the $t$-distribution exhibits heavier tails than the standard normal distribution, reflecting the extra uncertainty introduced by estimating $\sigma$ via $s$.
3. Asymptotic Convergence: As degrees of freedom $\nu \to \infty$, the sample variance $s^2$ converges in probability to $\sigma^2$, and the $t$-distribution approaches the standard normal distribution $Z$: $$\lim_{\nu \to \infty} t_{\alpha, \nu} = Z_\alpha$$

---

4. Confidence Intervals for a Single Population Mean (μ)

A confidence interval (CI) provides an estimated range of values calculated from sample data that is likely to include an unknown population parameter with a specified confidence level $100(1 - \alpha)\%$.

Case 1: Population Variance σ Known (Z-Interval)

When $\sigma$ is known and either the underlying population is normal or $n \ge 30$:

$$\bar{x} \pm Z_{\alpha/2} \left(\frac{\sigma}{\sqrt{n}}\right)$$

Common critical values for standard normal two-sided intervals:

• $90\%$ Confidence ($\alpha = 0.10, \alpha/2 = 0.05$): $Z_{0.05} = 1.645$
• $95\%$ Confidence ($\alpha = 0.05, \alpha/2 = 0.025$): $Z_{0.025} = 1.960$
• $99\%$ Confidence ($\alpha = 0.01, \alpha/2 = 0.005$): $Z_{0.005} = 2.576$

Case 2: Population Variance σ Unknown (t-Interval)

When $\sigma$ is unknown and estimated by sample standard deviation $s$ (assuming an approximately normal population):

$$\bar{x} \pm t_{\alpha/2, \, n-1} \left(\frac{s}{\sqrt{n}}\right)$$

where $t_{\alpha/2, \, n-1}$ represents the upper $\alpha/2$ critical value from the Student's $t$-table with $\nu = n - 1$ degrees of freedom.

Rigorous Statistical Interpretation of Confidence Intervals A $95\%$ confidence interval does not mean there is a $95\%$ probability that the true parameter $\mu$ falls within the computed numerical bounds. In frequentist statistics, $\mu$ is a fixed, non-random constant. The correct interpretation is: If an identical sampling procedure is repeated an infinite number of times under identical conditions, $95\%$ of the independently constructed confidence intervals will encapsulate the true population mean $\mu$.

---

5. Confidence Intervals for the Difference Between Two Means (μ₁ - μ₂)

Comparing two production lines, supplier batches, or process settings requires two-sample inference. The formulation depends fundamentally on whether samples are independent or paired (dependent).

Independent Samples: Variances Known

$$(\bar{x}_1 - \bar{x}_2) \pm Z_{\alpha/2} \sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$$

Independent Samples: Variances Unknown but Assumed Equal (Pooled t-Interval)

When $\sigma_1^2 = \sigma_2^2 = \sigma^2$ is unknown, the two sample variances $s_1^2$ and $s_2^2$ are combined into a weighted average known as the pooled sample variance $s_p^2$:

$$s_p^2 = \frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}$$

The confidence interval is:

$$(\bar{x}_1 - \bar{x}_2) \pm t_{\alpha/2, \, n_1+n_2-2} \cdot s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}$$

with pooled degrees of freedom $\nu = n_1 + n_2 - 2$.

Independent Samples: Variances Unknown and Unequal (Welch's t-Interval)

When $\sigma_1^2 \neq \sigma_2^2$, pooling is invalid. The confidence interval utilizes the unpooled standard error:

$$(\bar{x}_1 - \bar{x}_2) \pm t_{\alpha/2, \, \nu} \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$$

where the effective degrees of freedom $\nu$ is determined via the Welch-Satterthwaite approximation (rounded down to the nearest integer):

$$\nu = \frac{\left(\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}\right)^2}{\frac{(s_1^2/n_1)^2}{n_1 - 1} + \frac{(s_2^2/n_2)^2}{n_2 - 1}}$$

Paired (Dependent) Samples (Paired t-Interval)

When observations in sample 1 naturally pair with observations in sample 2 (e.g., measuring worker cycle time before and after ergonomic training on the same $n$ operators), the bivariate problem collapses to a single sample of differences:

$$d_i = x_{1i} - x_{2i}, \quad i = 1, 2, \dots, n$$ $$\bar{d} = \frac{1}{n}\sum_{i=1}^n d_i, \quad s_d = \sqrt{\frac{\sum_{i=1}^n (d_i - \bar{d})^2}{n - 1}}$$

The paired confidence interval is constructed with $\nu = n - 1$ degrees of freedom:

$$\bar{d} \pm t_{\alpha/2, \, n-1} \left(\frac{s_d}{\sqrt{n}}\right)$$

---

6. Confidence Intervals for Proportions and Differences in Proportions

Single Population Proportion (p)

Let $x$ be the number of successes (e.g., non-defective parts) observed in a random sample of size $n$, with point estimator $\hat{p} = x / n$. When both $n\hat{p} \ge 10$ and $n(1 - \hat{p}) \ge 10$, the normal approximation to the binomial distribution applies:

$$\hat{p} \pm Z_{\alpha/2} \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}$$

Difference Between Two Proportions (p₁ - p₂)

For two independent random samples of sizes $n_1$ and $n_2$:

$$(\hat{p}_1 - \hat{p}_2) \pm Z_{\alpha/2} \sqrt{\frac{\hat{p}_1(1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2(1 - \hat{p}_2)}{n_2}}$$

---

7. Confidence Interval for Population Variance (σ²)

Estimating process consistency is central to Six Sigma and statistical tolerance design. When sampling from a normal population, the transformed variable follows a Chi-Square ($\chi^2$) distribution with $\nu = n - 1$ degrees of freedom:

$$\frac{(n - 1)s^2}{\sigma^2} \sim \chi_{n-1}^2$$

Because the Chi-Square distribution is non-negative and asymmetric (skewed right for small $\nu$), the two-sided $100(1 - \alpha)\%$ confidence interval for $\sigma^2$ is bounded by two distinct critical values:

$$\frac{(n - 1)s^2}{\chi_{\alpha/2, \, n-1}^2} \le \sigma^2 \le \frac{(n - 1)s^2}{\chi_{1 - \alpha/2, \, n-1}^2}$$

Taking square roots provides the confidence interval for the population standard deviation $\sigma$.

Crucial Chi-Square Denominator Inversion Notice that the upper critical value $\chi_{\alpha/2, \, n-1}^2$ (the larger number from the right tail) divides the numerator to form the lower confidence limit, while the lower critical value $\chi_{1 - \alpha/2, \, n-1}^2$ (the smaller number from the left tail) divides the numerator to form the upper confidence limit.

---

8. Sample Size Determination for Specified Margin of Error

Designing sampling audits requires determining the minimum sample size $n$ necessary to achieve an acceptable margin of error $E$ at a specified confidence level $100(1 - \alpha)\%$.

Sample Size for Estimating a Population Mean (μ)

The half-width margin of error is $E = Z_{\alpha/2} \frac{\sigma}{\sqrt{n}}$. Solving algebraically for $n$:

$$n = \left( \frac{Z_{\alpha/2} \cdot \sigma}{E} \right)^2$$

Rule: Always round $n$ up to the nearest integer (ceiling function $\lceil n \rceil$) to guarantee that the achieved margin of error does not exceed $E$.

Sample Size for Estimating a Population Proportion (p)

The half-width margin of error is $E = Z_{\alpha/2} \sqrt{\frac{p(1 - p)}{n}}$. Solving for $n$:

$$n = \left( \frac{Z_{\alpha/2}}{E} \right)^2 p(1 - p)$$

• If a reliable historical estimate of $p$ is available, substitute that value.
• If no preliminary estimate exists, use the conservative value $p = 0.5$, which maximizes the product $p(1 - p) = 0.25$, guaranteeing the resulting sample size will satisfy the margin of error regardless of the true proportion.

---

9. Step-by-Step Worked Engineering Examples

Worked Example 7.1.1: CNC Lathe Shaft Tolerance and Sample Size Sizing

Problem: A precision machining cell turns steel drive shafts. Historical tool wear data indicates that shaft diameter standard deviation is $\sigma = 0.040\text{ mm}$.

1. A random sample of $n = 16$ shafts yields a sample mean of $\bar{x} = 25.018\text{ mm}$. Construct a $95\%$ confidence interval for the true mean diameter $\mu$.
2. Quality management mandates that the true mean diameter must be estimated within a margin of error of $E = \pm 0.007\text{ mm}$ with $95\%$ confidence. What is the minimum sample size required?

Solution:

1. Confidence Interval Calculation:
   • Parameters: $\sigma = 0.040\text{ mm}$, $n = 16$, $\bar{x} = 25.018\text{ mm}$, confidence level $= 95\%$.
   • Critical value: $Z_{\alpha/2} = Z_{0.025} = 1.960$.
   • Standard error: $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} = \frac{0.040}{\sqrt{16}} = \frac{0.040}{4} = 0.010\text{ mm}$.
   • Margin of error: $E = Z_{0.025} \cdot \sigma_{\bar{x}} = 1.960 \times 0.010 = 0.0196\text{ mm}$.
   • Confidence Interval: $$\text{CI} = 25.018 \pm 0.0196 = [24.9984\text{ mm}, \, 25.0376\text{ mm}]$$
2. Sample Size Determination:
   • Target margin of error: $E = 0.007\text{ mm}$.
   • Using the sample size formula: $$n = \left( \frac{Z_{\alpha/2} \cdot \sigma}{E} \right)^2 = \left( \frac{1.960 \times 0.040}{0.007} \right)^2 = \left( \frac{0.0784}{0.007} \right)^2 = (11.2)^2 = 125.44$$
   • Applying the integer ceiling rule: $n = \lceil 125.44 \rceil = 126$.
   • Final Answer: A minimum sample of $126$ shafts must be measured.

Worked Example 7.1.2: Ergonomic Workstation Re-Design Paired Study

Problem: An industrial engineer measures assembly cycle times (in seconds) for $6$ operators before and after implementing a tilted fixture and gravity-fed parts bins. The observed differences ($d_i = t_{\text{before}} - t_{\text{after}}$) are:

$$+4.2\text{ s}, \; +1.8\text{ s}, \; +3.5\text{ s}, \; -0.5\text{ s}, \; +2.6\text{ s}, \; +3.4\text{ s}$$

Construct a $90\%$ confidence interval for the true mean reduction in assembly cycle time $\mu_d$.

Solution:

1. Compute Sample Statistics for Differences:
   • Sample size: $n = 6$.
   • Sum of differences: $\sum d_i = 4.2 + 1.8 + 3.5 - 0.5 + 2.6 + 3.4 = 15.0\text{ s}$.
   • Mean difference: $\bar{d} = \frac{15.0}{6} = 2.50\text{ s}$.
   • Sum of squared deviations: $\sum (d_i - \bar{d})^2 = (1.7)^2 + (-0.7)^2 + (1.0)^2 + (-3.0)^2 + (0.1)^2 + (0.9)^2 = 2.89 + 0.49 + 1.00 + 9.00 + 0.01 + 0.81 = 14.20$.
   • Sample variance: $s_d^2 = \frac{14.20}{6 - 1} = 2.84\text{ s}^2$.
   • Sample standard deviation: $s_d = \sqrt{2.84} \approx 1.6852\text{ s}$.
2. Identify Critical Value:
   • Degrees of freedom: $\nu = n - 1 = 6 - 1 = 5$.
   • Confidence level: $90\% \implies \alpha = 0.10 \implies \alpha/2 = 0.05$.
   • From the Student's $t$-table: $t_{0.05, \, 5} = 2.015$.
3. Compute Confidence Interval: $$\text{Margin of Error} = t_{0.05, 5} \left(\frac{s_d}{\sqrt{n}}\right) = 2.015 \left(\frac{1.6852}{\sqrt{6}}\right) = 2.015 \times 0.6880 \approx 1.386\text{ s}$$ $$\text{CI} = 2.50 \pm 1.386 = [1.114\text{ s}, \, 3.886\text{ s}]$$
4. Engineering Conclusion: Because the entire interval is strictly positive ($1.11\text{ s}$ to $3.89\text{ s}$), the engineer can conclude with $90\%$ confidence that the ergonomic fixture provides a statistically significant mean cycle time reduction of at least $1.11$ seconds per cycle.

---

10. NCEES Reference Handbook Tips & Realistic Exam Traps

• Sample Size Rounding Trap: Never round sample size determinations down. In Worked Example 7.1.1, rounding $125.44$ down to $125$ would yield an actual margin of error slightly larger than the prescribed $0.007\text{ mm}$, violating the engineering specification. Always round up to the next whole integer.
• Z-Test vs. t-Test Selection: Look carefully at the problem statement. If the problem specifies "a sample of $15$ items has a standard deviation of $0.5$", that is $s$ (sample standard deviation), requiring a $t$-distribution. Only use $Z$ if the problem explicitly says "the population standard deviation is known to be $\sigma$" or if sample size $n$ is very large.
• Paired vs. Independent Two-Sample Traps: If the same subjects, machines, or parts are tested before-and-after or under two operating conditions, do not use the two-sample pooled $t$-test. You must treat it as a paired $t$-test ($n - 1$ degrees of freedom, not $2n - 2$). Failing to account for pairing inflates the standard error by ignoring the covariance between paired observations.
• Chi-Square Upper/Lower Tail Identification: The FE reference handbook tables often list $\chi^2_{\alpha, \nu}$ as the upper tail area. Be certain to verify whether your table lists upper-tail percentiles or cumulative probabilities so you do not accidentally invert the confidence bounds.