14.3 Normal Distribution Properties, Empirical Rule & Standard Normal Z-Scores

14.3 Normal Distribution Properties, Empirical Rule & Standard Normal Z-Scores

Continuous Probability Distributions and Density Curves

Unlike discrete random variables where probability concentrates at distinct points, a continuous random variable $X$ can take any real value within an interval. A continuous distribution is governed by a probability density function (pdf), denoted $f(x)$, satisfying two axiomatic requirements:

1. Non-negativity across its entire domain: $f(x) \ge 0$ for all $x \in \mathbb{R}$.
2. The total area enclosed beneath the density curve equals exactly 1: $$\int_{-\infty}^\infty f(x)\,dx = 1$$

The probability that $X$ falls within a continuous interval $[a, b]$ equals the definite integral—or geometric area—under the density curve between those limits:

$$P(a \le X \le b) = \int_a^b f(x)\,dx$$

Because a geometric line segment has zero width, the probability of a continuous variable assuming any exact single real number is identically zero:

$$P(X = c) = \int_c^c f(x)\,dx = 0$$

Consequently, boundary inclusions do not alter probabilities in continuous distributions:

$$P(a \le X \le b) = P(a < X \le b) = P(a \le X < b) = P(a < X < b)$$

---

The Normal Family: Geometric and Analytic Properties

The normal (or Gaussian) distribution, denoted $N(\mu, \sigma)$, is the most prominent probability model in theoretical and applied statistics. Its density function is defined as:

$$f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x - \mu)^2}{2\sigma^2}}, \quad -\infty < x < \infty$$

The parameter $\mu$ represents the distribution mean (location parameter), while $\sigma > 0$ represents the population standard deviation (scale parameter).

The normal curve possesses distinctive mathematical and geometric properties:

1. Unimodal and Bell-Shaped: The density achieves its unique global maximum at the center $x = \mu$, where $f(\mu) = \frac{1}{\sigma \sqrt{2\pi}}$.
2. Symmetry: The curve is symmetric about the vertical axis of symmetry $x = \mu$. Consequently, the mean, median, and mode coincide: $\mu = M = \text{Mode}$. Exactly 50% of the area lies to the left of $\mu$ and 50% to the right.
3. Asymptotic Tails: As $x \to \infty$ or $x \to -\infty$, $f(x)$ approaches zero asymptotically, extending indefinitely in both directions without ever touching the horizontal axis.
4. Inflection Points: The second derivative $f''(x)$ reveals the curve's concavity: $$f''(x) = \frac{1}{\sigma^3 \sqrt{2\pi}} e^{-\frac{(x - \mu)^2}{2\sigma^2}} \left[ \left(\frac{x - \mu}{\sigma}\right)^2 - 1 \right]$$ Setting $f''(x) = 0$ yields the condition $\left(\frac{x - \mu}{\sigma}\right)^2 = 1$, which establishes inflection points at: $$x = \mu - \sigma \quad \text{and} \quad x = \mu + \sigma$$ The curve is concave downward ($f''(x) < 0$) on the interval $(\mu - \sigma, \mu + \sigma)$ and transitions to concave upward ($f''(x) > 0$) in the tails beyond $\mu \pm \sigma$.

---

The Empirical Rule (68-95-99.7 Rule)

For any distribution that is approximately normal (mound-shaped and symmetric), the Empirical Rule provides precise benchmarks for the proportion of observations residing within integer standard deviations of the mean:

• Within 1 Standard Deviation ($\mu \pm 1\sigma$): Approximately 68.27% (commonly rounded to 68%) of all observations fall within $[\mu - \sigma, \mu + \sigma]$. By symmetry, exactly half of this area—approximately 34.13%—resides in each sub-interval $[\mu - \sigma, \mu]$ and $[\mu, \mu + \sigma]$.
• Within 2 Standard Deviations ($\mu \pm 2\sigma$): Approximately 95.45% (commonly rounded to 95%) of observations fall within $[\mu - 2\sigma, \mu + 2\sigma]$. The area between $1\sigma$ and $2\sigma$ on either side encompasses approximately 13.59% of the distribution: $\frac{95.45\% - 68.27\%}{2} \approx 13.59\%$.
• Within 3 Standard Deviations ($\mu \pm 3\sigma$): Approximately 99.73% (commonly rounded to 99.7%) of observations fall within $[\mu - 3\sigma, \mu + 3\sigma]$. The area between $2\sigma$ and $3\sigma$ on either side encompasses approximately 2.14%: $\frac{99.73\% - 95.45\%}{2} \approx 2.14\%$.
• Extreme Tails (Beyond $\mu \pm 3\sigma$): Only $100\% - 99.73\% = 0.27\%$ of the entire distribution lies beyond 3 standard deviations, leaving approximately 0.135% in each extreme outer tail ($x < \mu - 3\sigma$ and $x > \mu + 3\sigma$).

---

Empirical Rule Percentage Breakdown Table

---

Standardizing Data: The Standard Normal Distribution and Z-Scores

To compare observations originating from different normal distributions, raw scores are standardized to the standard normal distribution, denoted $Z \sim N(0, 1)$, having mean $\mu = 0$ and standard deviation $\sigma = 1$.

The Z-Score Formula

The standard score (or $z$-score) represents the directed algebraic distance between an individual observation $x$ and the population mean $\mu$, measured in units of standard deviation $\sigma$:

$$z = \frac{x - \mu}{\sigma}$$

• If $z > 0$, the observation lies above the mean.
• If $z < 0$, the observation lies below the mean.
• If $z = 0$, the observation equals the mean.

Because $z$-scores are dimensionless scale invariants, they enable objective comparisons across different metrics (e.g., comparing a SAT score to an ACT score).

Inverting the Z-Score (Percentile Cutoff Determination)

When solving for an unknown raw value $x$ corresponding to a specified percentile rank or tail probability, the standardization formula is inverted algebraically:

$$x = \mu + z \sigma$$

Standard critical values from the standard normal cumulative distribution $\Phi(z) = P(Z \le z)$ include:

• 90th percentile ($P = 0.90$): $z \approx 1.282$
• 95th percentile ($P = 0.95$): $z \approx 1.645$
• 97.5th percentile ($P = 0.975$): $z \approx 1.960$ (bounding the central 95%)
• 99th percentile ($P = 0.99$): $z \approx 2.326$

---

Assessing Normality in Empirical Datasets

Statistical procedures often rely on the assumption of normality. Before applying normal-model inference, educators and researchers must verify normality using three diagnostic methodologies:

1. Visual Symmetry and Shape: Construct a histogram, dot plot, or stemplot. Look for a single central peak, bilateral symmetry, and smoothly tapering tails without extreme outliers or gaps.
2. Empirical Rule Proportions: Calculate the sample mean $\bar{x}$ and sample standard deviation $s$. Tally the proportion of sample observations within $[\bar{x} \pm s]$, $[\bar{x} \pm 2s]$, and $[\bar{x} \pm 3s]$. Substantial deviation from 68%, 95%, and 99.7% indicates non-normality.
3. Normal Probability (Q-Q) Plots: A normal quantile-quantile plot graphs ordered sample values against theoretical standard normal quantiles:

• Linear Trend: If the plotted points align approximately along a straight diagonal line, the data support the normality assumption.
• Systematic Curvature: Systematic deviations from linearity diagnose distribution shape:
  • Concave-up curve (J-shape): Indicates positive skewness (skewed right).
  • Concave-down curve: Indicates negative skewness (skewed left).
  • S-shaped curve: Points sagging below the line at the left and rising above at the right indicate heavy tails (leptokurtic); the opposite S-shape indicates light tails (platykurtic).

---

Worked Exemplar: Standard Normal Z-Score and Cutoff Determination

Problem: A state department of education administers a comprehensive mathematics licensing exam. Test scores are normally distributed with a mean of $\mu = 520$ and a standard deviation of $\sigma = 60$.

1. Using the Empirical Rule, calculate the approximate percentage of examinees scoring between $460$ and $640$.
2. Candidate $A$ scores $610$ on this test. Candidate $B$ takes a different certification exam with $\mu = 75$ and $\sigma = 8$, scoring $89$. Standardize both scores using $z$-scores to determine which candidate demonstrated superior relative performance.
3. The state establishes an honors endorsement awarded exclusively to candidates scoring in the top 2.5% of all examinees. Calculate the minimum cutoff score required for this endorsement.

• Step 1: Apply the Empirical Rule for the Interval $[460, 640]$. Convert the raw scores to standard deviations from the mean:

  $$z_1 = \frac{460 - 520}{60} = \frac{-60}{60} = -1.0$$ $$z_2 = \frac{640 - 520}{60} = \frac{120}{60} = +2.0$$

  The interval spans from $\mu - 1\sigma$ to $\mu + 2\sigma$. Decompose the area into two asymmetric segments:

  • Area from $\mu - 1\sigma$ to $\mu$: approximately $34.13\%$
  • Area from $\mu$ to $\mu + 2\sigma$: approximately $\frac{95.45\%}{2} \approx 47.72\%$ Summing these percentages:
  $$P(460 \le X \le 640) \approx 34.13\% + 47.72\% = 81.85\% \approx 81.5\%$$

• Step 2: Standardize Scores for Relative Performance Comparison.

  • Candidate $A$: $$z_A = \frac{610 - 520}{60} = \frac{90}{60} = +1.50$$
  • Candidate $B$: $$z_B = \frac{89 - 75}{8} = \frac{14}{8} = +1.75$$
  • Conclusion: Candidate $B$ performed superiorly relative to peers ($z_B = +1.75 > z_A = +1.50$), placing $1.75$ standard deviations above the mean compared to Candidate $A$'s $1.50$ standard deviations.

• Step 3: Determine the Honors Cutoff Score. The top 2.5% corresponds to an upper-tail probability of $\alpha = 0.025$, meaning the cumulative area to the left is:

  $$\Phi(z) = 1 - 0.025 = 0.9750$$

  From the standard normal distribution (and the central 95% interval bounded by $\pm 1.96\sigma$), the critical standard score is $z = 1.96$. Invert the transformation formula:

  $$x = \mu + z\sigma = 520 + (1.96)(60) = 520 + 117.6 = 637.6$$

  Rounding to the nearest whole integer or standard test unit, an examinee must score at least 638 (or $637.6$) to qualify for the honors endorsement.