15.2 Survey Sampling, Margin of Error & Population Estimation

15.2 Survey Sampling, Margin of Error & Population Estimation

Parameters vs. Statistics and Sampling Distributions

In statistical inference, a fundamental distinction is maintained between the numerical properties of an entire population and those of an observed sample. A parameter is a fixed, typically unknown numerical summary of a population. Greek letters denote parameters: population mean $\mu$, population proportion $p$, and population standard deviation $\sigma$. Conversely, a statistic is a numerical value calculated from sample data. Latin letters or accented symbols denote statistics: sample mean $\bar{x}$, sample proportion $\hat{p} = \frac{x}{n}$, and sample standard deviation $s$.

Because different random samples drawn from the same population yield different sample members, statistics vary from sample to sample—a phenomenon known as sampling variability. The probability distribution of all possible values taken by a statistic across infinitely many independent random samples of identical size $n$ from the same population is the sampling distribution of that statistic.

For a sample proportion $\hat{p}$ drawn from a population with true parameter $p$:

• The mean of the sampling distribution is strictly unbiased: $\mu_{\hat{p}} = p$.
• The standard deviation of the sampling distribution is: $$\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}$$ provided the sample meets the 10% Condition ($n \le 0.10N$), which ensures that sampling without replacement behaves approximately like independent trials.
• By the Large Counts Condition, when both expected successes and expected failures equal or exceed 10 ($np \ge 10$ and $n(1-p) \ge 10$), the sampling distribution of $\hat{p}$ is approximately normal: $\hat{p} \sim N\left(p, \sqrt{\frac{p(1-p)}{n}}\right)$.

---

Standard Error and the Margin of Error (MOE)

In real-world survey sampling, the true parameter $p$ is unknown. We substitute the sample proportion $\hat{p}$ into the standard deviation formula to obtain the standard error of the sample proportion:

$$\text{SE}(\hat{p}) = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

The standard error quantifies the typical distance that a sample proportion $\hat{p}$ deviates from the true population proportion $p$ in repeated samples of size $n$.

A confidence interval for a population parameter takes the universal structure:

$$\text{Confidence Interval} = \text{Point Estimate} \pm \text{Margin of Error}$$

For a population proportion, the margin of error (MOE) represents the maximum anticipated difference between the sample statistic and the true population parameter at a specified level of confidence:

$$\text{MOE} = z^* \times \text{SE}(\hat{p}) = z^* \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

where $z^*$ is the standard normal critical value capturing central probability $C$:

• For $90\%$ confidence ($C = 0.90$): $z^* = 1.645$
• For $95\%$ confidence ($C = 0.95$): $z^* = 1.960 \approx 2.0$
• For $99\%$ confidence ($C = 0.99$): $z^* = 2.576$

The Conservative Margin of Error Rule of Thumb

When planning a survey before data collection, $\hat{p}$ is unknown. The quadratic expression $f(p) = p(1-p)$ achieves its absolute maximum at $p = 0.5$, where $p(1-p) = (0.5)(0.5) = 0.25$. Substituting this maximal variance into the 95% confidence formula with $z^* \approx 2$ produces the famous conservative rule of thumb:

$$\text{MOE}_{95\%} \approx 2 \sqrt{\frac{0.25}{n}} = 2\left(\frac{0.5}{\sqrt{n}}\right) = \frac{1}{\sqrt{n}}$$

This conservative formula guarantees that regardless of the true value of $p$, the margin of error will never exceed $\frac{1}{\sqrt{n}}$ at the approximate 95% confidence level. Crucially, the margin of error is governed by an inverse-square-root relationship with sample size $n$:

• Quadrupling the sample size ($4n$) cuts the margin of error in half: $\frac{1}{\sqrt{4n}} = \frac{1}{2\sqrt{n}} = \frac{1}{2}\text{MOE}$.
• To reduce the margin of error by a factor of 10, the sample size must be multiplied by $10^2 = 100$.

---

Probabilistic Interpretation of Confidence Intervals

A critical conceptual competency on the FTCE examination is the precise interpretation of confidence intervals. Consider a survey reporting that $58\%$ of voters support a municipal ordinance with a margin of error of $\pm 3\%$ at a $95\%$ confidence level, producing the interval $(0.55, 0.61)$ or $55\%$ to $61\%$.

• Correct Interpretation: "If we were to draw a large number of independent random samples of size $n$ from this population and construct a $95\%$ confidence interval from each sample, approximately $95\%$ of the resulting intervals would successfully capture the true, fixed population proportion of voters who support the ordinance."
• Common Statistical Misconceptions:
  1. The Parameter Probability Fallacy: Stating that 'there is a 95% probability that the true population proportion lies between 0.55 and 0.61' is false in frequentist statistics. The population proportion $p$ is a fixed, constant value. Once a specific numerical interval is calculated, it either contains $p$ or it does not; the probability is strictly 1 or 0.
  2. The Sample Statistic Fallacy: Stating that '95% of future sample proportions will fall between 0.55 and 0.61' is false. Future sample proportions cluster around $p$, not necessarily around the initial sample's point estimate $\hat{p} = 0.58$.
  3. The Individual Data Fallacy: Stating that '95% of individual voters hold values between 55% and 61%' is nonsensical; categorical survey responses are binary (support or oppose), not continuous distributions.

---

Determining Required Sample Size for a Target Margin of Error

To determine the minimum sample size $n$ required to achieve a desired margin of error $m$ at a specified confidence level, we solve the margin of error equation algebraically for $n$:

$$m = z^* \sqrt{\frac{p^*(1-p^*)}{n}} \implies m^2 = (z^*)^2 \frac{p^*(1-p^*)}{n} \implies n = \frac{(z^*)^2 p^*(1-p^*)}{m^2}$$

where $p^*$ represents a prior planning estimate of the population proportion (derived from pilot studies or historical records). If no reliable prior estimate exists, researchers use the conservative planning value $p^* = 0.50$, maximizing $p^*(1-p^*) = 0.25$:

$$n = \frac{(z^*)^2 (0.25)}{m^2}$$

Because fractional individuals cannot be surveyed, the calculated value of $n$ must always be rounded up to the next integer, guaranteeing that the true margin of error does not exceed $m$.

---

Estimating Population Totals

Survey results are frequently utilized to estimate total counts or aggregate sums across a finite population of size $N$:

• Total Count Estimate: For a binary categorical attribute with sample proportion $\hat{p}$, the estimated total count in the population is: $$\hat{T} = N \hat{p}$$ The standard error of the estimated total scales directly by population size $N$: $$\text{SE}(\hat{T}) = N \cdot \text{SE}(\hat{p}) = N \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$ The confidence interval for the total count is $\hat{T} \pm z^* \cdot \text{SE}(\hat{T}) = N(\hat{p} \pm \text{MOE})$.
• Aggregate Sum Estimate: For a quantitative variable with sample mean $\bar{x}$, the estimated aggregate population total is $\hat{T}_{\text{sum}} = N \bar{x}$, with standard error $\text{SE}(\hat{T}_{\text{sum}}) = N \left(\frac{s}{\sqrt{n}}\right)$.

---

Worked Exemplar: Sample Size and Population Total Estimation

Problem: A school district with $N = 30,000$ enrolled high school students wishes to estimate the proportion planning to take Advanced Placement (AP) examinations. The school board requires a $95\%$ confidence level ($z^* = 1.96$) with a margin of error of at most $\pm 3.5\%$ ($m = 0.035$). A preliminary survey suggests that roughly $40\%$ of students plan to take AP exams ($p^* = 0.40$).

1. Determine the Minimum Sample Size:

   $$n = \frac{(1.96)^2 (0.40)(1 - 0.40)}{(0.035)^2} = \frac{(3.8416)(0.40)(0.60)}{0.001225} = \frac{0.921984}{0.001225} \approx 752.64$$

   Rounding strictly up to the next integer yields a minimum sample size of $n = 753$ students.

2. Estimate the Population Total: Suppose a random sample of $n = 800$ students is surveyed, and $x = 336$ students confirm they plan to take AP exams. The sample proportion is $\hat{p} = \frac{336}{800} = 0.42$.

   • Point estimate of total students: $\hat{T} = N\hat{p} = 30,000 \times 0.42 = 12,600$ students.
   • Standard error of the proportion: $$\text{SE}(\hat{p}) = \sqrt{\frac{(0.42)(0.58)}{800}} = \sqrt{\frac{0.2436}{800}} = \sqrt{0.0003045} \approx 0.01745$$
   • Margin of error for the proportion: $\text{MOE} = 1.96 \times 0.01745 \approx 0.0342$ ($3.42\%$).
   • Margin of error for the population total: $$\text{MOE}_{\text{Total}} = N \times \text{MOE} = 30,000 \times 0.0342 = 1,026\text{ students}$$
   • The $95\%$ confidence interval for total AP students is $12,600 \pm 1,026$, or $[11,574, 13,626]$ students.

---

Margin of Error and Standard Error Formula Comparison