4.2 Statistical Studies, Sampling, and Inference

4.2 Statistical Studies, Sampling, and Inference

Quick Answer: MA.912.DP.3.1 asks you to design and critique statistical studies, distinguish sampling methods (simple random, stratified, systematic, convenience), judge whether a sample is representative, interpret margin of error, and separate correlation from causation. Expect scenario-heavy items, not calculations.

Types of Statistical Studies

The key distinction: only a randomized experiment supports a cause-and-effect conclusion. An observational study can show association, but a hidden confounding variable may explain the link, so a causal claim is unjustified.

Sampling Methods

• Simple random sample (SRS): every possible sample of size n has an equal chance of being selected, and every individual has an equal probability of selection. The gold standard for representativeness.
• Stratified random sample: divide the population into homogeneous groups (strata) — for example, by grade level — then take an SRS from each stratum. Guarantees every subgroup is represented and reduces variability.
• Systematic sample: select every kth individual from an ordered list after a random start. Easy to implement, but risky if the list has a repeating pattern aligned with k.
• Convenience sample: use whoever is easy to reach. This is biased and not representative; results cannot be generalized.

Worked example. A school district wants to estimate the average Algebra 1 EOC score district-wide.

• Convenience (asking your own classmates): biased, because your class may be an honors section.
• SRS (randomly pick 50 IDs from the roster): representative.
• Stratified (randomly pick 5 from each of 10 schools): guarantees every school represented, useful if schools differ in demographics.

Representative Samples and Bias

A sample is representative when it mirrors the population's relevant characteristics. Bias enters through:

• Undercoverage — some groups are omitted from the sampling frame.
• Nonresponse — selected individuals refuse to participate; those who do may differ from those who do not.
• Wording bias — leading or confusing question phrasing.
• Response bias — respondents answer inaccurately due to social desirability or memory.

The EOC gives a scenario and asks which source of bias is most likely; identify the specific mechanism, not just "the sample is bad".

Margin of Error

A point estimate plus or minus its margin of error (MOE) gives a confidence interval. On the EOC you will not compute MOE from scratch, but you must interpret it. A statement such as "62% (±4%) of students pass on the first attempt" means the true population proportion plausibly lies between 58% and 66%. Key relationships:

• Larger sample size → smaller margin of error (more information reduces uncertainty).
• Higher confidence level → larger margin of error (more certainty requires a wider interval).
• More variability in the population → larger margin of error.

Trap: a choice saying "the true value is exactly 58% to 66%" overstates certainty. The interval is a plausible range, not a guarantee.

Evaluating Claims from Studies

When a study claim is reported, ask: Was the sample random and representative? Is the sample size large enough for the stated margin of error? Is the study observational or experimental (only experiments support causation)? Are confounding variables controlled? Does the reported effect exceed the margin of error?

Example claim: "A new online Algebra tutorial raises EOC scores by 8 points." Check for a control group, random assignment, and whether the 8-point difference exceeded the reported MOE. If the study observed students who self-selected into the tutorial, the gain may reflect motivation, not the tutorial.

Correlation vs Causation

Correlation is a statistical association between two variables: as one changes, the other tends to change. Causation means a change in one variable causes a change in the other. The EOC repeatedly tests the trap of inferring causation from correlation.

A lurking (confounding) variable affects both variables and creates a spurious association. Example: ice cream sales and drowning correlate, but temperature drives both; temperature is the lurking variable, and ice cream sales do not cause drowning.

Rules for the EOC:

• A strong correlation coefficient (covered in 7.3) does not by itself prove causation.
• Only a randomized controlled experiment can establish cause and effect; observational studies can establish association but not causation.
• A plausible lurking variable is a valid reason to reject a causal claim.

Common Exam Traps

1. "Convenience" disguised as "random." Selecting the first 30 students who arrive at the cafeteria is convenience sampling, not random sampling.
2. Confounding ignored. A claim that "students who use flashcards score higher" ignores that motivated students are more likely to use flashcards; motivation is confounded with flashcard use.
3. Margin of error direction reversed. A larger sample produces a smaller MOE, not a larger one. A choice saying "increase sample size to widen the margin of error" reverses the relationship.
4. Causation from observational data. Any choice that draws a causal conclusion from a survey or observational study is wrong; only a randomized experiment supports that leap.

On test day, expect to identify a sampling method from a description and judge representativeness, name the source of bias in a scenario, interpret a reported estimate with margin of error as a plausible interval, state whether an effectiveness claim is justified given the study design, and identify a lurking variable that could explain an observed correlation.