10.1 Statistical Questions, Samples & Inferences About a Population
Key Takeaways
- A statistical question is one that anticipates variability in the answers, unlike a question with a single fixed answer.
- A simple random sample gives every member of the population an equal chance of selection, which is what makes generalization valid.
- Convenience and voluntary-response samples produce bias because the people most easily reached or most motivated to respond differ systematically from the population.
- Larger samples reduce sampling variability but cannot correct a biased selection method.
- Stratified sampling divides the population into subgroups and samples within each, ensuring representation that a simple random sample may miss by chance.
10.1 Statistical Questions, Samples & Inferences About a Population
Skill 5 of Competency 4 covers the front end of statistics: asking a question that statistics can answer, and gathering data that legitimately represents the group you care about.
What makes a question statistical
A statistical question anticipates variability in the answers.
| Statistical | Not statistical |
|---|---|
| How tall are the students in my school? | How tall is Mr. Alvarez? |
| How many hours do eighth graders sleep? | How many hours did I sleep last night? |
| What is a typical price of a used car in this city? | What is the price of this car? |
The test is whether you expect a distribution of answers or a single answer. "How old am I?" has one answer and needs no statistics. "How old are the teachers at this school?" produces a spread of values that requires a measure of center and a measure of spread to summarize.
This distinction is worth precision because it is also a middle-grades standard, so items may present it as a classroom task: which of four student-generated questions is statistical.
Population, sample, parameter, statistic
- Population — the entire group you want to describe (all 1,400 students in a school)
- Sample — the subset you actually measure (80 of those students)
- Parameter — a numerical fact about the population, usually unknown
- Statistic — a numerical fact about the sample, computed from your data
You use a statistic to estimate a parameter. A census measures everyone, making sampling unnecessary, but it is often impractical or too costly.
Sampling methods
+---------------------------------------------------------------------------+
| RANDOM METHODS (support valid generalization) |
| Simple random every member equally likely; names drawn from a hat |
| Systematic every k-th member from an ordered list |
| Stratified split into subgroups, sample randomly WITHIN each |
| Cluster split into groups, randomly choose whole groups |
| |
| NON-RANDOM METHODS (produce BIAS) |
| Convenience whoever is easiest to reach |
| Voluntary response whoever chooses to answer |
| Judgment whoever the researcher picks as "typical" |
+---------------------------------------------------------------------------+
Simple random sampling is the standard because it gives every member of the population an equal chance of selection, and it is that property — not the sample's size — that licenses generalizing to the population.
Stratified sampling is useful when the population has meaningful subgroups. To study a school with very different grade levels, sample randomly within each grade in proportion to grade size. This guarantees representation that a simple random sample might miss by chance.
Cluster sampling selects entire naturally occurring groups, such as randomly choosing 6 of 40 classrooms and surveying everyone in them. It is cheaper but riskier, since members of a cluster tend to resemble one another.
Recognizing bias
Bias is a systematic tendency for a sample to differ from its population. Common sources:
- Convenience bias. Surveying students in the library about study habits oversamples students who study.
- Voluntary response bias. An online poll captures people with strong opinions, usually negative ones.
- Undercoverage. Calling landlines only misses households without one.
- Question wording. "Don't you agree that the unfair new schedule should be changed?" leads the respondent.
- Nonresponse. If most surveys are returned by one type of person, the returned sample is not the sent sample.
[!WARNING] A larger sample does not fix bias. Surveying 5,000 people leaving a football game about whether the district should fund athletics produces a badly biased estimate no matter how large the sample. Increasing the size only makes a wrong estimate more precise. Bias is fixed by changing the selection method, not the sample size.
Sample size does matter for variability: larger random samples produce estimates that vary less from sample to sample and so tend to land closer to the true parameter. The two ideas are distinct, and items pair them to see whether you conflate them.
Judging whether an inference is valid
Given a scenario, work through three questions in order:
- Was the sample selected randomly from the population of interest? If not, generalization is not supported.
- Does the sample actually come from the target population? Sampling one middle school does not support claims about the whole district.
- Is the sample large enough to be informative? Ten responses give a noisy estimate even if the selection was perfectly random.
A principal wants to know how many of the school's 900 students would join an after-school coding club. She surveys the 28 students in the computer lab during lunch. Is the inference valid? No. Students in the computer lab are far more likely than average to be interested in coding. This is convenience sampling, and it will overestimate interest. A valid design would draw a random sample from the full student roster.
Items also present a valid design and ask what conclusion it supports, so the answer is not always "biased." Read for whether randomization actually occurred.
Which of the following is a statistical question?
A city wants to estimate what percent of residents support a new bus route. A researcher surveys 3,000 people waiting at existing bus stops. What is the main problem?
A school has 1,200 students spread unevenly across grades 6, 7, and 8. A researcher wants to ensure each grade is represented in proportion to its size. Which sampling method should be used?