7.4 Hypothesis Testing, ANOVA and Tests of Association
Key Takeaways
- TOS competencies A.2.3 and A.2.4 name hypothesis testing in forms such as ANOVA and association, and integration of the chi-square test of independence and the Pearson and Spearman tests.
- The null hypothesis states no difference or no relationship; the alternative states that one exists.
- A result is statistically significant when the probability of obtaining it under the null hypothesis is below the chosen alpha level, conventionally 0.05.
- Chi-square tests association between categorical variables; the t-test compares two means; ANOVA compares three or more means.
- Pearson r requires interval or ratio data and linearity, while Spearman rho is the rank-order equivalent for ordinal data.
7.4 Hypothesis Testing, ANOVA and Tests of Association
Blueprint anchor. TOS competencies A.2.3 ("apply hypothesis testing in various forms such as ANOVA, Association") and A.2.4 ("integrate Chi-Square Test of Independence, Pearson and Spearman tests") close the Social Work Statistics topic. Because calculators are barred from the examination room, items test test selection and interpretation, not computation.
1. The Logic of Hypothesis Testing
| Element | Meaning | Example |
|---|---|---|
| Null hypothesis (H₀) | Statement of no difference or no relationship | "There is no difference in school attendance between grant and non-grant households." |
| Alternative hypothesis (H₁) | Statement that a difference or relationship exists | "Attendance differs between the two groups." |
| Alpha (α) | Pre-set risk of wrongly rejecting a true null; conventionally 0.05 | 5 percent |
| p-value | Probability of a result at least this extreme if H₀ were true | p = 0.03 |
| Decision rule | Reject H₀ when p < α | 0.03 < 0.05 → reject H₀ |
What significance does and does not mean. A significant result says the observed pattern is unlikely to be a product of sampling variation alone. It does not say the effect is large, important, or causal. A trivial difference can be significant in a very large sample; a substantial difference can fail significance in a small one. Professional reporting therefore pairs significance with effect size and with practical judgement.
Type I and Type II Error
| H₀ is actually true | H₀ is actually false | |
|---|---|---|
| Reject H₀ | Type I error (α) — claiming an effect that does not exist | Correct decision (power) |
| Fail to reject H₀ | Correct decision | Type II error (β) — missing a real effect |
In welfare evaluation, a Type I error may expand an ineffective programme; a Type II error may discontinue an effective one. Neither is automatically worse, and the professional judgement is about which consequence the agency can better tolerate.
One-tailed versus two-tailed. A directional hypothesis ("the programme increases attendance") is tested one-tailed; a non-directional hypothesis ("attendance differs") is tested two-tailed. Directional tests must be specified before seeing the data.
2. Choosing the Right Test
| Research question | Variables | Appropriate test |
|---|---|---|
| Are two categorical variables related? | Both nominal (for example, sex × programme completion) | Chi-square test of independence |
| Does an observed category distribution match an expected one? | One nominal variable | Chi-square goodness of fit |
| Do two group means differ? | One nominal grouping variable with 2 levels; one interval/ratio outcome | Independent-samples t-test |
| Did the same people change from before to after? | One interval/ratio outcome measured twice | Paired-samples t-test |
| Do three or more group means differ? | One nominal grouping variable with 3+ levels; one interval/ratio outcome | One-way ANOVA |
| Do two grouping factors and their interaction affect an outcome? | Two nominal factors; one interval/ratio outcome | Two-way ANOVA |
| How strongly do two continuous variables move together? | Both interval/ratio, linear | Pearson product-moment correlation (r) |
| How strongly do two ranked variables move together? | Both ordinal, or non-linear/non-normal | Spearman rank-order correlation (rho) |
[!IMPORTANT] The most examinable rule in this topic. Count the groups and read the level of measurement. Two group means → t-test. Three or more group means → ANOVA. Two categorical variables → chi-square. Interval correlation → Pearson. Ordinal correlation → Spearman. Nearly every item in this block resolves on that grid.
3. Chi-Square in Detail
The chi-square test of independence compares observed cell frequencies in a contingency table with the frequencies expected if the two variables were unrelated. A large discrepancy yields a large chi-square value and a small p-value.
- Requires frequencies (counts), never percentages or means, as input.
- Requires reasonably sized expected cell counts; very sparse tables are unreliable.
- Tells you that an association exists, not its direction or strength; strength requires an additional measure of association.
Example. A 2 × 2 table of sex (female, male) against completion of a livelihood programme (completed, dropped out). A significant chi-square says completion is associated with sex; inspection of the cells says which direction.
4. ANOVA in Detail
Analysis of variance compares the variability between group means with the variability within groups. If between-group variability is large relative to within-group variability, the F ratio is large and the null hypothesis of equal means is rejected.
- Used for three or more groups; using repeated t-tests instead inflates the Type I error rate.
- A significant F says at least one pair of means differs, not which pair; post hoc tests identify the specific contrasts.
- Two-way ANOVA additionally tests the interaction — whether the effect of one factor depends on the level of the other.
Welfare example. Comparing average case-closure time across three intervention modalities — casework only, casework plus group work, and casework plus community referral. Three groups, one continuous outcome → one-way ANOVA, followed by a post hoc test to locate the difference.
5. Correlation in Detail
Pearson r ranges from −1.00 to +1.00.
| Value | Meaning |
|---|---|
| +1.00 | Perfect positive linear relationship |
| +0.70 to +0.99 | Strong positive |
| +0.40 to +0.69 | Moderate positive |
| +0.10 to +0.39 | Weak positive |
| 0.00 | No linear relationship |
| Negative values | Same magnitudes, inverse direction |
Spearman rho applies the same interpretation to ranked data and is the correct choice when a variable is ordinal, when the relationship is monotonic but not linear, or when extreme values would distort Pearson r.
[!IMPORTANT] Correlation is not causation. Three alternatives always remain: reverse causation, a confounding third variable, or coincidence. A correlation between number of home visits and case improvement may mean visits help, or that improving cases are easier to visit, or that both track worker caseload size.
6. Worked Practice Application
A regional office evaluates a parenting-capability programme and reports four findings. Identify the test each requires and the interpretive limit.
| Finding | Required test | Interpretive limit |
|---|---|---|
| "Completion rates differ between female and male participants." | Chi-square test of independence on the 2 × 2 count table | Establishes association only; cell inspection gives direction |
| "Mean parenting-scale scores rose from pre-test to post-test in the same participants." | Paired-samples t-test | Without a comparison group, maturation and other events remain rival explanations |
| "Mean scores differ across the three implementing agencies." | One-way ANOVA with post hoc comparison | F alone does not say which agency differs from which |
| "Higher session attendance is associated with higher post-test scores." | Pearson r if scores are interval and the relation is linear; Spearman rho if attendance is banded into ranks | Association only; motivated participants may both attend more and score higher |
The fourth row is the one that decides careers in evaluation: the correct professional sentence is "attendance is positively associated with post-test scores", never "attending more sessions raised scores", unless the design supports a causal claim.
An evaluator compares mean case-closure times across three intervention modalities. Which test is appropriate, and why not repeated t-tests?
A study finds that the number of home visits correlates positively with case improvement (r = 0.62, p < 0.05). Which conclusion is defensible?
An evaluator wants to know whether programme completion is related to participants' sex. The data are counts in a two-by-two table. Which test applies, and what does a significant result establish?