15.3 Bivariate and Multivariate Techniques: t-Tests, ANOVA Models, Correlation, and Regression

Bivariate and Multivariate Techniques: t-Tests, ANOVA Models, Correlation, and Regression

Psychological investigations evaluate hypotheses spanning diverse designs, from comparing treatment groups to modeling complex multivariate associations. Selecting the appropriate statistical test requires assessing the scale of measurement, the number of independent and dependent variables, the research design (between-subjects vs. within-subjects), and adherence to distributional assumptions.

1. Assumptions of Parametric Statistical Tests

Parametric statistical tests (e.g., $t$-tests, ANOVA, Pearson correlation, linear regression) estimate population parameters under specific mathematical assumptions. Violating these assumptions can inflate Type I error rates or diminish statistical power.

The Four Core Parametric Assumptions

1. Scale of Measurement: The dependent variable must be measured on a continuous interval or ratio scale.
2. Independence of Observations: Each observation within a group must be statistically independent of every other observation. Violations of independence (e.g., testing participants in social groups without accounting for clustering) represent the most severe threat to validity, inflating Type I error rates substantially.
3. Normality: The population distribution from which samples are drawn (or the distribution of sample residuals) is normally distributed. Due to the Central Limit Theorem, parametric tests are relatively robust to moderate violations of normality when sample sizes are sufficiently large ($N \ge 30$ per group).
4. Homogeneity of Variance (Homoscedasticity): In multi-group designs, the variance of the populations must be equal across all groups:

$\sigma_1^2 = \sigma_2^2 = \dots = \sigma_k^2$

• Diagnostic Assessment: Homogeneity of variance is assessed empirically using Levene's test or Hartley's $F_{\max}$ test. A statistically significant Levene's test ($p < .05$) indicates that the assumption of equal variances has been violated.
• Remedy: When variances are unequal (heteroscedasticity), particularly with unequal sample sizes, researchers use Welch's $t$-test (which applies Satterthwaite's adjusted degrees of freedom) or the Welch/Brown-Forsythe robust ANOVA.

2. Student's t-Test Family

Developed by William Sealy Gosset under the pseudonym "Student" while working at the Guinness Brewery, the $t$-distribution accounts for the added uncertainty of estimating the population variance from a sample.

Why Paired t-Tests Yield Greater Statistical Power

In a paired-samples design, researchers compute a difference score for each participant:

$D_i = X_{1i} - X_{2i}$

By analyzing difference scores, the paired $t$-test eliminates between-subject variability (individual differences in baseline intelligence, personality, or genetics) from the error denominator ($s_D$). Because between-subject variance is often substantial, removing it shrinks the standard error of difference scores, yielding a larger $t$-value and substantially higher statistical power than an independent-samples test with an equivalent number of observations.

3. Analysis of Variance (ANOVA)

When an experiment involves three or more group means, conducting multiple independent $t$-tests causes exponential inflation of the familywise Type I error rate ($\alpha_{\text{FW}}$).

The Problem of Multiple Comparisons

If an experiment compares $k$ groups, the number of unique pairwise comparisons is given by:

$c = \frac{k(k - 1)}{2}$

The probability of committing at least one Type I error across $c$ independent comparisons at $\alpha = .05$ is calculated as:

$\alpha_{\text{FW}} = 1 - (1 - \alpha)^c$

• For $k = 3$ groups: $c = 3$ comparisons $\rightarrow \alpha_{\text{FW}} = 1 - (0.95)^3 \approx .143$ ($14.3\%$ error risk).
• For $k = 4$ groups: $c = 6$ comparisons $\rightarrow \alpha_{\text{FW}} = 1 - (0.95)^6 \approx .265$ ($26.5\%$ error risk).
• For $k = 5$ groups: $c = 10$ comparisons $\rightarrow \alpha_{\text{FW}} = 1 - (0.95)^{10} \approx .401$ ($40.1\%$ error risk).

Ronald Fisher developed Analysis of Variance (ANOVA) to solve this problem. ANOVA performs a single omnibus test evaluating whether any differences exist among the $k$ group means while holding the overall false-positive rate at $\alpha = .05$.

Partitioning Variance in One-Way Between-Subjects ANOVA

ANOVA decomposes the total variation in the data ($SS_{\text{total}}$) into two independent additive components:

$SS_{\text{total}} = SS_{\text{between}} + SS_{\text{within}}$

1. Between-Groups Variance ($SS_{\text{between}}$): Reflects differences between group means and the grand mean. This variation stems from two sources: genuine experimental treatment effects PLUS random individual sampling error.
2. Within-Groups Variance ($SS_{\text{within}}$ or Error Variance): Reflects variability among participants within the same treatment condition. Because all participants within a group receive identical treatment, this variability is attributable purely to random individual differences and measurement error.

                               Total Variance (SStotal, df = N - 1)
                                                │
                  ┌─────────────────────────────┴─────────────────────────────┐
                  ▼                                                           ▼
     Between-Groups Variance                                     Within-Groups Variance
    (SSbetween, df = k - 1)                                     (SSwithin, df = N - k)
  Treatment Effect + Random Error                                 Random Error Alone
                  │                                                           │
                  └─────────────────────────────┬─────────────────────────────┘
                                                ▼
                              F-Ratio = MSbetween / MSwithin

The F-Ratio and Expected Mean Squares

Dividing each sum of squares by its respective degrees of freedom produces the Mean Squares ($MS$):

$MS_{\text{between}} = \frac{SS_{\text{between}}}{k - 1} \quad \text{and} \quad MS_{\text{within}} = \frac{SS_{\text{within}}}{N - k}$

The test statistic is the $F$-ratio:

$F = \frac{MS_{\text{between}}}{MS_{\text{within}}} = \frac{\text{Treatment Effect} + \text{Random Error}}{\text{Random Error}}$

• Under $H_0$ (No Treatment Effect): Treatment Effect $= 0$. Therefore, $F = \frac{0 + \text{Random Error}}{\text{Random Error}} \approx 1.0$. The expected value of $F$ under the null hypothesis is approximately $1.0$.
• Under $H_1$ (Treatment Effect Exists): Treatment Effect $> 0$. Therefore, $MS_{\text{between}}$ exceeds $MS_{\text{within}}$, producing an $F$-ratio substantially greater than $1.0$. If $F$ exceeds the critical value, $H_0$ is rejected.

Post-Hoc Tests vs. Planned Comparisons

A significant omnibus $F$-test confirms that at least one group mean differs from another, but does not indicate which specific pairs differ. Researchers resolve this ambiguity using planned contrasts or post-hoc tests:

• Planned (A Priori) Contrasts: Specified before data collection based on explicit theoretical hypotheses. They do not require a significant omnibus $F$ and possess higher statistical power.
• Post-Hoc (A Posteriori) Tests: Conducted after obtaining a significant omnibus $F$ to explore all possible pairwise differences while strictly controlling $\alpha_{\text{FW}}$:
  • Tukey's Honestly Significant Difference (HSD): The standard post-hoc test in psychology; controls familywise error across all pairwise comparisons while maintaining high power.
  • Bonferroni Correction: Extremely versatile but conservative adjustment that divides nominal alpha by the number of comparisons ($\alpha_{\text{adj}} = \alpha / c$).
  • Scheffé Test: The most conservative post-hoc test; controls familywise error for all possible simple and complex linear combinations of means.

4. Factorial ANOVA and Multivariate Extensions

Factorial ANOVA ($A \times B$ Designs)

In a factorial ANOVA, two or more categorical independent variables (factors) are manipulated simultaneously. In a $2 \times 2$ factorial design, two independent variables each possess two levels, yielding four distinct treatment cells.

A two-way factorial ANOVA yields three independent $F$-tests:

1. Main Effect of Factor A: The effect of Factor A averaged across all levels of Factor B.
2. Main Effect of Factor B: The effect of Factor B averaged across all levels of Factor A.
3. Interaction Effect ($A \times B$): Evaluates whether the effect of Factor A on the dependent variable depends upon the specific level of Factor B.

Factorial ANOVA Interaction Graphs:

       No Interaction (Parallel Lines)              Significant Interaction (Non-Parallel Lines)
  DV                                           DV
  High ┌────────────────────── B2 (Caffeine)   High ┌─────────\────────────── B2 (Caffeine)
       │                      /                     │          \            /
       │                     /                      │           \          /
       │                    /                       │            \        /
       │                   /                        │             \      /
       │                  /                         │              \    /
       │                 /                          │               \  /
       │                /                           │                \/
   Low └───────────────/────── B1 (Placebo)     Low └────────────────/\────── B1 (Placebo)
      Low Stress   High Stress                     Low Stress   High Stress

Repeated-Measures ANOVA and Sphericity

When the same participants are tested across three or more experimental conditions, researchers employ a repeated-measures ANOVA. This design partitions inter-subject variation out of the error term, maximizing power.

• The Sphericity Assumption: Repeated-measures ANOVA requires sphericity—the assumption that the variances of the differences between all possible pairs of conditions are equal.
• Evaluation and Correction: Sphericity is evaluated using Mauchly's test of sphericity. If violated ($p < .05$), degrees of freedom are reduced using the Greenhouse-Geisser ($\hat{\epsilon}$) or Huynh-Feldt epsilon adjustments to prevent Type I error inflation.

Analysis of Covariance (ANCOVA)

ANCOVA combines ANOVA with linear regression. It evaluates group differences on a continuous dependent variable while statistically controlling for one or more continuous extraneous variables known as covariates. Controlling for the covariate removes extraneous error variance, increasing statistical precision and power.

5. Correlation and Linear Regression

Pearson Product-Moment Correlation ($r$)

Developed by Karl Pearson, $r$ quantifies the direction and strength of the linear association between two continuous variables ($X$ and $Y$):

$r = \frac{\text{Covariance}(X, Y)}{s_X s_Y} = \frac{\sum (X - \bar{X})(Y - \bar{Y})}{\sqrt{\sum (X - \bar{X})^2 \sum (Y - \bar{Y})^2}}$

• Properties:
  • Values range strictly from $-1.00$ to $+1.00$.
  • The sign indicates direction: positive (direct) or negative (inverse).
  • The absolute value ($|r|$) indicates strength.
  • It is scale-invariant; linear transformations of $X$ or $Y$ do not alter $r$.
• Coefficient of Determination ($r^2$): Squaring the correlation coefficient yields $r^2$, representing the proportion of variance in $Y$ that is predictable from or shared with $X$. If $r = .50$, $r^2 = .25$, meaning $25\%$ of the variance is shared and $75\%$ remains unexplained.

Critical Limitations and Nuances of Correlation

1. Correlation Does Not Imply Causation: An observed correlation between $X$ and $Y$ may stem from $X$ causing $Y$, $Y$ causing $X$ (directionality problem), or an unmeasured third variable $Z$ causing both (third-variable problem).
2. Restriction of Range: Artificially truncating the range of either variable dramatically attenuates (depresses) the observed correlation coefficient relative to the true population value (e.g., assessing the correlation between SAT scores and college GPA exclusively among Ivy League students).
3. Curvilinear Relationships: Pearson's $r$ evaluates strictly linear relationships. If two variables share a strong non-linear relationship—such as the Yerkes-Dodson Law relating physiological arousal to task performance (an inverted-U function)—the Pearson $r$ can be approximately $0.00$ despite a powerful, deterministic association.

Specialized Correlation Coefficients

Simple and Multiple Linear Regression

While correlation describes association, regression uses scores on one or more predictor variables to predict performance on a criterion variable.

Simple Linear Regression

The trajectory of a simple bivariate regression line is expressed as:

$\hat{Y} = bX + a$

• $\hat{Y}$ is the predicted value of the criterion variable.
• $b$ is the unstandardized regression slope, reflecting the predicted change in $Y$ for every one-unit increase in $X$:

$b = r \left( \frac{s_Y}{s_X} \right)$

• $a$ is the $Y$-intercept, representing the predicted value of $Y$ when $X = 0$:

$a = \bar{Y} - b\bar{X}$

• Ordinary Least Squares (OLS) Criterion: The regression line is fitted by minimizing the sum of squared vertical residuals (prediction errors) between observed data points and the line:

$\sum_{i=1}^N (Y_i - \hat{Y}_i)^2 = \text{minimum}$

Multiple Linear Regression

When multiple continuous predictors ($X_1, X_2, \dots, X_k$) predict a single continuous criterion ($Y$):

$\hat{Y} = b_1 X_1 + b_2 X_2 + \dots + b_k X_k + a$

• Standardized Beta Weights ($\beta$): When predictors are converted to standard scores ($z$-scores), their coefficients are expressed as $\beta$. A standardized beta weight represents the unique contribution of that specific predictor to the criterion, holding all other predictors constant.
• Multicollinearity: Occurs when two or more predictor variables in a multiple regression model are highly correlated with each other ($r > .80$). Multicollinearity inflates the standard errors of the regression coefficients, rendering individual $\beta$ estimates unstable and erratic.

6. Nonparametric Statistical Methods

When research data violate the assumptions of parametric tests—such as analyzing categorical counts, ordinal ranks, or small samples with extreme skewness and heteroscedasticity—researchers turn to nonparametric tests (distribution-free tests).

Chi-Square ($\chi^2$) Tests for Categorical Data

Chi-square tests evaluate frequencies of observations across discrete categorical bins.

1. Chi-Square Goodness-of-Fit Test

Evaluates whether an observed sample frequency distribution across a single nominal variable conforms to a theoretical or hypothesized population distribution:

$\chi^2 = \sum \frac{(O - E)^2}{E} \quad \text{with} \quad df = k - 1$

• $O$ is the observed frequency in each category.
• $E$ is the expected frequency under the null hypothesis.
• $k$ is the number of categories.

2. Chi-Square Test of Independence

Evaluates whether two categorical variables are statistically independent or contingency-associated in a two-way contingency table:

$\chi^2 = \sum \frac{(O - E)^2}{E} \quad \text{with} \quad df = (r - 1)(c - 1)$

• $r$ is the number of rows and $c$ is the number of columns.
• The expected frequency for each cell is computed as:

$E = \frac{(\text{Row Total}) \times (\text{Column Total})}{N_{\text{total}}}$

Nonparametric Rank-Order Tests and Their Parametric Counterparts