2.4 Hypothesis Testing, Goodness of Fit, and Linear Regression

2.4 Hypothesis Testing, Goodness of Fit, and Linear Regression

Hypothesis testing asks whether an observed difference is larger than sampling variation can plausibly explain, and regression asks how well a straight line describes a data set. Both are examined under the NCEES sub-topics sample distributions and sizes and goodness of fit, and both are formula-driven once the correct test or statistic is identified.

1. Hypothesis Testing Methodology

Hypothesis testing evaluates competing claims regarding a population parameter using sample evidence.

Step-by-Step Procedure

1. Formulate Hypotheses:
   • Null Hypothesis ($H_0$): Baseline status-quo claim (e.g., $H_0: \mu = \mu_0$ or $H_0: \mu \le \mu_0$).
   • Alternative Hypothesis ($H_a$): Research hypothesis (e.g., $H_a: \mu e \mu_0$, $H_a: \mu > \mu_0$, or $H_a: \mu < \mu_0$).
2. Select Significance Level ($\alpha$): Common values are $\alpha = 0.05$ (5%) or $\alpha = 0.01$ (1%).
3. Compute Test Statistic:
   • $Z$-test (Known $\sigma$ or $n \ge 30$): $Z_{calc} = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}}$
   • $t$-test (Unknown $\sigma$, $n < 30$): $t_{calc} = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}$
4. Determine Critical Region / $p$-value:
   • Critical Value Method: Compare test statistic to $z_{crit}$ or $t_{crit}$.
   • $p$-value Method: Compute $p$-value $= P(\text{observing test statistic} \mid H_0 \text{ is true})$.
5. State Decision Rule:
   • If $|Z_{calc}| > Z_{crit}$ (or $p\text{-value} < \alpha$), Reject $H_0$.
   • If $|Z_{calc}| \le Z_{crit}$ (or $p\text{-value} \ge \alpha$), Fail to Reject $H_0$.

Decision Errors in Hypothesis Testing

Worked Engineering Example: One-Tailed $Z$-Test

Problem: An environmental compliance regulation limits lead content in discharge water to $\mu_0 \le 15.0\text{ ppb}$. An inspector tests $n = 36$ samples, finding a sample mean $\bar{x} = 16.2\text{ ppb}$. The known population standard deviation is $\sigma = 3.6\text{ ppb}$. Test at $\alpha = 0.05$ significance level whether the plant violates effluent standards ($Z_{crit} = 1.645$ for upper one-tailed test).

Solution: Step 1: Set Hypotheses: $H_0: \mu \le 15.0\text{ ppb} \quad \text{vs.} \quad H_a: \mu > 15.0\text{ ppb}$

Step 2: Calculate Test Statistic $Z_{calc}$: $Z_{calc} = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}} = \frac{16.2 - 15.0}{3.6 / \sqrt{36}} = \frac{1.2}{3.6 / 6} = \frac{1.2}{0.60} = 2.00$

Step 3: Compare to Critical Value: Since $Z_{calc} = 2.00 > Z_{crit} = 1.645$, the calculated value lies in the critical rejection region.

Step 4: Conclusion: Reject $H_0$ at the $\alpha = 0.05$ level ($p$-value $= 1 - \Phi(2.00) = 0.0228 < 0.05$). There is sufficient evidence to conclude that the plant's mean discharge lead concentration significantly exceeds 15.0 ppb.

2. Simple Linear Regression and Correlation

Simple linear regression models the relationship between an independent predictor variable $x$ and a dependent response variable $y$.

Least-Squares Regression Equations

$\text{Model: } \hat{y} = a + b x$ Where $b$ is the slope and $a$ is the $y$-intercept.

NCEES Formulas for Sum of Squares

$S_{xx} = \sum_{i=1}^n (x_i - \bar{x})^2 = \sum x_i^2 - \frac{(\sum x_i)^2}{n}$ $S_{yy} = \sum_{i=1}^n (y_i - \bar{y})^2 = \sum y_i^2 - \frac{(\sum y_i)^2}{n}$ $S_{xy} = \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y}) = \sum x_i y_i - \frac{(\sum x_i)(\sum y_i)}{n}$

Least-Squares Parameters

$\text{Slope: } b = \frac{S_{xy}}{S_{xx}}$ $\text{Intercept: } a = \bar{y} - b \bar{x}$

Sample Correlation Coefficient ($r$) and Coefficient of Determination ($R^2$)

• Sample Correlation Coefficient ($r$): $r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}}, \quad -1 \le r \le 1$
• Coefficient of Determination ($R^2$): $R^2 = r^2$ $R^2$ represents the proportion of total variation in $y$ explained by the linear relationship with $x$. For example, $R^2 = 0.81$ means 81% of the variation in $y$ is explained by $x$.

Worked Engineering Example: Stress-Strain Calibration Curve

Problem: An experiment measures force $x$ (kN) and resulting deflection $y$ (mm) for a structural bracket ($n = 5$ data points):

• $S_{xx} = 20.0\text{ kN}^2$
• $S_{yy} = 80.0\text{ mm}^2$
• $S_{xy} = 36.0\text{ kN}\cdot\text{mm}$
• $\bar{x} = 10.0\text{ kN}, \quad \bar{y} = 25.0\text{ mm}$

Find slope $b$, intercept $a$, regression equation $\hat{y}$, correlation coefficient $r$, coefficient of determination $R^2$, and predicted deflection for $x = 15.0\text{ kN}$.

Solution: Step 1: Compute Slope $b$ and Intercept $a$: $b = \frac{S_{xy}}{S_{xx}} = \frac{36.0}{20.0} = 1.80\text{ mm/kN}$ $a = \bar{y} - b \bar{x} = 25.0 - (1.80 \times 10.0) = 25.0 - 18.0 = 7.0\text{ mm}$ $\text{Regression Line: } \hat{y} = 7.0 + 1.80 x$

Step 2: Compute Correlation $r$ and $R^2$: $r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} = \frac{36.0}{\sqrt{20.0 \times 80.0}} = \frac{36.0}{\sqrt{1600}} = \frac{36.0}{40.0} = 0.900$ $R^2 = r^2 = (0.900)^2 = 0.8100 \quad (81.0\% \text{ of variance explained})$

Step 3: Predict Response at $x = 15.0\text{ kN}$: $\hat{y}(15.0) = 7.0 + (1.80 \times 15.0) = 7.0 + 27.0 = 34.0\text{ mm}$

Goodness of Fit: What Each Statistic Actually Tells You

NCEES lists goodness of fit as its own sub-topic and names three measures: the correlation coefficient, standard errors, and $R^2$. They answer different questions, and the exam tests whether you know which is which.

$R^2 = r^2 = 1 - \frac{SSE}{SST}, \qquad S_e = \sqrt{\frac{SSE}{n-2}}$

where $SSE = \sum(y_i - \hat{y}_i)^2$ is the residual sum of squares and $SST = \sum(y_i - \bar{y})^2$ is the total sum of squares. The $n-2$ divisor reflects the two parameters (slope and intercept) already estimated from the data.

The Three Traps NCEES Builds Into Regression Items

1. Sign confusion. $R^2 = 0.81$ is consistent with $r = +0.9$ or $r = -0.9$. You must read the slope's sign to know the direction. If a question gives you $R^2$ and asks for $r$, it must also tell you whether the trend rises or falls.
2. "High $R^2$ means the model is right." It does not. Fitting a straight line to genuinely curved data can still yield $R^2 > 0.9$ over a narrow range while extrapolating disastrously. Goodness of fit is a statement about residuals inside the sampled range only.
3. Extrapolation. A calibration curve fit over $0$–$50\ \text{kPa}$ says nothing about behavior at $200\ \text{kPa}$.

Interpretation script for the exam: "$R^2 = 0.94$ means 94% of the observed variation in the response is accounted for by the linear model; the remaining 6% is residual scatter." That sentence, with the right number substituted, is the correct answer to most conceptual regression items.

3. NCEES Reference Summary Table