14.4 Random Variables, Expected Value, Variance, and Common Distributions

14.4 Random Variables, Expected Value, Variance, and Common Distributions

Random variables and probability distributions on the GRE Mathematics Subject Test require mastery of distribution functions, linearity of expectation, variance and covariance algebra, common discrete and continuous families, and asymptotic limit theorems.

---

Random Variables and Distribution Functions

A random variable $X$ on $(\Omega, \mathcal{F}, P)$ is a Borel-measurable function $X: \Omega \to \mathbb{R}$.

PMF, PDF, and CDF

• Discrete: Described by PMF $p_X(x) = P(X = x) \ge 0$ with $\sum_x p_X(x) = 1$.
• Continuous: Described by PDF $f_X(x) \ge 0$ with $\int_{-\infty}^\infty f_X(x) , dx = 1$, where $P(a \le X \le b) = \int_a^b f_X(x) , dx$. For any single point $c$, $P(X = c) = 0$.
• Cumulative Distribution Function (CDF): $F_X(x) = P(X \le x)$ for all variables. Every CDF satisfies:
  1. Monotonically non-decreasing: $x_1 < x_2 \implies F(x_1) \le F(x_2)$.
  2. Right-continuous: $\lim_{t \to x^+} F(t) = F(x)$.
  3. Limits: $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to \infty} F(x) = 1$.
  4. For continuous variables, $f_X(x) = F_X'(x)$ almost everywhere.

---

Expected Value and Linearity

The expected value $E[X]$ measures the center of the distribution: $E[X] = \begin{cases} \sum_x x \, p_X(x) & \text{discrete} \\[4pt] \int_{-\infty}^\infty x \, f_X(x) \, dx & \text{continuous} \end{cases}$ provided the sum or integral converges absolutely.

LOTUS and Unconditional Linearity

• LOTUS: $E[g(X)] = \int_{-\infty}^\infty g(x) f_X(x) , dx$ or $\sum_x g(x) p_X(x)$.
• Linearity: For any random variables $X_1, \dots, X_n$ and scalars $a_i, c \in \mathbb{R}$: $E\left[\sum_{i=1}^n a_i X_i + c\right] = \sum_{i=1}^n a_i E[X_i] + c$ Linearity holds unconditionally, requiring no independence assumptions.

---

Variance, Covariance, and Independence

The variance measures dispersion: $\operatorname{Var}(X) = E[(X - \mu)^2] = E[X^2] - (E[X])^2$

• Scaling: $\operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X)$. Standard deviation is $\sigma = \sqrt{\operatorname{Var}(X)}$.

Covariance and Variance of Sums

The covariance is $\operatorname{Cov}(X, Y) = E[XY] - E[X]E[Y]$. $\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y)$ $\operatorname{Var}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \operatorname{Var}(X_i) + 2 \sum_{i < j} \operatorname{Cov}(X_i, X_j)$

• Independence: If $X$ and $Y$ are independent, $\operatorname{Cov}(X, Y) = 0$, so $\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$.
• Exam Alert: $\operatorname{Cov}(X, Y) = 0$ does not imply independence (e.g., $X \sim \text{Uniform}(-1, 1)$ and $Y = X^2$).

---

Catalogue of Common Distributions

Memoryless Property

Geometric and Exponential distributions uniquely satisfy $P(X > s + t \mid X > s) = P(X > t)$ for $s, t \ge 0$.

---

Tail Inequalities and Limit Theorems

• Markov's Inequality: For $X \ge 0$ and $a > 0$, $P(X \ge a) \le \frac{E[X]}{a}$.
• Chebyshev's Inequality: For finite variance $\sigma^2$, $P(|X - \mu| \ge \epsilon) \le \frac{\sigma^2}{\epsilon^2}$ or $P(|X - \mu| \ge k\sigma) \le \frac{1}{k^2}$.
• Weak Law of Large Numbers (WLLN): For i.i.d. $X_i$ with mean $\mu$, sample mean $\bar{X}_n \xrightarrow{P} \mu$.
• Central Limit Theorem (CLT): For i.i.d. $X_i$ with mean $\mu$ and variance $\sigma^2 \in (0, \infty)$: $Z_n = \frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} \xrightarrow{d} \mathcal{N}(0, 1)$

---

Step-by-Step Worked Problem

Problem: A bin contains 50 lightbulbs, of which 10 are defective. If 5 bulbs are selected at random without replacement, let $X$ be the number of defective bulbs chosen. Find $E[X]$ and $\operatorname{Var}(X)$.

Solution:

1. Indicator representation: Let $I_j = 1$ if the $j$-th drawn bulb is defective, and 0 otherwise ($j = 1, \dots, 5$). Then $X = \sum_{j=1}^5 I_j$.
2. Expectation: By symmetry, $P(I_j = 1) = 10/50 = 1/5$. By linearity of expectation: $E[X] = \sum_{j=1}^5 E[I_j] = 5 \times \frac{1}{5} = 1$
3. Variance components: Since $I_j^2 = I_j$, $\operatorname{Var}(I_j) = \frac{1}{5} - \frac{1}{25} = \frac{4}{25}$. For $j \neq k$, $P(I_j = 1, I_k = 1) = \frac{10}{50} \times \frac{9}{49} = \frac{9}{245}$. $\operatorname{Cov}(I_j, I_k) = \frac{9}{245} - \left(\frac{1}{5}\right)^2 = \frac{9}{245} - \frac{1}{25} = -\frac{4}{1225}$
4. Summing variance: $\operatorname{Var}(X) = 5\left(\frac{4}{25}\right) + 20\left(-\frac{4}{1225}\right) = \frac{4}{5} - \frac{16}{245} = \frac{180}{245} = \frac{36}{49}$

---

GRE Exam Traps & Pitfalls

Trap 1: Dropping $a^2$ in Variance Scaling Remember $\operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X)$. Scalar constants pull out squared, while additive constants vanish. $\operatorname{Var}(-X) = \operatorname{Var}(X)$, never $-\operatorname{Var}(X)$.

Trap 2: Assuming Zero Covariance Implies Independence Zero covariance implies no linear relationship, but non-linear deterministic dependence (like $Y = X^2$ for symmetric $X$) still produces $\operatorname{Cov}(X, Y) = 0$.

Trap 3: Adding Variances of Dependent Variables $\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$ requires uncorrelatedness. Without it, the cross-term $2\operatorname{Cov}(X, Y)$ must be included.

Trap 4: Geometric Support Ambiguity Verify whether Geometric($p$) counts total trials ($k \in {1, 2, \dots}, E = 1/p$) or failures before success ($k \in {0, 1, \dots}, E = (1-p)/p$).