14.3 Probability Spaces, Conditional Probability, and Bayes' Theorem

14.3 Probability Spaces, Conditional Probability, and Bayes' Theorem

Probability on the GRE Mathematics Subject Test evaluates Kolmogorov's axiomatic framework, conditional probability, algebraic independence, the Law of Total Probability, and Bayesian inference. Precise handling of conditional spaces, event partitions, and prior-to-posterior probability updates is essential for success.

---

Axiomatic Probability Spaces

A formal probability model is a triple $(\Omega, \mathcal{F}, P)$:

1. Sample Space $\Omega$: The set of all possible outcomes.
2. $\sigma$-Algebra $\mathcal{F}$: A collection of events containing $\Omega$, closed under complementation ($A \in \mathcal{F} \implies A^c \in \mathcal{F}$) and countable unions ($\bigcup_{i=1}^\infty A_i \in \mathcal{F}$).
3. Probability Measure $P$: A function $P: \mathcal{F} \to [0, 1]$ satisfying Kolmogorov's Axioms:
   • Non-negativity: $P(A) \ge 0$ for all $A \in \mathcal{F}$.
   • Normalization: $P(\Omega) = 1$.
   • Countable Additivity: For pairwise disjoint events ${A_i}{i=1}^\infty$, $P\left(\bigcup{i=1}^\infty A_i\right) = \sum_{i=1}^\infty P(A_i)$.

Derived Properties

• $P(\emptyset) = 0$ and $P(A^c) = 1 - P(A)$
• Monotonicity: $A \subseteq B \implies P(A) \le P(B)$ and $P(B \setminus A) = P(B) - P(A)$
• Inclusion-Exclusion: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
• Boole's Inequality (Union Bound): $P\left(\bigcup_{i=1}^n A_i\right) \le \sum_{i=1}^n P(A_i)$
• Continuity of Measure: If $A_n \uparrow A$, then $\lim P(A_n) = P(A)$; if $A_n \downarrow A$, then $\lim P(A_n) = P(A)$.

---

Conditional Probability and the Chain Rule

For events $A, B$ with $P(B) > 0$, the conditional probability of $A$ given $B$ is: $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$ The conditional measure $P(\cdot \mid B)$ is a valid probability measure satisfying all Kolmogorov axioms.

Multiplication (Chain) Rule

$P(A \cap B) = P(A \mid B) P(B)$ For $n$ events with $P(A_1 \cap \dots \cap A_{n-1}) > 0$: $P(A_1 \cap \dots \cap A_n) = P(A_1) P(A_2 \mid A_1) P(A_3 \mid A_1 \cap A_2) \cdots P(A_n \mid A_1 \cap \dots \cap A_{n-1})$

---

Independent Events vs. Mutually Exclusive Events

Pairwise vs. Mutual Independence

Events ${A_1, \dots, A_n}$ are mutually independent if for every non-empty sub-index set $J \subseteq {1, \dots, n}$: $P\left(\bigcap_{j \in J} A_j\right) = \prod_{j \in J} P(A_j)$ Pairwise independence ($P(A_i \cap A_j) = P(A_i)P(A_j)$ for all $i \neq j$) does not imply mutual independence. A classic counterexample is two fair coin tosses where $A$ is heads on toss 1, $B$ is heads on toss 2, and $C$ is both tosses match.

---

Law of Total Probability

If events ${B_1, \dots, B_n}$ form a partition of $\Omega$ (disjoint with $\bigcup B_i = \Omega$ and $P(B_i) > 0$), then for any event $A$: $P(A) = \sum_{i=1}^n P(A \cap B_i) = \sum_{i=1}^n P(A \mid B_i) P(B_i)$

---

Bayes' Theorem and Posterior Calculations

Bayes' Theorem updates prior probabilities $P(B_k)$ to posterior probabilities $P(B_k \mid A)$ upon observing evidence $A$: $P(B_k \mid A) = \frac{P(A \mid B_k) P(B_k)}{P(A)} = \frac{P(A \mid B_k) P(B_k)}{\sum_{i=1}^n P(A \mid B_i) P(B_i)}$

• Prior $P(B_k)$: Baseline probability before evidence $A$.
• Likelihood $P(A \mid B_k)$: Probability of evidence given state $B_k$.
• Posterior $P(B_k \mid A)$: Updated belief given evidence $A$.

Diagnostic Testing and the Base Rate Fallacy

• Sensitivity: $P(+ \mid \text{Disease}) = 1 - \text{False Negative Rate}$
• Specificity: $P(- \mid \text{Healthy}) = 1 - \text{False Positive Rate}$
• Positive Predictive Value (PPV): $P(\text{Disease} \mid +) = \frac{P(+ \mid \text{Disease}) P(\text{Disease})}{P(+ \mid \text{Disease}) P(\text{Disease}) + P(+ \mid \text{Healthy}) P(\text{Healthy})}$ When prevalence $P(\text{Disease})$ is rare, false positives from the large healthy subpopulation can swamp true positives, resulting in a low PPV despite high test accuracy.

---

Step-by-Step Worked Problem

Problem: Box 1 has 4 green and 2 yellow balls. Box 2 has 3 green and 5 yellow balls. A die is rolled: if 1 or 2 appears, Box 1 is chosen; otherwise, Box 2 is chosen. Two balls are drawn without replacement, and both are green ($GG$). Find $P(B_1 \mid GG)$.

Solution:

1. Priors: $P(B_1) = 2/6 = 1/3$ and $P(B_2) = 4/6 = 2/3$.
2. Likelihoods:
   • Box 1 (total 6): $P(GG \mid B_1) = \binom{4}{2} / \binom{6}{2} = 6/15 = 2/5$.
   • Box 2 (total 8): $P(GG \mid B_2) = \binom{3}{2} / \binom{8}{2} = 3/28$.
3. Law of Total Probability: $P(GG) = \left(\frac{2}{5}\right)\left(\frac{1}{3}\right) + \left(\frac{3}{28}\right)\left(\frac{2}{3}\right) = \frac{2}{15} + \frac{1}{14} = \frac{43}{210}$
4. Bayes' Theorem: $P(B_1 \mid GG) = \frac{P(GG \mid B_1) P(B_1)}{P(GG)} = \frac{2/15}{43/210} = \frac{2}{15} \times \frac{210}{43} = \frac{28}{43}$

---

GRE Exam Traps & Pitfalls

Trap 1: Confusing Independence with Disjointness Disjoint events satisfy $P(A \cap B) = 0$. For events with positive probability, $P(A)P(B) > 0 \neq 0$, making disjoint events strictly dependent.

Trap 2: Transposing Conditionals (Prosecutor's Fallacy) Never equate $P(A \mid B)$ with $P(B \mid A)$. Conflating sensitivity $P(+ \mid \text{Disease})$ with posterior $P(\text{Disease} \mid +)$ ignores the prior base rate.

Trap 3: Pairwise vs. Mutual Independence Checking that all pairs satisfy $P(A_i \cap A_j) = P(A_i)P(A_j)$ does not verify full independence. Complete independence requires all higher-order subset products to hold.

Trap 4: Omitting Shift in Without-Replacement Likelihoods When sampling without replacement, conditional probabilities shift after each draw ($4/6 \times 3/5$). Applying independent product probabilities produces wrong likelihoods.