5.3 Probability Theory: Independent Events, Conditional Probability & Bayes' Basics

5.3 Probability Theory: Independent Events, Conditional Probability & Bayes' Basics

Probability theory quantifies uncertainty, providing the mathematical framework for decision analysis, economics, and finance. On the Bocconi Admission Test, probability questions demand rigorous application of set-theoretic axioms, multi-stage conditional reasoning, and combinatorial sample-space evaluation. Rather than relying on intuitive heuristics—which test writers deliberately exploit with subtle traps—candidates must apply formal addition rules, conditional formulas, and Bayes' Theorem systematically to secure rapid, exact solutions.

---

1. Axiomatic Foundations and Classical Probability

A formal probability model is defined by a sample space $S$ (the set of all mutually exclusive elementary outcomes) and an event $E \subseteq S$ (a subset of outcomes).

The Kolmogorov Axioms

Every valid probability measure $P$ assigns a real number to each event $E$ satisfying three non-negotiable axioms:

1. Non-negativity: $P(E) \ge 0$ for all $E \subseteq S$.
2. Normalization (Certainty): $P(S) = 1$.
3. Additivity: For any sequence of mutually disjoint (exclusive) events $E_1, E_2, \dots$ where $E_i \cap E_j = \emptyset$ for all $i \neq j$: $P\left(\bigcup_{i=1}^\infty E_i\right) = \sum_{i=1}^\infty P(E_i)$

Classical Definition of Probability (Equally Likely Outcomes)

When the sample space $S$ is finite and all elementary outcomes are symmetric and equiprobable: $P(E) = \frac{|E|}{|S|} = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$

Immediate Deductions

• Impossible Event: $P(\emptyset) = 0$.
• Universal Bounds: $0 \le P(E) \le 1$ for any event $E$.
• Complementary Event Rule: The probability that $E$ does not occur is: $P(E^c) = 1 - P(E)$

---

2. The Addition Rule & Mutually Exclusive Events

When evaluating the probability that event $A$ or event $B$ occurs (the union $A \cup B$):

General Addition Rule

For any two events $A$ and $B$: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Mutually Exclusive (Disjoint) Events

Events $A$ and $B$ are mutually exclusive if they cannot occur simultaneously: $A \cap B = \emptyset$. Thus, $P(A \cap B) = 0$, simplifying the addition rule to: $P(A \cup B) = P(A) + P(B)$

---

3. Independent Events vs. Mutually Exclusive Events: The Cardinal Exam Trap

One of the most frequent errors on the Bocconi test is conflating 'independent' events with 'mutually exclusive' events.

Statistical Independence

Two events $A$ and $B$ are statistically independent if and only if the occurrence of one provides zero information about the probability of the other: $P(A \cap B) = P(A) \cdot P(B)$ Equivalently, $P(A \mid B) = P(A)$ and $P(B \mid A) = P(B)$.

Independence Invariance Theorem

If events $A$ and $B$ are independent, then:

• $A$ and $B^c$ are independent: $P(A \cap B^c) = P(A) \cdot P(B^c) = P(A)(1 - P(B))$.
• $A^c$ and $B$ are independent: $P(A^c \cap B) = P(A^c) \cdot P(B) = (1 - P(A))P(B)$.
• $A^c$ and $B^c$ are independent: $P(A^c \cap B^c) = P(A^c) \cdot P(B^c) = (1 - P(A))(1 - P(B))$.

The Incompatibility Theorem

Can two non-trivial events ($P(A) > 0, P(B) > 0$) be both mutually exclusive and independent?

• If they are mutually exclusive $\implies P(A \cap B) = 0$.
• If they are independent $\implies P(A \cap B) = P(A) \cdot P(B) > 0$.
• Because $0 \neq P(A)P(B)$, two events with non-zero probabilities can NEVER be both mutually exclusive and independent!

---

4. Conditional Probability & The Multiplication Rule

The conditional probability of event $A$ given that event $B$ has already occurred (with $P(B) > 0$) is defined as: $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$ Intuition: Knowing that event $B$ has occurred collapses the admissible universe from $S$ to $B$. The only outcomes favorable to $A$ are those lying inside the intersection $A \cap B$.

General Multiplication Rule

Rearranging the conditional probability definition gives the probability of a joint occurrence: $P(A \cap B) = P(B) \cdot P(A \mid B) = P(A) \cdot P(B \mid A)$

General Chain Rule for $n$ Events

For sequential experiments across multiple dependent stages: $P(A_1 \cap A_2 \cap \dots \cap A_n) = P(A_1) \cdot P(A_2 \mid A_1) \cdot P(A_3 \mid A_1 \cap A_2) \cdots P(A_n \mid A_1 \cap \dots \cap A_{n-1})$

---

5. Law of Total Probability and Bayes' Theorem

Law of Total Probability

Let $B_1, B_2, \dots, B_k$ form a partition of the sample space $S$ (mutually exclusive: $B_i \cap B_j = \emptyset$, and exhaustive: $\bigcup B_i = S$). For any event $A$: $P(A) = \sum_{i=1}^k P(A \cap B_i) = \sum_{i=1}^k P(A \mid B_i) P(B_i)$

Bayes' Theorem (Inversion of Conditioning)

Bayes' Theorem calculates the posterior probability of a specific cause $B_j$ given observed effect $A$: $P(B_j \mid A) = \frac{P(A \cap B_j)}{P(A)} = \frac{P(A \mid B_j) P(B_j)}{\sum_{i=1}^k P(A \mid B_i) P(B_i)}$

• Prior Probability $P(B_j)$: Base-rate probability before new evidence is known.
• Likelihood $P(A \mid B_j)$: Probability that evidence $A$ occurs under hypothesis $B_j$.
• Posterior Probability $P(B_j \mid A)$: Revised probability of hypothesis $B_j$ incorporating evidence $A$.

---

6. Multi-Stage Experiments: Sampling With vs. Without Replacement

When drawing items from a finite population of size $N$:

Sampling With Replacement (Independent Trials)

• Each drawn item is returned to the pool before the next draw.
• The total count $N$ and subgroup proportions remain constant.
• Successive draws are independent: $P(A_2 \mid A_1) = P(A_2)$.

Sampling Without Replacement (Dependent Trials)

• Drawn items are removed permanently.
• The population size drops ($N, N - 1, N - 2, \dots$) and composition changes.
• Successive draws are dependent: conditional probabilities update after every stage.

The 'At Least One' Probability Shortcut

In $n$ independent trials where each trial has probability of success $p$ (and failure $q = 1 - p$): $P(\text{at least 1 success}) = 1 - P(\text{all failures}) = 1 - (1 - p)^n$ Example: If a fair die is rolled 4 times, what is the probability of obtaining at least one 6? $P(\text{at least one 6}) = 1 - \left(\frac{5}{6}\right)^4 = 1 - \frac{625}{1296} = \frac{671}{1296} \approx 0.5177$

---

7. Common Bocconi Traps & Fast Shortcuts

1. The Inverse Conditioning Fallacy: Assuming $P(A \mid B) = P(B \mid A)$. For instance, $P(\text{Fever} \mid \text{Malaria})$ is nearly $1.0$, but $P(\text{Malaria} \mid \text{Fever})$ is very small because many common illnesses cause fever.
2. The Gambler's Fallacy: Believing that past independent outcomes influence future trials. If a fair coin lands on Heads 6 times in a row, the probability of Heads on the 7th toss remains exactly $\frac{1}{2}$.
3. Failure to Adjust Denominators in Without-Replacement Problems: When selecting 2 cards from a standard 52-card deck, the second draw operates over a denominator of 51, never 52.

---

8. Worked Step-by-Step Examples

Example 1: Sampling Without Replacement (Urn Model)

An urn contains 5 red, 4 blue, and 3 green marbles (12 marbles total). Two marbles are drawn sequentially without replacement. What is the probability that both marbles are of different colors?

1. Use the Complementary Approach: $P(\text{different colors}) = 1 - P(\text{same color})$
2. Compute Total Pairs: Total ways to draw 2 marbles from 12 without order: $\binom{12}{2} = \frac{12 \times 11}{2} = 66$
3. Compute Pairs of Same Color:
   • Both Red: $\binom{5}{2} = \frac{5 \times 4}{2} = 10$
   • Both Blue: $\binom{4}{2} = \frac{4 \times 3}{2} = 6$
   • Both Green: $\binom{3}{2} = 3$
   • Total monochrome pairs = $10 + 6 + 3 = 19$.
4. Calculate Final Probability: $P(\text{same color}) = \frac{19}{66} \implies P(\text{different colors}) = 1 - \frac{19}{66} = \frac{47}{66}$

Example 2: Industrial Defect Identification via Bayes' Theorem

A manufacturing plant uses Machine A to produce 60% of its components and Machine B for the remaining 40%. Machine A has a defect rate of 2%, while Machine B has a defect rate of 5%. If a randomly selected component is found to be defective, what is the probability it was manufactured by Machine B?

1. Identify Given Priors and Conditionals: $P(A) = 0.60, \quad P(B) = 0.40$ $P(D \mid A) = 0.02, \quad P(D \mid B) = 0.05$
2. Calculate Total Probability of Defect $P(D)$: $P(D) = P(D \mid A)P(A) + P(D \mid B)P(B) = (0.02)(0.60) + (0.05)(0.40) = 0.012 + 0.020 = 0.032$
3. Apply Bayes' Theorem for $P(B \mid D)$: $P(B \mid D) = \frac{P(D \mid B)P(B)}{P(D)} = \frac{0.020}{0.032} = \frac{20}{32} = \frac{5}{8} = 0.625 \quad (62.5\%)$

Example 3: Independence Verification and Joint Evaluation

Events $A$ and $B$ are statistically independent with $P(A) = 0.4$ and $P(B) = 0.5$. Find $P(A^c \cap B^c)$.

1. Method 1 (Independence of Complements): $P(A^c) = 1 - 0.4 = 0.6$ and $P(B^c) = 1 - 0.5 = 0.5$. Since $A$ and $B$ are independent, $A^c$ and $B^c$ are independent: $P(A^c \cap B^c) = P(A^c) \cdot P(B^c) = 0.6 \times 0.5 = 0.30$
2. Method 2 (De Morgan's Identity): $A^c \cap B^c = (A \cup B)^c$. $P(A \cup B) = P(A) + P(B) - P(A \cap B) = 0.4 + 0.5 - (0.4 \times 0.5) = 0.9 - 0.2 = 0.7$. $P((A \cup B)^c) = 1 - 0.7 = 0.30$