16.2 Addition Rule, Multiplication Rule, Independent Events & Conditional Probability

16.2 Addition Rule, Multiplication Rule, Independent Events & Conditional Probability

Axiomatic Foundations of Probability & The Complement Rule

Modern probability theory is formalized through the Kolmogorov Axioms. Given a sample space $S$ representing the set of all possible mutually exclusive outcomes of a random phenomenon, a probability measure $P$ assigns a real number to each event $A \subseteq S$ satisfying three foundational axioms:

1. Axiom 1 (Non-Negativity): For any event $A$, $P(A) \ge 0$.
2. Axiom 2 (Unitarity / Normalization): The probability of the entire sample space is unity: $P(S) = 1$.
3. Axiom 3 (Countable Additivity): If $A_1, A_2, A_3, \dots$ is a countable sequence of pairwise mutually exclusive (disjoint) events (where $A_i \cap A_j = \emptyset$ for all $i \ne j$), then: $P\left(\bigcup_{i=1}^\infty A_i\right) = \sum_{i=1}^\infty P(A_i)$

From these three axioms, all fundamental probability theorems emerge. Because the sample space can be partitioned into any event $A$ and its complement $A'$ (or $A^c$, representing all outcomes in $S$ not in $A$) such that $A \cap A' = \emptyset$ and $A \cup A' = S$, applying Axioms 2 and 3 yields $P(A \cup A') = P(A) + P(A') = P(S) = 1$. This establishes the Complement Rule: $P(A') = 1 - P(A) \iff P(A) = 1 - P(A')$

A vital application on the FTCE Mathematics 6-12 examination is solving 'at least one' compound probability problems. When an experiment involves $n$ independent trials, each with success probability $p$, finding the probability of obtaining at least one success directly requires summing multiple binomial terms. Applying the complement rule collapses the calculation to a single step: $P(\text{at least 1 success}) = 1 - P(\text{zero successes}) = 1 - (1 - p)^n$

---

The Addition Rule: Disjoint vs. Overlapping Events

The union of two events, denoted $A \cup B$ (or '$A$ or $B$'), represents the event that event $A$ occurs, event $B$ occurs, or both occur.

Mutually Exclusive (Disjoint) Events

Two events $A$ and $B$ are mutually exclusive (or disjoint) if they cannot occur simultaneously. Their intersection is the empty set: $A \cap B = \emptyset$, which implies $P(A \cap B) = 0$. By Axiom 3, the addition rule for mutually exclusive events is strictly additive: $P(A \cup B) = P(A) + P(B)$

Overlapping (Non-Mutually Exclusive) Events

When events $A$ and $B$ can occur concurrently ($A \cap B \ne \emptyset$), adding $P(A)$ and $P(B)$ counts the outcomes in the intersection twice. To correct for this double-counting, we subtract the joint probability, yielding the General Addition Rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Geometrically visualized through a Venn diagram, this corresponds to the principle of inclusion-exclusion. For three overlapping events $A, B$, and $C$, the rule extends to: $P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)$

---

Conditional Probability & The General Multiplication Rule

In many probabilistic systems, obtaining partial information alters the likelihood of an event. The conditional probability of an event $A$ given that event $B$ has occurred, denoted $P(A|B)$, is defined mathematically as: $P(A|B) = \frac{P(A \cap B)}{P(B)}, \quad \text{provided } P(B) > 0$

Conceptual Meaning of Conditioning

Conditioning on event $B$ physically collapses the sample space: outcomes outside of $B$ are discarded, making $B$ the new, restricted sample space. Within this restricted universe, the only favorable outcomes for $A$ are those residing in the intersection $A \cap B$. Dividing by $P(B)$ rescales the remaining probabilities so that $P(B|B) = 1$.

Rearranging the conditional probability definition yields the General Multiplication Rule, which calculates the joint probability of two events occurring together: $P(A \cap B) = P(B) \times P(A|B) = P(A) \times P(B|A)$ For a sequential chain of $k$ events, the rule generalizes via the chain rule of probability: $P(A_1 \cap A_2 \cap \dots \cap A_k) = P(A_1) \times P(A_2|A_1) \times P(A_3|A_1 \cap A_2) \times \dots \times P(A_k|A_1 \cap \dots \cap A_{k-1})$

---

Independent vs. Dependent Events and Sampling Paradigms

Two events $A$ and $B$ are statistically independent if the occurrence of one event provides zero information about the likelihood of the other event. Formally, $A$ and $B$ are independent if and only if: $P(A|B) = P(A) \quad \text{or} \quad P(B|A) = P(B)$ Substituting this condition into the General Multiplication Rule produces the Multiplication Rule for Independent Events: $P(A \cap B) = P(A) \times P(B)$ This product identity serves as the definitive mathematical test for independence on the examination: if $P(A \cap B) = P(A) \cdot P(B)$, the events are independent; if $P(A \cap B) \ne P(A) \cdot P(B)$, the events are dependent.

Crucial Conceptual Distinction: Mutually Exclusive vs. Independent

A ubiquitous trap for mathematics educators is confusing mutually exclusive events with independent events:

• Mutually exclusive means the events cannot happen together: $P(A \cap B) = 0$.
• Independent means the occurrence of one does not affect the other: $P(A \cap B) = P(A)P(B)$.

If two events $A$ and $B$ have non-zero probabilities ($P(A) > 0$ and $P(B) > 0$) and are mutually exclusive, can they ever be independent? No, never! $P(A \cap B) = 0 \ne P(A) \times P(B)$ Knowing that event $B$ occurred completely guarantees that event $A$ cannot occur ($P(A|B) = 0 \ne P(A)$). Mutually exclusive events with positive probabilities are maximally dependent.

Sampling With Replacement vs. Without Replacement

• Sampling with replacement: The sampled object is returned to the population before the next draw. Successive trials are independent, and probabilities remain constant across all stages.
• Sampling without replacement: The sampled object is removed from the population. Successive trials are dependent, as the pool size and item counts decrease, altering conditional probabilities at each subsequent stage.

---

Probability Tree Diagrams, Law of Total Probability & Bayes' Theorem

A probability tree diagram models sequential multi-stage random experiments:

1. The root branches into primary events representing a partition of the sample space ${B_1, B_2, \dots, B_k}$ with prior probabilities $P(B_i)$.
2. Secondary branches from each $B_i$ represent conditional probabilities $P(A|B_i)$ of an observed event $A$.
3. Multiplying probabilities along any path from root to leaf yields the joint probability of that specific path: $P(B_i \cap A) = P(B_i) \times P(A|B_i)$.

The Law of Total Probability

Because the branches ${B_1, B_2, \dots, B_k}$ form a complete partition of the sample space ($B_i \cap B_j = \emptyset$ and $\bigcup B_i = S$), the overall unconditional probability of event $A$ is the sum of the disjoint path intersections: $P(A) = \sum_{i=1}^k P(B_i \cap A) = \sum_{i=1}^k P(B_i) \times P(A|B_i)$

Bayes' Theorem: Inverting Conditional Probabilities

Bayes' Theorem calculates posterior probabilities—the probability that a specific underlying cause $B_j$ occurred given that observed outcome $A$ has taken place: $P(B_j|A) = \frac{P(B_j \cap A)}{P(A)} = \frac{P(B_j) \times P(A|B_j)}{\sum_{i=1}^k P(B_i) \times P(A|B_i)}$

---

Probability Rules & Axiomatic Formulas Reference Matrix

---

Worked Exemplar: Medical Diagnostic Screening & False Positive Analysis

Problem: A rare medical condition is present in $1%$ of a population ($P(D) = 0.01$). A laboratory diagnostic screening test has a sensitivity of $98%$ ($P(+|D) = 0.98$, the true positive rate) and a false positive rate of $3%$ ($P(+|D^c) = 0.03$, meaning specificity is $97%$). A randomly selected individual tests positive. What is the probability that this person actually has the condition?

Analytical Solution via Tree Diagram and Bayes' Theorem

1. Define the Prior Probabilities:

   • Patient has the disease: $P(D) = 0.01$
   • Patient does not have the disease: $P(D^c) = 1 - 0.01 = 0.99$

2. Identify the Conditional Branch Probabilities:

   • True Positive: $P(+|D) = 0.98$
   • False Negative: $P(-|D) = 1 - 0.98 = 0.02$
   • False Positive: $P(+|D^c) = 0.03$
   • True Negative: $P(-|D^c) = 1 - 0.03 = 0.97$

3. Compute Joint Path Probabilities for a Positive Test Result:

   • Path 1 (Has disease and tests positive): $P(D \cap +) = P(D) \times P(+|D) = (0.01)(0.98) = 0.0098$
   • Path 2 (Healthy and tests positive - false positive): $P(D^c \cap +) = P(D^c) \times P(+|D^c) = (0.99)(0.03) = 0.0297$

4. Compute Total Probability of Testing Positive (Law of Total Probability): $P(+) = P(D \cap +) + P(D^c \cap +) = 0.0098 + 0.0297 = 0.0395$

5. Evaluate Posterior Probability via Bayes' Theorem: $P(D|+) = \frac{P(D \cap +)}{P(+)} = \frac{0.0098}{0.0395} = \frac{98}{395} \approx 0.2481 \quad (24.81\%)$

Pedagogical Insight: Resolving the False Positive Paradox

Even though the screening test boasts $98%$ sensitivity and $97%$ specificity, a person testing positive has only an approximately $24.8%$ probability of having the disease! This counter-intuitive result stems from base rate neglect: because the disease is rare ($1%$), the large pool of healthy individuals ($99%$) produces nearly three times as many false positives ($0.0297$) as the tiny pool of diseased individuals produces true positives ($0.0098$).