15.1 Classical Probability and Conditional Probability

15.1 Classical Probability and Conditional Probability

1. Random Experiments, Sample Space, and Events

In probability theory, a random experiment is an action or process that results in one of several possible outcomes, where the exact outcome cannot be predicted with certainty beforehand, but all possible outcomes are known in advance.

1.1 Sample Space and Elementary Events

• Sample Space ($\Omega$): The set of all possible outcomes of a random experiment is called the sample space, denoted by $\Omega$.
• Elementary Event ($\omega$): An individual element $\omega \in \Omega$ representing a single, indivisible outcome of the experiment.
• Event ($A$): Any subset $A \subseteq \Omega$ of the sample space is called an event. We say event $A$ occurs if the observed outcome $\omega$ belongs to $A$.

1.2 Event Operations and Relationships

1. Inclusion ($A \subseteq B$): If every outcome in $A$ is also in $B$, occurrence of $A$ implies occurrence of $B$.
2. Union ($A \cup B$ or $A + B$): The event containing all outcomes in $A$ or $B$ (or both). $A \cup B$ occurs if at least one of $A$ or $B$ occurs.
3. Intersection ($A \cap B$ or $AB$): The event containing outcomes common to both $A$ and $B$. $AB$ occurs if both $A$ and $B$ occur simultaneously.
4. Mutually Exclusive (Incompatible) Events: Events $A$ and $B$ are mutually exclusive if $A \cap B = \emptyset$, meaning they cannot occur together.
5. Complementary Events ($\overline{A}$): The event containing all outcomes in $\Omega$ not in $A$. Satisfies $A \cup \overline{A} = \Omega$ and $A \cap \overline{A} = \emptyset$.

---

2. Classical Probability Model

A probability model is called a Classical Probability Model (also known as a Laplace model) if it satisfies two essential conditions:

1. Finiteness: The sample space $\Omega$ contains a finite number of elementary events, $n(\Omega) = N < \infty$.
2. Equal Likelihood: Each elementary event in $\Omega$ has an equal probability of occurring.

2.1 Classical Probability Formula

For any event $A \subseteq \Omega$ containing $m$ elementary outcomes: $P(A) = \frac{n(A)}{n(\Omega)} = \frac{m}{N} = \frac{\text{Number of outcomes favorable to } A}{\text{Total number of possible outcomes in } \Omega}$

2.2 Core Axioms and Addition Rules

• Axiom of Non-negativity: $0 \le P(A) \le 1$ for any event $A$.
• Axiom of Certainty: $P(\Omega) = 1$, and for the impossible event $\emptyset$, $P(\emptyset) = 0$.
• Addition Rule for Mutually Exclusive Events: If $A \cap B = \emptyset$, then: $P(A \cup B) = P(A) + P(B)$
• General Addition Rule: For any two events $A$ and $B$: $P(A \cup B) = P(A) + P(B) - P(AB)$

---

3. Conditional Probability

Conditional probability measures the probability of an event $B$ occurring given that another event $A$ has already occurred.

3.1 Definition and Formula

Let $A$ and $B$ be two events associated with a sample space $\Omega$, with $P(A) > 0$. The conditional probability of $B$ given $A$, denoted by $P(B|A)$, is defined as: $P(B|A) = \frac{P(AB)}{P(A)}$

In a classical probability model with finite sample space $\Omega$: $P(B|A) = \frac{n(AB)}{n(A)}$ This effectively reduces the reference sample space from $\Omega$ to the subset $A$.

3.2 Basic Properties of Conditional Probability

1. Range: $0 \le P(B|A) \le 1$.
2. Certain Event: $P(\Omega|A) = 1$, and $P(A|A) = 1$.
3. Addition Rule for Mutually Exclusive Events: If $B_1 \cap B_2 = \emptyset$, then: $P(B_1 \cup B_2 | A) = P(B_1|A) + P(B_2|A)$
4. Complement Rule: $P(\overline{B}|A) = 1 - P(B|A)$.

3.3 Multiplication Rule of Probability

Rearranging the conditional probability definition yields the multiplication rule: $P(AB) = P(A) P(B|A) = P(B) P(A|B) \quad (\text{provided } P(A) > 0, P(B) > 0)$ For three events $A, B, C$: $P(ABC) = P(A) P(B|A) P(C|AB)$

---

4. Law of Total Probability and Bayes' Theorem

4.1 Partition of Sample Space

A set of events $B_1, B_2, \dots, B_n$ is said to form a partition of the sample space $\Omega$ if:

1. $B_i \cap B_j = \emptyset$ for all $i \neq j$ (pairwise mutually exclusive);
2. $\bigcup_{i=1}^n B_i = \Omega$ (collectively exhaustive);
3. $P(B_i) > 0$ for all $i = 1, 2, \dots, n$.

4.2 Law of Total Probability

If $B_1, B_2, \dots, B_n$ form a partition of $\Omega$, then for any event $A \subseteq \Omega$: $P(A) = \sum_{i=1}^n P(AB_i) = \sum_{i=1}^n P(B_i) P(A|B_i)$ Interpretation: The total probability of event $A$ is the weighted sum of its conditional probabilities across all possible background scenarios $B_i$.

4.3 Bayes' Theorem (Inverse Probability)

If $B_1, B_2, \dots, B_n$ form a partition of $\Omega$ and $P(A) > 0$, then for any $k \in {1, 2, \dots, n}$: $P(B_k|A) = \frac{P(B_k AB_k)}{P(A)} = \frac{P(B_k) P(A|B_k)}{\sum_{i=1}^n P(B_i) P(A|B_i)}$

• $P(B_k)$ is called the prior probability of scenario $B_k$.
• $P(A|B_k)$ is the likelihood of observing $A$ under scenario $B_k$.
• $P(B_k|A)$ is the posterior probability of scenario $B_k$ after observing event $A$.

---

5. Worked Gaokao Exam Examples

Example 1: Sampling Without Replacement (Urn Model)

Problem: An urn contains $5$ balls of identical size: $3$ white balls and $2$ red balls. Two balls are drawn sequentially without replacement.

1. Find the probability that the second ball drawn is red given that the first ball drawn is white.
2. Find the probability that the second ball drawn is red.

Detailed Solution: Let $W_1$ be the event "first ball is white", $R_1$ be "first ball is red", and $R_2$ be "second ball is red".

1. Conditional Probability $P(R_2|W_1)$: After drawing 1 white ball, 4 balls remain in the urn ($2$ white, $2$ red). $P(R_2|W_1) = \frac{2}{4} = \frac{1}{2}$ Alternatively, using the formula: $P(W_1) = \frac{3}{5}$. The event $W_1 R_2$ corresponds to selecting 1 white first and 1 red second out of ordered pairs. $P(W_1 R_2) = \frac{3 \times 2}{5 \times 4} = \frac{6}{20} = \frac{3}{10}$. $P(R_2|W_1) = \frac{P(W_1 R_2)}{P(W_1)} = \frac{3/10}{3/5} = \frac{1}{2}$

2. Unconditional Probability $P(R_2)$ via Total Probability Law: The events $W_1$ and $R_1$ partition the sample space for the first draw: $P(W_1) = \frac{3}{5}$, $P(R_1) = \frac{2}{5}$. Conditional probabilities for the second draw: $P(R_2|W_1) = \frac{2}{4} = \frac{1}{2}$, and $P(R_2|R_1) = \frac{1}{4}$. By the Law of Total Probability: $P(R_2) = P(W_1)P(R_2|W_1) + P(R_1)P(R_2|R_1) = \frac{3}{5} \times \frac{1}{2} + \frac{2}{5} \times \frac{1}{4} = \frac{3}{10} + \frac{1}{10} = \frac{4}{10} = \frac{2}{5}$ Key Takeaway: In sequential sampling without replacement without knowing prior outcomes, the marginal probability of drawing a specific color on any draw equals the initial proportion of that color ($\frac{2}{5}$).

Example 2: Diagnostic Quality Control (Bayes' Rule)

Problem: Factory A produces $60%$ of a product, and Factory B produces $40%$. The defect rate of Factory A is $2%$, and the defect rate of Factory B is $5%$. A product is randomly selected from the combined inventory and found to be defective.

1. What is the probability that the chosen product is defective?
2. What is the probability that this defective product was manufactured by Factory A?

Detailed Solution: Let $B_1$ be the event "produced by Factory A", $B_2$ be "produced by Factory B", and $D$ be "product is defective". Given: $P(B_1) = 0.60$, $P(B_2) = 0.40$, $P(D|B_1) = 0.02$, $P(D|B_2) = 0.05$.

1. Total Probability $P(D)$: $P(D) = P(B_1)P(D|B_1) + P(B_2)P(D|B_2) = (0.60 \times 0.02) + (0.40 \times 0.05) = 0.012 + 0.020 = 0.032$ The overall defect rate in the warehouse is $3.2%$.

2. Posterior Probability $P(B_1|D)$ via Bayes' Theorem: $P(B_1|D) = \frac{P(B_1)P(D|B_1)}{P(D)} = \frac{0.012}{0.032} = \frac{12}{32} = \frac{3}{8} = 0.375$ Even though Factory A produces $60%$ of total goods, because its quality control is higher (lower defect rate), a defective item has only a $37.5%$ chance of having originated from Factory A.