> anishgoyal █


Notes › EENG 3421: Probability and Statistics for Engineers Lecture 3

Law of Total Probability and Bayes' Theorem

calendar_today   article 381 words   access_time 3 min   replay Modified

Table of Contents

Partitions

Definition 03.1 (Partition of the Sample Space).

A collection $\{B_1, B_2, \dots, B_m\}$ is a partition of $S$ if

  • $B_i \cap B_j = \emptyset$ for $i \ne j$
  • $\bigcup_{i=1}^m B_i = S$
$$ A = A \cap S = \bigcup_{i=1}^m (A \cap B_i) $$ $$ A = \bigcup_{i=1}^m C_i $$

Law of Total Probability

Theorem 03.2 (Law of Total Probability).

Let $\{B_1, \dots, B_m\}$ be a partition of $S$ with $P[B_i] > 0$ for all $i$ Then for any event $A$

$$ P[A] = \sum_{i=1}^m P[A \mid B_i] P[B_i] $$

Proof of Law of Total Probability. Since $A = \bigcup_{i=1}^m (A \cap B_i)$ and the sets are disjoint

$$ P[A] = \sum_{i=1}^m P[A \cap B_i] $$

By definition of conditional probability

$$ P[A \cap B_i] = P[A \mid B_i] P[B_i] $$

Joint and Marginal Probabilities

$$ P[A] = \sum_j P[A \cap B_j] $$

Bayes Theorem

Theorem 03.3 (Bayes Theorem).

For events $A$ and $B$ with $P[A] > 0$

$$ P[B \mid A] = \frac{P[A \mid B] P[B]}{P[A]} $$

Proof of Bayes Theorem. From the definition of conditional probability

$$ P[A \cap B] = P[B \mid A] P[A] = P[A \mid B] P[B] $$

Solving for $P[B \mid A]$ yields the result

Bayes Theorem with a Partition

Corollary 03.4 (Bayes Theorem for Partitions).

Let $\{B_1, \dots, B_m\}$ be a partition with $P[B_i] > 0$ Then

$$ P[B_k \mid A] = \frac{P[A \mid B_k] P[B_k]}{\sum_{i=1}^m P[A \mid B_i] P[B_i]} $$

Proof of Bayes Theorem for Partitions. Apply Bayes theorem and substitute

$$ P[A] = \sum_{i=1}^m P[A \mid B_i] P[B_i] $$

from the Law of Total Probability

Examples

Example 03.5.

Let $\{B_1, B_2, B_3\}$ denote machines producing resistors Let $A$ denote the event a resistor is acceptable Given

$$ P[A \mid B_1], \; P[A \mid B_2], \; P[A \mid B_3] $$

and

$$ P[B_1], \; P[B_2], \; P[B_3] $$

Then

$$ P[A] = \sum_{i=1}^3 P[A \mid B_i] P[B_i] $$
Example 03.6.

Given $A$ occurred, the probability the resistor came from machine $B_3$ is

$$ P[B_3 \mid A] = \frac{P[A \mid B_3] P[B_3]}{P[A]} $$

References

Sources

Graph

mindmap ↗