Total probability and Bayes’ theorem

The essentials

Events E₁, E₂, …, Eₙ form a partition of the sample space if they are pairwise disjoint, together cover the whole sample space, and each has non-zero probability. Think of them as the possible "causes" or sources, such as which bag was chosen or which machine made an item.

Theorem of total probability: for any event A, P(A) = P(E₁)P(A|E₁) + P(E₂)P(A|E₂) + … + P(Eₙ)P(A|Eₙ).

Bayes' theorem reverses the conditioning. Once A has been observed, the probability that it came from cause Eᵢ is P(Eᵢ|A) = P(Eᵢ)P(A|Eᵢ) / [Σ P(Eⱼ)P(A|Eⱼ)]. The P(Eᵢ) are called prior probabilities and the P(Eᵢ|A) are posterior probabilities.

Key points

• The denominator in Bayes' theorem is exactly the total probability P(A).
• A tree diagram shows each branch product P(Eᵢ)P(A|Eᵢ); Bayes' answer is "my branch ÷ all branches leading to A".
• The posterior probabilities over all causes add to 1.
• Example: machines making 70% and 30% of output with defect rates 2% and 5% give P(defective) = 0.014 + 0.015 = 0.029.

Exam tips

• Write the given data in two columns: P(Eᵢ) and P(A|Eᵢ). The rest is arithmetic.
• Convert percentages to decimals before multiplying.

Common mistakes

• Confusing P(A|Eᵢ) (given in the question) with P(Eᵢ|A) (asked for).
• Forgetting to include every cause in the denominator.