Bayes' Theorem (with Example!)

Introduction to Bayes' Theorem
📌 Bayes' theorem is a cornerstone concept in statistics, crucial for machine learning, and used extensively in solving inverse problems in engineering.
💡 It allows updating the probability of an event $A$ occurring given new, partial information about another event $B$ (conditional probability $P(A|B)$).
📏 The basic definition states that $P(A|B) = \frac{P(A \text{ and } B)}{P(B)}$, which implies the multiplication law: $P(A \text{ and } B) = P(A|B) \cdot P(B)$.

Bayes' Theorem and Inverse Problems
🔄 Bayes' theorem is vital for inverse problems, where one computes the probability of an underlying cause (like disease, $B$) given an observable measurement (like a test result, $A$), which is often more useful than the forward problem ($P(A|B)$).
📜 The standard form derived from setting two expressions for $P(A \text{ and } B)$ equal is: $$P(B|A) = \frac{P(A|B) \cdot P(B)}{P(A)}$$
🔢 The components of the theorem are named: $P(B|A)$ is the posterior, $P(B)$ is the prior (initial belief), and $P(A|B)$ is the update factor.

Sequential Updates and Total Probability
🔄 Bayesian statistics relies on sequential updates: the posterior from one experiment becomes the prior for the next, allowing continuous refinement of probability estimations as new data is gathered.
➕ A more comprehensive formulation uses the law of total probability for the denominator: $$P(B|A) = \frac{P(A|B) \cdot P(B)}{P(A|B) \cdot P(B) + P(A|B^c) \cdot P(B^c)}$$
🌐 This can be generalized for multiple disjoint covering sets $B_j$: $$P(B_j|A) = \frac{P(A|B_j) \cdot P(B_j)}{\sum_i P(A|B_i) \cdot P(B_i)}$$

Application Example: Cancer Screening Diagnostics
🔬 An example involving a rare cancer (prevalence $P(\text{Disease}) = 0.001$) and a 99% accurate test ($P(\text{Positive}|\text{Disease}) = 0.99$) was analyzed.
📉 Even with 99% accuracy, if a patient tests positive, the probability of actually having the disease ($P(\text{Disease}|\text{Positive})$) was calculated to be only about 9%.
⚠️ This illustrates that for rare diseases, a positive result is often a false positive (91% chance in this example), necessitating screening followed by secondary, more accurate testing rather than immediate treatment based on the initial test alone.

Key Points & Insights
➡️ Bayes' theorem is the mathematical tool for solving inverse problems—inferring causes from effects.
➡️ The prior belief ($P(B)$) is updated by the likelihood/update factor ($P(A|B)$) to yield the posterior probability ($P(B|A)$).
➡️ Medical screening for rare conditions can yield high rates of false positives if the low prevalence (the prior) is not heavily factored into the assessment.
➡️ In Bayesian inference, new data sequentially updates previous results, where the posterior becomes the next prior, facilitating continuous learning.

📸 Video summarized with SummaryTube.com on Mar 04, 2026, 11:31 UTC