Bayesian Posterior Probability Estimator
Update a prior probability with new evidence using Bayes' theorem and get the posterior probability with a worked calculation.
Reviewed by Daniel Agrici, Founder & Lead Developer
Formula
P(H|D) = P(D|H) x P(H) / [P(D|H) x P(H) + P(D|not H) x P(not H)]
Where P(H|D) is the posterior probability of the hypothesis given data, P(D|H) is the likelihood of data given hypothesis, P(H) is the prior probability, and the denominator is the total probability of the data (marginal likelihood).
Worked Examples
Example 1: Medical Diagnostic Test
Problem:A disease has a 1% prevalence (prior = 0.01). A test has 95% sensitivity (likelihood if true = 0.95) and 10% false positive rate (likelihood if false = 0.10). One positive test observed out of 1 test.
Solution:P(D|disease) = C(1,1) x 0.95^1 x 0.05^0 = 0.95\nP(D|no disease) = C(1,1) x 0.10^1 x 0.90^0 = 0.10\nP(D) = 0.95 x 0.01 + 0.10 x 0.99 = 0.0095 + 0.099 = 0.1085\nP(disease|positive) = 0.0095 / 0.1085 = 0.0876 = 8.76%\nBayes Factor = 0.95 / 0.10 = 9.5 (strong evidence)
Result:Posterior probability of disease given a positive test: 8.76%. Despite the positive result, there is still a 91.24% chance of no disease.
Example 2: A/B Test Conversion Rate
Problem:Prior belief that variant B is better: 50% (prior = 0.5). If B is better, likelihood of 7 out of 10 conversions = 0.85. If B is not better, likelihood = 0.35. Sample: 10 users, 7 converted.
Solution:P(D|B better) = C(10,7) x 0.85^7 x 0.15^3 = 120 x 0.3204 x 0.003375 = 0.1298\nP(D|B not better) = C(10,7) x 0.35^7 x 0.65^3 = 120 x 0.000628 x 0.2746 = 0.02069\nP(D) = 0.1298 x 0.5 + 0.02069 x 0.5 = 0.07524\nP(B better|D) = 0.0649 / 0.07524 = 86.3%
Result:Posterior probability that variant B is better: 86.3%. Bayes Factor: 6.27 (substantial evidence).
Frequently Asked Questions
What is a Bayesian posterior probability and how is it calculated?
A Bayesian posterior probability represents the updated belief about a hypothesis after observing new data. It is calculated using Bayes theorem, which combines the prior probability (your initial belief) with the likelihood of the observed data under the hypothesis. The formula is P(H|D) = P(D|H) x P(H) / P(D), where P(H|D) is the posterior, P(D|H) is the likelihood, P(H) is the prior, and P(D) is the marginal likelihood or evidence. This approach allows you to systematically update beliefs as new information becomes available, making it foundational in statistics, machine learning, and scientific reasoning.
How do I choose an appropriate prior probability for my analysis?
Choosing a prior depends on available background knowledge. Informative priors use existing research, expert opinion, or historical data to set an initial belief. For example, if previous studies show a treatment works 40 percent of the time, you might set the prior at 0.4. Uninformative or flat priors like 0.5 express ignorance and let the data dominate the posterior. Sensitivity analysis involves running the calculation with different priors to see how much the posterior changes. If the posterior is robust across reasonable priors, your conclusion is more reliable. In practice, the more data you have the less the prior matters.
What is the difference between Bayesian and frequentist statistics?
Bayesian statistics treats probability as a degree of belief that gets updated with evidence, while frequentist statistics treats probability as the long-run frequency of events. In frequentist inference, a parameter is fixed and unknown, and you use p-values and confidence intervals. In Bayesian inference, parameters have probability distributions that represent uncertainty. Bayesian methods incorporate prior knowledge, produce intuitive probability statements about hypotheses, and naturally handle small sample sizes. Frequentist methods do not require priors, are computationally simpler for many problems, and have well-established regulatory acceptance in fields like clinical trials.
Can Bayesian posterior estimation be used in machine learning applications?
Bayesian posterior estimation is widely used in machine learning for tasks like classification, regression, model selection, and hyperparameter tuning. Naive Bayes classifiers use posterior probabilities to assign categories to data points. Bayesian optimization uses posterior distributions over objective functions to efficiently search hyperparameter spaces. Bayesian neural networks place distributions over weights to capture model uncertainty, which is critical in safety-sensitive applications like autonomous driving and medical diagnosis. Gaussian processes are another Bayesian approach providing uncertainty-aware predictions. The main challenge is computational cost, often addressed through variational inference or Markov Chain Monte Carlo sampling methods.
References
Reviewed by Daniel Agrici, Founder & Lead Developer · Editorial policy