Seven responses in twenty patients
A clinical team in Hanoi is running a small early-phase trial of a new treatment for a skin condition. With the standard treatment, about 20% of patients respond. Twenty patients have now received the new one, and 7 of them responded: 35%. (The trial is invented for this lesson, but trials of exactly this shape are run every week.)
The team has to decide whether the treatment deserves a larger, more expensive trial, and the questions they ask are natural ones:
- How probable is it that the new treatment's response rate is better than 20%?
- What range of response rates is plausible given what we have seen?
- If we treat 40 more patients, how many are likely to respond?
The methods of the Foundations level answer neighbouring questions. A one-sided test of the null hypothesis that the rate is 0.20 gives a p-value of 0.087: if the true rate were 20%, seven or more responses in twenty would happen about 9% of the time. A 95% confidence interval runs from about 0.15 to 0.59. Both statements are correct and useful, and both describe what the data would do if we repeated the trial. Neither says how probable it is that the rate exceeds 20%: in the frequentist framework the rate is a fixed unknown constant with no probability distribution.
Bayesian inference gives the rate a probability distribution, and so answers all three questions directly. The price is that you must say what you believed about the rate before seeing the data: a prior. This lesson builds the machinery, computes everything for the trial, and asks honestly what the prior costs and what it buys.
Updating beliefs with data
You met Bayes' theorem for events in Lumenary Math's conditional probability: a positive result from a good test for a rare disease still leaves the disease fairly unlikely, because the prior probability (the prevalence) was small. Bayesian inference applies the same rule to hypotheses about a parameter.
Start with a coarse version. Suppose the team considered only three possibilities for the new treatment's response rate : no better than standard care (), moderately better (), or much better (). Before the trial, from laboratory work and experience with similar drugs, they gave these weights 0.5, 0.3 and 0.2. Each hypothesis predicts the observed result, 7 responses in 20, with a binomial probability: the likelihood. Bayes' theorem says multiply each prior weight by its likelihood, then rescale so the results add to 1.
| Hypothesis | Prior | Likelihood | Prior × likelihood | Posterior |
|---|---|---|---|---|
| 0.50 | 0.0545 | 0.0273 | 0.280 | |
| 0.30 | 0.1844 | 0.0553 | 0.568 | |
| 0.20 | 0.0739 | 0.0148 | 0.152 | |
| Total | 1.00 | 0.0974 | 1.000 |
The hypothesis that predicted the data best, , gained weight (from 0.30 to 0.57); the one that predicted it worst lost weight. Nothing is ever ruled out by one small trial: "no better than standard" still has probability 0.28. And the total in the fourth column, 0.0974, is the probability of the data averaged over all the hypotheses, which is the number we divide by.
Real response rates are not restricted to three values, so we replace the table with a curve: a prior density over all between 0 and 1. The rule is unchanged — posterior is proportional to prior times likelihood — but the sum in the denominator becomes an integral.
Analogy, not a formal construction
A navigator before satellites kept a running estimate of the ship's position from speed, heading and time. A sighting of a known lighthouse did not replace that estimate but was combined with it: a sharp sighting moved it a lot, a hazy one a little. The prior is the dead reckoning, the likelihood is the sighting, and the posterior is the new position, with a smaller uncertainty.
Prior, likelihood, posterior
Definition— Bayes' theorem for a parameter
Let be an unknown parameter and the observed data. A prior density describes uncertainty about before the data; the likelihood is the model's probability of the observed data for each value of ; the posterior density is The denominator , the evidence or marginal likelihood, does not depend on ; it only makes the posterior integrate to 1. So we often write : posterior is proportional to likelihood times prior.
The likelihood is the same function you maximised in Likelihood and model selection. Maximum likelihood keeps only its peak; Bayesian inference keeps its whole shape and weights it by the prior.
For a proportion the natural prior family is the beta distribution, , with density proportional to on , mean , and a spread that shrinks as grows. is the uniform distribution.
Theorem— Beta–binomial conjugacy
If the prior is and, given , the number of successes in independent trials is , then after observing the posterior is
Proof
Up to factors that do not involve , the prior is and the likelihood is . Their product is , which is the kernel of a density. A density is determined by its kernel, because the normalising constant is whatever makes the total integral 1, so the posterior is that beta distribution.
∎
A prior family that yields a posterior in the same family is conjugate to the likelihood. Updating then means adding counts, as if the prior had already shown us about successes and failures. The posterior mean makes this precise: a weighted average of the prior mean and the observed proportion, in which the prior counts as observations. As grows, shrinks and the data take over.
Definition— credible interval
A 95% credible interval is an interval that contains with posterior probability 0.95. The equal-tailed interval runs from the 2.5% to the 97.5% quantile of the posterior. The highest posterior density (HPD) interval is the shortest such interval; for a skewed posterior the two differ.
Definition— posterior predictive distribution
The posterior predictive distribution of new data averages the model's predictions over the posterior: . It includes both the randomness of new observations and the remaining uncertainty about .
The trial, worked by hand
Example— A uniform prior
Suppose the team is willing to say only that every response rate between 0 and 1 is equally plausible: prior . After 7 responses and 13 non-responses the posterior is .
- Posterior mean , slightly pulled from the observed 0.35 towards the prior mean 0.5, with prior weight .
- Posterior mode , the maximum likelihood estimate: with a flat prior the posterior is the likelihood, rescaled.
- Posterior standard deviation .
The quantiles and tail probabilities need the beta distribution function, which the computer supplies below: a 95% credible interval of , and .
Example— Learning in two instalments
Suppose the first 10 patients gave 2 responses and the next 10 gave 5. After the first batch, the uniform prior becomes . Using that as the prior for the second batch gives : exactly the posterior from analysing all 20 at once. Yesterday's posterior is today's prior, and the order in which independent observations arrive does not matter. This coherence is one of the attractions of the Bayesian approach for data that accumulate over time, as in monitoring a trial or a production line.
Computing the posterior
SciPy's stats.beta gives quantiles (ppf) and tail probabilities (sf, for "survival function") of any beta distribution. To make sure the conjugate formula is not doing anything mysterious, the program also computes the posterior the brute-force way, on a grid of 2,001 values of : multiply prior by likelihood at each point and divide by the total. Then, for comparison, it prints the frequentist answers to the neighbouring questions.
import numpy as np
from scipy import stats
# Synthetic single-arm trial: 7 of 20 patients respond. Standard care helps 20%.
k, n = 7, 20
a0, b0 = 1, 1 # uniform prior, Beta(1, 1)
post = stats.beta(a0 + k, b0 + n - k) # conjugate update: Beta(8, 14)
print("posterior: Beta(%d, %d)" % (a0 + k, b0 + n - k))
print("mean %.3f sd %.3f mode %.3f" % (post.mean(), post.std(), (a0 + k - 1) / (a0 + b0 + n - 2)))
lo, hi = post.ppf([0.025, 0.975])
print("95%% equal-tailed credible interval: [%.3f, %.3f]" % (lo, hi))
print("P(theta > 0.20 | data) = %.3f" % post.sf(0.20))
print("P(theta > 0.30 | data) = %.3f" % post.sf(0.30))
# The same posterior by brute force on a grid: prior x likelihood, then normalise.
grid = np.linspace(0, 1, 2001)
unnorm = stats.beta.pdf(grid, a0, b0) * stats.binom.pmf(k, n, grid)
weights = unnorm / unnorm.sum()
print("\ngrid check: mean %.3f P(theta > 0.20) = %.3f" % ((grid * weights).sum(), weights[grid > 0.20].sum()))
# Frequentist answers to neighbouring questions, for comparison.
print("\none-sided p-value, H0: theta = 0.20: P(X >= 7) = %.3f" % stats.binom.sf(k - 1, n, 0.20))
ci = stats.binomtest(k, n).proportion_ci(confidence_level=0.95, method="exact")
print("95%% Clopper-Pearson confidence interval: [%.3f, %.3f]" % (ci.low, ci.high))
ci = stats.binomtest(k, n).proportion_ci(confidence_level=0.95, method="wilson")
print("95%% Wilson confidence interval: [%.3f, %.3f]" % (ci.low, ci.high))Compare the two kinds of answer. The posterior probability that the new rate is no better than 20% is ; the one-sided p-value is 0.087. They are not two estimates of one quantity: the p-value is the probability, assuming , of data at least as extreme as ours; the posterior probability is the probability of given the data and a uniform prior. Yet the Wilson confidence interval, , almost equals the credible interval: for simple models and moderate data the two schools often agree numerically while differing in interpretation.
The third question, about the next 40 patients, is answered by the posterior predictive distribution. For a beta posterior and binomial data it is the beta-binomial distribution, which SciPy provides; we can also simulate it in two steps, drawing a rate from the posterior and then a count from that rate.
import numpy as np
from scipy import stats
# Posterior Beta(8, 14) from 7 responses in 20 patients and a uniform prior.
a, b = 8, 14
m = 40 # the next 40 patients
# Posterior predictive: average Binomial(40, theta) over the posterior (a beta-binomial).
pred = stats.betabinom(m, a, b)
plug = stats.binom(m, a / (a + b)) # plug-in: pretend theta is exactly 0.364
print(f"{'':20} {'mean':>5} {'sd':>5} 90% interval")
for name, d in [("posterior predictive", pred), ("plug-in binomial", plug)]:
lo, hi = d.ppf(0.05), d.ppf(0.95)
print(f"{name:20} {d.mean():5.1f} {d.std():5.2f} {int(lo)} to {int(hi)}")
# The same predictive by simulation: draw theta from the posterior, then a count.
rng = np.random.default_rng(1763)
theta = rng.beta(a, b, 100_000)
counts = rng.binomial(m, theta)
print("simulated predictive: mean %.1f sd %.2f 90%% interval %d to %d"
% (counts.mean(), counts.std(), np.quantile(counts, 0.05), np.quantile(counts, 0.95)))
print("P(at least 10 of the next 40 respond): predictive %.3f, plug-in %.3f"
% (pred.sf(9), plug.sf(9)))Both predictions are centred on 14.5 responses, but the honest one is much wider: 7 to 23, against 10 to 20 for the "plug-in" binomial that pretends the rate is exactly 0.364. The plug-in makes ten or more responses look 96% certain; allowing for what we do not know about , it is 84%. Ignoring parameter uncertainty is one of the commonest ways forecasts become overconfident.
Credible intervals and confidence intervals
The two kinds of interval make different promises.
- A 95% confidence interval comes from a procedure that, for every fixed , captures in (about or at least) 95% of repeated samples. Once computed, either contains or it does not; the 95% describes the method.
- A 95% credible interval contains with posterior probability 0.95, given the model and the prior — the statement people usually want. Its guarantee is an average: if really were drawn from the prior again and again, credible intervals would capture it 95% of the time averaged over those draws. For particular values of they may do worse.
A simulation makes both promises visible. We fix a true rate, simulate 20,000 trials of 20 patients, and count how often three kinds of 95% interval capture the truth: the flat-prior credible interval, the textbook Wald interval , and the Clopper–Pearson "exact" interval. In the last row, every simulated trial gets a fresh true rate drawn from the uniform prior.
import numpy as np
from scipy import stats
rng = np.random.default_rng(42)
n, reps = 20, 20_000
def intervals(k, n):
"""95% intervals for a proportion from k successes in n trials."""
cred = stats.beta.ppf([0.025, 0.975], 1 + k[:, None], 1 + n - k[:, None]) # uniform prior
p = k / n
half = 1.96 * np.sqrt(p * (1 - p) / n)
wald = np.stack([p - half, p + half], axis=1)
cp_lo = np.where(k == 0, 0.0, stats.beta.ppf(0.025, k, n - k + 1))
cp_hi = np.where(k == n, 1.0, stats.beta.ppf(0.975, k + 1, n - k))
return {"credible (flat prior)": cred, "Wald": wald, "Clopper-Pearson": np.stack([cp_lo, cp_hi], axis=1)}
def coverage(theta):
k = rng.binomial(n, theta)
return {name: np.mean((iv[:, 0] <= theta) & (theta <= iv[:, 1])) for name, iv in intervals(k, n).items()}
print("Coverage of nominal 95% intervals, n = 20, 20,000 simulated trials each")
print(f"{'true theta':>12} {'credible':>9} {'Wald':>7} {'Clopper-P':>10}")
for th in [0.05, 0.20, 0.35, 0.50]:
c = coverage(np.full(reps, th))
print(f"{th:12.2f} {c['credible (flat prior)']:9.3f} {c['Wald']:7.3f} {c['Clopper-Pearson']:10.3f}")
# Bayesian calibration: draw a new theta from the prior for every simulated trial.
c = coverage(rng.uniform(0, 1, reps))
print(f"{'theta ~ prior':>12} {c['credible (flat prior)']:9.3f} {c['Wald']:7.3f} {c['Clopper-Pearson']:10.3f}")The Wald interval, still widely taught, fails for small samples and extreme rates: at it captures the truth only 64% of the time, because whenever its width collapses to zero. The Clopper–Pearson interval keeps its promise at every by being conservative (96–98%). The credible interval covers close to 95% for middle values, dips to 92% at , and covers 94.8% when really is drawn from the prior, as the Bayesian guarantee says it must (the shortfall is within simulation error). Each school keeps its own promise, and a careful statistician of either persuasion also checks a method against the other's criterion — an attitude Roderick Little calls calibrated Bayes.
How much does the prior matter?
The uniform prior is not the only defensible choice, and a Bayesian analysis should show how much its conclusions depend on the choice. We compare four priors:
- uniform, ;
- Jeffreys, , derived from the model's Fisher information by a rule that gives the same answer whatever scale the parameter is measured on;
- sceptical, , centred on the standard treatment's 0.20 and worth about 20 patients, a stance a regulator might ask for;
- optimistic, , centred on 0.50 and worth about 10 patients, a stance an enthusiastic developer might take.
We analyse the actual trial (7 of 20) and a hypothetical larger trial with the same response rate (70 of 200).
from scipy import stats
priors = {
"uniform Beta(1, 1)": (1.0, 1.0),
"Jeffreys Beta(.5, .5)": (0.5, 0.5),
"sceptical Beta(4, 16)": (4.0, 16.0), # centred on 0.20, worth about 20 patients
"optimistic Beta(5, 5)": (5.0, 5.0), # centred on 0.50, worth about 10 patients
}
for k, n in [(7, 20), (70, 200)]:
print(f"data: {k} responses in {n} patients (observed rate {k / n:.2f})")
print(f" {'prior':26} {'post. mean':>10} {'95% credible interval':>22} {'P(theta <= 0.2)':>16}")
for name, (a, b) in priors.items():
post = stats.beta(a + k, b + n - k)
lo, hi = post.ppf([0.025, 0.975])
print(f" {name:26} {post.mean():10.3f} [{lo:.3f}, {hi:.3f}] {post.cdf(0.2):16.2g}")
print()
# What each prior believed before any data: its probability that theta <= 0.2.
for name, (a, b) in priors.items():
print(f"prior {name:26} P(theta <= 0.2) = {stats.beta(a, b).cdf(0.2):.3f}")With 20 patients the prior matters: the posterior probability that the treatment is no better than standard ranges from 0.007 (optimistic prior) to 0.14 (sceptical), and the posterior mean from 0.275 to 0.400. With 200 patients the four analyses agree closely: means from 0.336 to 0.357, and all four put a probability below on "no better than standard". For well-behaved models with a fixed number of parameters this is general: as data accumulate, the posterior concentrates around the truth whatever the prior (unless the prior rules the truth out) and becomes approximately normal, centred near the maximum likelihood estimate with the frequentist standard error as its spread. This is the Bernstein–von Mises theorem, and it is why Bayesian and frequentist intervals nearly coincide in large, regular problems.
The last lines of the output deserve a moment. Before any data, the uniform prior already gave probability 0.80 to "better than 20%", simply because 80% of the interval from 0 to 1 lies above 0.20. Whether that is "uninformative" depends on what you thought beforehand.
In practice, informative priors encode real evidence (earlier trials, physical constraints, carefully recorded expert elicitation); weakly informative priors rule out absurd values while letting the data dominate within the plausible range; reference priors such as Jeffreys's aim for minimal influence by a stated criterion. A prior predictive check — simulating data from the prior before seeing the real data — shows whether a prior implies anything silly.
Two views of probability
The disagreement between Bayesian and frequentist statistics is not about arithmetic — both accept Bayes' theorem as a theorem — but about what probability is allowed to describe.
| Frequentist view | Bayesian view | |
|---|---|---|
| Probability is | a long-run frequency in repeatable experiments | a degree of uncertainty, given stated information |
| The parameter is | a fixed unknown constant | uncertain, so it has a distribution |
| What is random | the data (and so the estimate and interval) | our knowledge of ; the data are conditioned on |
| A 95% interval means | the procedure captures in 95% of repetitions | lies in it with probability 0.95, given model and prior |
| Main tools | estimators, confidence intervals, tests, error rates | prior, posterior, predictive distributions, decision analysis |
What the frequentist approach does well. Its guarantees hold for every value of the parameter without anyone having to agree on a prior, which matters when results must convince sceptical outsiders: regulators, courts, rival laboratories. Error rates of procedures (how often a design approves an ineffective drug) are exactly what a regulator must control, and pre-specified frequentist designs remain the default for confirmatory trials.
What the Bayesian approach does well. It answers direct questions about parameters and predictions, combines evidence from several sources coherently, updates naturally as data accumulate, handles models with many related parameters (the multilevel models of the Advanced level), and feeds straight into decisions that weigh costs and benefits. Its conclusions do not depend on what the experimenter would have done had the data been different, such as when they would have stopped; p-values do.
The costs. Bayesian results depend on the prior, which must be defended, and on computation that can be heavy and can fail silently. Frequentist results depend on choices too — the model, the test statistic, the stopping rule, the significance level — that are less visible but no less real. And both depend entirely on the model of how the data arose: a posterior or a p-value from a wrong model is precise nonsense.
In 2026 most working statisticians are pragmatic: they use Bayesian models where structure and prior information help, judge them by frequentist properties such as coverage, and use frequentist designs where error-rate guarantees are required. Regulators reflect the mix. The US Food and Drug Administration has had guidance on Bayesian statistics for medical-device trials since 2010, and its January 2026 draft guidance for drugs and biological products asks that the prior be pre-specified and justified and that a design's operating characteristics — how often it would support correct conclusions — be evaluated, usually by simulation: a frequentist check on a Bayesian method.
A short history
Thomas Bayes, a Presbyterian minister in Tunbridge Wells, solved a version of the beta–binomial problem: given that an event has happened times in trials, what is the chance that its probability lies between two given values? After his death in 1761 his friend Richard Price, also a minister and a pioneer of actuarial science, edited the essay and presented it to the Royal Society in 1763. Pierre-Simon Laplace, independently and far more systematically, developed "inverse probability" from 1774, applying it from astronomy to birth records; his "rule of succession", , is the posterior mean under a uniform prior.
In the early twentieth century R. A. Fisher attacked inverse probability for its arbitrary priors and built likelihood-based inference instead, while Jerzy Neyman and Egon Pearson built tests and confidence intervals around long-run error rates. Bayesian ideas survived through Harold Jeffreys's Theory of Probability (1939), the work of Bruno de Finetti and Leonard Savage on subjective probability, and practical successes such as the "weight of evidence" calculations Alan Turing and I. J. Good used against German naval codes at Bletchley Park. But the integral in the denominator of Bayes' theorem limited what could be computed until, around 1990, statisticians took up Markov chain Monte Carlo, invented by physicists in the 1950s. That is the next lesson.
Bayesian analysis in practice
Software. For conjugate models, a few lines of SciPy, as above, are all you need. For realistic models, analysts write the model in a probabilistic programming language and let the software compute the posterior: Stan (used from R, Python and other languages), PyMC and NumPyro in Python, Turing in Julia, and the R package brms for regression models written in R's formula syntax. The next lesson shows how these systems work. The trial analysis in R, for comparison (not run here):
# R, not run here: the uniform-prior analysis of 7 responses in 20
a <- 1 + 7; b <- 1 + 13
qbeta(c(0.025, 0.975), a, b) # 95% credible interval: 0.181 0.570
1 - pbeta(0.20, a, b) # P(theta > 0.20 | data): 0.957Workflow. Experienced Bayesian analysts follow a cycle that Andrew Gelman and colleagues call the Bayesian workflow: write a model, simulate data from the prior to check that it makes sense, fit the model, check the computation, compare the model's predictions with the real data (posterior predictive checks), and revise. Model checking is not optional for Bayesians any more than for anyone else.
Reporting. Report the prior and why it was chosen, the posterior summaries that answer the question, a sensitivity analysis over reasonable priors, and diagnostics of the computation. Bayesian methods are now routine in adaptive clinical trials, epidemic modelling, ecology, astronomy, genetics, election forecasting, small-area estimation for official statistics, and product experiments.
Common misconceptions
Misconception
"A 95% confidence interval has a 95% probability of containing the parameter." Not in the frequentist framework that produced it: there, the parameter is fixed and the 95% describes the procedure over repeated samples. The statement is true of a 95% credible interval, given its model and prior. That the two intervals are often numerically similar is why the confusion survives.
Misconception
"The p-value is the probability that the null hypothesis is true." It is the probability of data at least as extreme as those observed, assuming the null hypothesis. In the trial, the p-value was 0.087 while the posterior probability of "no better than standard" ranged from 0.007 to 0.14 depending on the prior. A probability for a hypothesis needs a prior; the p-value does not use one.
Misconception
"With enough data, the prior never matters." It fades for well-behaved models with a fixed number of parameters. It keeps mattering for rare events, for parameters the data say little about (such as the variance between groups when there are only a few groups), for models whose number of parameters grows with the data, and for comparing models by their evidence , which stays sensitive to the prior however much data you have.
Misconception
"The posterior tells us the truth about ." It tells us what follows from the prior, the data and the model. If the patients in the trial were not independent (several were treated by one clinician with an unusual technique), or the response rate differs between hospitals, the binomial model is wrong and the posterior inherits the error.
Exercises
1. Bayesian inference
Warm-upHumanIn the trial (7 responses in 20 patients), a uniform prior gave the 95% credible interval for the response rate , and the Clopper–Pearson method gave the 95% confidence interval . Which statement is correct?
2. Bayesian inference
Warm-upHumanA quality engineer in Porto models the defect rate of a new moulding machine with the prior , based on similar machines. In the first 30 parts, 6 are defective. What is the posterior mean of , as a percentage?
3. Bayesian inference
CoreHumanWith a prior and binomial observations, the posterior mean is a weighted average of the prior mean and the observed proportion, with weight on the observed proportion. A team uses the prior . How many observations are needed before the observed proportion gets weight 0.75?
4. Bayesian inference
CoreHumanIn which of these analyses does the prior have the largest influence on the posterior mean of a proportion?
5. Bayesian inference
CoreHumanWhich of these statements would a careful statistician of either school accept? Select all that apply.
6. Bayesian inference
CoreComputationalWrite posterior_summary(a, b, k, n) for a prior and successes in binomial trials. It returns a dictionary with the posterior parameters "a" and "b", the posterior "mean", and "interval", the 95% equal-tailed credible interval as a pair (low, high). Use scipy.stats.beta.
7. Bayesian inference
StretchComputationalA library in Nairobi tries two versions of a text-message reminder. Version A: of members return their books on time; version B: of . With independent uniform priors on the two return rates, the posteriors are independent beta distributions.
Write prob_b_better(k_a, n_a, k_b, n_b, draws=200_000, seed=0) that estimates by Monte Carlo: draw draws values from each posterior with np.random.default_rng(seed) and return the fraction of pairs in which is larger. The tests compare your answer with an exact integral to within 0.01.
Further reading and resources
- Bayesian Data Analysis (third edition) · Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari and Donald RubinThe standard graduate text; the authors offer a PDF for non-commercial use, with course materials and code.
- Statistical Rethinking · Richard McElreathA gentle, code-first introduction to Bayesian modelling, with free recorded lectures.
- An Essay towards Solving a Problem in the Doctrine of Chances · Thomas Bayes, communicated by Richard Price · 1763The paper that gave Bayes' theorem its name, published after Bayes's death.
- Bayesians, Frequentists, and Scientists · Bradley Efron · 2005A short, even-handed essay on where each school is strong.
- Calibrated Bayes: A Bayes/Frequentist Roadmap · Roderick Little · 2006The case for Bayesian models judged by frequentist calibration.
- Bayesian Workflow · Andrew Gelman and others · 2020How applied Bayesian analysis is done in practice, from prior checks to model comparison.
- Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products (draft guidance) · US Food and Drug Administration · 2026A regulator's view of when and how Bayesian evidence can support approval.