Did the programme work?
A city education office runs free after-school maths tutoring. At the end of the year an analyst compares the pupils who attended with those who did not, and finds that the tutored pupils scored about five marks lower. Should the city close the programme?
Almost certainly not: the pupils who signed up were the ones who were struggling, and would have scored lower with or without tutoring. The comparison mixes what tutoring did with who chose it, and the raw numbers cannot separate the two.
The questions people care about most are causal. Does this drug reduce strokes? Does a higher minimum wage cost jobs? Do smaller classes help children learn? Each asks about a world we did not observe: what would have happened to the same people under a different choice. Regression (Multiple regression and diagnostics) describes how outcomes vary with variables in the world we did observe; causal inference asks when, and under what assumptions, such descriptions answer a "what if" question.
This lesson builds the subject in three layers: a precise language for causal effects (potential outcomes); the design that solves the problem cleanly (randomisation); and the tools for when we cannot randomise: causal diagrams to decide what to adjust for, propensity scores to do the adjusting, and three natural experiments that exploit accidents of nature and policy, whose champions shared the 2021 Nobel prize in economics.
Potential outcomes
Give each pupil two numbers: , the mark they would get with tutoring, and , the mark they would get without. These are potential outcomes. Both exist, in the sense that each describes a definite possible world, but at most one is ever seen.
Definition— potential outcomes and causal effects
For a binary treatment , unit has potential outcomes and , and the observed outcome is . The individual causal effect is . The average treatment effect is over the population, and the average treatment effect on the treated is .
The notation rests on the stable unit treatment value assumption (SUTVA): each unit's outcome depends only on its own treatment (no interference), and there is only one version of the treatment. Vaccines, which protect the unvaccinated by slowing transmission, violate the first half; "exercise", which can mean anything from walking to marathons, strains the second.
Here are six pupils, seen with a God's-eye view of both potential outcomes. The three weakest chose tutoring.
| Pupil | Effect | Tutored? | Observed | ||
|---|---|---|---|---|---|
| Amina | 48 | 57 | +9 | yes | 57 |
| Bo | 52 | 59 | +7 | yes | 59 |
| Carlos | 61 | 66 | +5 | yes | 66 |
| Dilnoza | 70 | 73 | +3 | no | 70 |
| Eero | 75 | 77 | +2 | no | 75 |
| Farida | 80 | 80 | 0 | no | 80 |
Tutoring helped every pupil or left them unchanged: the ATE is marks and the ATT is . Yet the observed comparison is . The algebra of the gap is worth writing out once:
The selection-bias term compares the two groups in the world without tutoring. It is zero only if the groups would have done equally well untreated, and that is exactly what self-selection destroys. Paul Holland called the fact that we never see both columns for the same unit the fundamental problem of causal inference. Every method in this lesson is a way of filling in the missing column for a group, never for an individual, using an assumption we can state.
Randomisation, the gold standard
Suppose a fair coin decides who is offered tutoring. Then treatment is independent of everything about the pupil, including both potential outcomes:
Independence makes the selection-bias term vanish: , and likewise for . So the simple difference in group means is an unbiased estimate of the ATE. This is why Ronald Fisher's randomised experiments, which you met in Experimental design and ANOVA, are called the gold standard: they balance not only the variables we measured but every variable, known or unknown, in expectation.
The simulation below creates 2,000 pupils with both potential outcomes, so that we know the truth, and then plays out two worlds: one where pupils choose, and one where a coin decides.
import numpy as np
rng = np.random.default_rng(2021)
n = 2000
prior = rng.normal(60, 12, n) # last year's maths mark
y0 = 12 + 0.8 * prior + rng.normal(0, 6, n) # this year's mark WITHOUT tutoring
y1 = y0 + 5 + 0.15 * (60 - prior) # WITH tutoring: helps weaker pupils more
print(f"true average effect (ATE): {np.mean(y1 - y0):5.2f} marks")
# World A: pupils choose. Weaker pupils are far more likely to sign up.
signs_up = rng.random(n) < 1 / (1 + np.exp((prior - 52) / 5))
y_obs = np.where(signs_up, y1, y0) # we only ever see one potential outcome
naive = y_obs[signs_up].mean() - y_obs[~signs_up].mean()
att = np.mean((y1 - y0)[signs_up])
print(f"self-selection: {signs_up.mean():.0%} sign up; tutored minus untutored = {naive:6.2f}"
f" (effect on those who signed up: {att:.2f})")
# World B: the same pupils, but a fair coin decides who is offered tutoring.
coin = rng.random(n) < 0.5
y_obs = np.where(coin, y1, y0)
se = np.sqrt(y_obs[coin].var(ddof=1) / coin.sum() + y_obs[~coin].var(ddof=1) / (~coin).sum())
print(f"randomised: tutored minus untutored = {y_obs[coin].mean() - y_obs[~coin].mean():6.2f}"
f" (standard error from this one experiment: {se:.2f})")
# Repeat the coin-flipping 10,000 times: the estimate is right on average.
est = np.empty(10_000)
for r in range(est.size):
c = rng.random(n) < 0.5
est[r] = y1[c].mean() - y0[~c].mean()
print(f"10,000 randomisations: mean estimate {est.mean():.3f}, standard deviation {est.std():.3f}")The true ATE is 4.94 marks. Under self-selection, the tutored pupils score 5.40 marks lower than the rest, even though tutoring raised every pupil's mark, and raised the marks of those who chose it by 6.59 on average (the ATT is larger than the ATE here because the programme helps weaker pupils more, and they are the ones who chose it). The single randomised experiment gives 5.18 with a standard error of 0.47, comfortably consistent with the truth.
But we cannot randomly assign smoking, schooling or a minimum wage, and sometimes we should not randomise even when we could. The rest of this lesson is about observational data, where people or circumstances chose the treatment, and the assumptions that substitute for the coin.
Confounding and causal diagrams
Take a second example, with real stakes for the method. A farming cooperative offers maize growers a text-message advisory service with planting and weather advice. Did subscribing raise yields? Farmers with more schooling and bigger plots were more likely to subscribe, and schooling and plot size also raise yields directly. Variables like these, which influence both the treatment and the outcome, are confounders. A causal diagram, or directed acyclic graph (DAG), draws each variable as a node and each direct causal influence as an arrow.
Association flows along paths in the diagram, whichever way the arrows point. Three small patterns decide whether a path carries association:
- a fork, : a common cause makes and associated; conditioning on (holding it fixed, stratifying, or including it in a regression) blocks the path;
- a chain, : the effect flows through the mediator; conditioning on blocks it, and removes that part of the causal effect;
- a collider, : two arrows meet head to head; the path is blocked unless we condition on (or on something causes), which opens it.
Colliders are the surprise. Suppose farmers were more likely to answer the follow-up survey if they subscribed (they feel engaged) or if their harvest was good (they are happy to talk). Among those who answered, a non-subscriber probably answered because the harvest was good, so non-subscribing respondents have unusually high yields, and the service looks worse than it is. Selecting on a common effect manufactured an association. The same mechanism, Berkson's paradox (after a 1946 study of hospital patients), makes traits that each help someone get selected, for a job, a university place or publication, look like rivals within the selected group.
Definition— backdoor criterion
A set of variables satisfies the backdoor criterion for the effect of on if no variable in is a descendant of , and blocks every path between and that starts with an arrow into (a "back-door" path). Then the effect is identified by the adjustment formula with an integral for continuous : compare treated and untreated within levels of , then average over the population's distribution of .
In the figure, satisfies the criterion. Adding fertiliser bought would violate it (a descendant of ), and so would restricting to survey respondents. The criterion, due to Judea Pearl, turns the old advice "control for confounders" into a rule that also says what not to control for.
Adjusting for confounders: propensity scores
If the backdoor criterion holds for a set of measured covariates , we say treatment is unconfounded given : . Within any group of units with the same , treatment is as good as random. With one or two discrete covariates we could simply stratify. With many covariates, strata become empty, and we need a summary.
Definition— propensity score
The propensity score is , the probability of treatment given the covariates. Overlap (or positivity) requires for every in the population: every kind of unit could have received either treatment.
Theorem— Rosenbaum and Rubin, 1983
If treatment is unconfounded given , it is also unconfounded given the single number : .
Proof
It suffices to show that does not depend on the potential outcomes. By the tower property, it equals . Unconfoundedness given makes the inner probability equal to , and the expectation of given (and anything else) is itself. So the probability of treatment given the potential outcomes and the score is just the score.
∎
So we can balance many covariates by balancing one number. Two standard ways to use it:
- Inverse probability weighting (IPW) weights each treated unit by and each untreated unit by . A treated farmer who was unlikely to subscribe stands in for many similar farmers who did not. The weighted groups each resemble the whole population, and the difference of weighted means estimates the ATE.
- Matching pairs each treated unit with the untreated unit whose score is closest, and averages the differences within pairs. This estimates the ATT.
The score itself is unknown and must be estimated, usually by logistic regression (Generalised linear models). In the simulation below, the true effect of the service is 0.30 tonnes per hectare for every farmer, and we fit the logistic regression with a few steps of Newton's method so that nothing is hidden in a library.
import numpy as np
rng = np.random.default_rng(7)
n = 3000
school = rng.integers(0, 13, n).astype(float) # years of schooling
area = rng.gamma(2.0, 0.6, n) # plot area in hectares
# Better-educated farmers with bigger plots are more likely to subscribe.
logit = -3.0 + 0.25 * school + 0.8 * area
t = rng.random(n) < 1 / (1 + np.exp(-logit))
# Yield (tonnes per hectare): the service adds 0.30 t/ha for everyone.
y = 1.0 + 0.08 * school + 0.25 * area + 0.30 * t + rng.normal(0, 0.4, n)
X = np.column_stack([np.ones(n), school, area])
print(f"{t.mean():.0%} subscribe. Naive difference in mean yield: {y[t].mean() - y[~t].mean():.3f} t/ha (truth 0.300)")
# 1. Propensity scores by logistic regression, fitted with Newton's method.
beta = np.zeros(3)
for _ in range(25):
p = 1 / (1 + np.exp(-X @ beta))
grad = X.T @ (t - p)
hess = X.T @ (X * (p * (1 - p))[:, None])
beta += np.linalg.solve(hess, grad)
e = 1 / (1 + np.exp(-X @ beta))
print("fitted propensity model:", np.round(beta, 2), " range of scores:", np.round([e.min(), e.max()], 3))
# 2. Inverse probability weighting (normalised weights).
w1, w0 = t / e, (~t) / (1 - e)
ipw = np.sum(w1 * y) / w1.sum() - np.sum(w0 * y) / w0.sum()
# 3. One-to-one nearest-neighbour matching on the logit of the score, with replacement (ATT).
lt, lc = np.log(e[t] / (1 - e[t])), np.log(e[~t] / (1 - e[~t]))
match = np.abs(lt[:, None] - lc[None, :]).argmin(axis=1)
att = np.mean(y[t] - y[~t][match])
# 4. Regression adjustment, for comparison.
coef = np.linalg.lstsq(np.column_stack([X, t]), y, rcond=None)[0]
print(f"IPW: {ipw:.3f} matching (ATT): {att:.3f} regression: {coef[3]:.3f}")
# Balance check: standardised mean differences before and after matching.
def smd(a, b):
return (a.mean() - b.mean()) / np.sqrt((a.var() + b.var()) / 2)
for name, v in [("schooling", school), ("plot area", area)]:
print(f" {name:9} SMD before {smd(v[t], v[~t]):+.2f} after matching {smd(v[t], v[~t][match]):+.2f}")The naive comparison says 0.625 t/ha, more than double the truth, because subscribers were better placed to begin with. Weighting gives 0.284, matching 0.280 and regression adjustment 0.284, all close to 0.30. The balance table is the step practitioners report first: before matching, subscribers had 0.77 standard deviations more schooling than non-subscribers; after matching, the difference is 0.03. A common rule of thumb treats standardised differences below 0.1 as adequate balance.
One caution about overlap: the estimated scores run from 0.057 to 0.989. A non-subscriber with gets weight , and a handful of such weights can dominate an IPW estimate. Practitioners inspect the scores in both groups, trim extreme ones, and say which population the trimmed estimate describes.
Finally, the three estimators agree with each other and with the truth only because the program wrote the truth. On real data, agreement among adjustment methods shows that the answer does not hinge on the functional form; it says nothing about unmeasured confounding, which biases all of them alike.
Natural experiments
When important confounders are unmeasured, adjustment fails. The alternative is to find a source of variation in the treatment that is as good as random for reasons outside the data: a lottery, a rule, a border, a date. These are natural experiments, and each comes with a design that exploits it.
Instrumental variables
An instrument is a variable that (1) shifts the treatment (relevance), (2) is as good as randomly assigned (independence), and (3) affects the outcome only through the treatment (exclusion restriction). A randomised encouragement is the cleanest case. Suppose a health service sends a reminder letter, chosen at random, encouraging adults to get a flu vaccine. Health-conscious people get vaccinated anyway and also take fewer sick days, so comparing vaccinated with unvaccinated people is confounded. But the letter is random. The Wald estimator divides the letter's effect on the outcome by its effect on the treatment:
Guido Imbens and Joshua Angrist showed in 1994 what this ratio estimates when effects differ between people. Divide the population into always-takers (vaccinated regardless), never-takers (never vaccinated) and compliers (vaccinated only if they get the letter), and assume there are no defiers who do the opposite of the encouragement (monotonicity). Then the ratio estimates the local average treatment effect (LATE): the average effect among compliers.
import numpy as np
rng = np.random.default_rng(11)
# --- Instrumental variable: a randomly sent reminder letter encourages flu vaccination ---
n = 20_000
kind = rng.choice(["always", "complier", "never"], size=n, p=[0.30, 0.25, 0.45])
z = rng.random(n) < 0.5 # letter sent? (random)
d = (kind == "always") | ((kind == "complier") & z) # vaccinated?
base = np.select([kind == "always", kind == "complier"], [2.0, 3.5], 4.5) # days off sick if unvaccinated
gain = np.select([kind == "always", kind == "complier"], [1.0, 1.5], 2.0) # days saved by the vaccine
y = base - gain * d + rng.normal(0, 1.0, n) # days off work this winter
print(f"vaccinated minus unvaccinated: {y[d].mean() - y[~d].mean():+.2f} days (confounded by health habits)")
itt = y[z].mean() - y[~z].mean() # effect of the LETTER
first_stage = d[z].mean() - d[~z].mean() # how much the letter moves vaccination
print(f"letter: effect on days {itt:+.3f}, effect on vaccination {first_stage:.3f}")
print(f"Wald (IV) estimate: {itt / first_stage:+.2f} days "
f"true effect for compliers {-gain[kind == 'complier'].mean():+.2f}, for everyone {-gain.mean():+.2f}")
# --- Difference-in-differences: free school meals in region A from this year ---
schools = 150
a_before = rng.normal(88.0, 3.0, schools) # attendance %, region A, last year
b_before = rng.normal(92.0, 3.0, schools) # region B, last year
a_after = a_before + 1.0 + 2.5 + rng.normal(0, 1.0, schools) # common trend +1.0, policy +2.5
b_after = b_before + 1.0 + rng.normal(0, 1.0, schools)
print(f"A after minus B after: {a_after.mean() - b_after.mean():+.2f} points")
print(f"A after minus A before: {a_after.mean() - a_before.mean():+.2f} points")
did = (a_after.mean() - a_before.mean()) - (b_after.mean() - b_before.mean())
se = np.sqrt(np.var(a_after - a_before, ddof=1) / schools + np.var(b_after - b_before, ddof=1) / schools)
print(f"difference-in-differences: {did:+.2f} points, standard error {se:.2f} (true effect +2.50)")Comparing vaccinated with unvaccinated people suggests the vaccine saves 2.99 days, double its real effect, because the healthiest people chose it. The letter reduced sick days by only 0.364 on average, but it moved only 24.3% of people into vaccination; dividing gives 1.50 days, exactly the effect for compliers. It is not the average effect for everyone (1.57 days here), because never-takers, who would gain most, are never moved by a letter. An instrument answers a question about the people it moves, which may or may not be the question a policy-maker needs.
The exclusion restriction is the assumption to attack: if the letter also reminded people to wash their hands, the ratio would credit the vaccine with the letter's other effects. And a weak instrument, which barely moves the treatment, divides by a number near zero and amplifies every small bias, which is why studies report the strength of the first stage.
Difference-in-differences
Suppose region A starts free school meals this year and region B does not. Comparing A with B this year mixes the policy with the regions' permanent differences; comparing A this year with A last year mixes the policy with everything else that changed. Difference-in-differences (DiD) subtracts B's change from A's change:
In the simulation, region A has lower attendance throughout, and both regions share an upward trend of one point. The after-only comparison says the policy lowered attendance by 1.34 points; the before–after comparison credits it with the whole rise of 3.53 points, trend included. DiD gives 2.65 with a standard error of 0.11, close to the true 2.50.
DiD rests on parallel trends: without the policy, A would have changed by the same amount as B. It cannot be tested for the period that matters, but it can be probed: if several years of data before the policy show the two regions moving in step, the assumption is more believable. The classic application is David Card and Alan Krueger's 1994 comparison of fast-food restaurants in New Jersey, which raised its minimum wage, and neighbouring eastern Pennsylvania, which did not. They found no evidence of the job losses the simplest economic model predicts, and started a long and productive debate.
Regression discontinuity
Many rules treat people differently on either side of a threshold: a scholarship for an exam score of at least 70, a benefit for households below an income line, a class split when enrolment passes 40. People just above and just below the cut-off are nearly identical in everything except the treatment, so the jump in the outcome at the cut-off estimates the effect there. This is a regression discontinuity design (RDD), introduced by the psychologists Donald Thistlethwaite and Donald Campbell in 1960.
import numpy as np
rng = np.random.default_rng(5)
n = 4000
score = rng.uniform(40, 100, n) # entrance-exam score
award = score >= 70 # the scholarship rule
# The first-year mark rises smoothly, and not in a straight line, with the exam score.
# The scholarship adds 3 marks on top.
mark = 25 + 60 / (1 + np.exp(-(score - 62) / 9)) + 3.0 * award + rng.normal(0, 5, n)
print(f"award holders minus the rest: {mark[award].mean() - mark[~award].mean():.2f} marks (truth 3.00)")
def side_fit(keep):
"""Least-squares line through (score - 70, mark); return its height at 70 and that height's variance."""
X = np.column_stack([np.ones(keep.sum()), score[keep] - 70])
coef, *_ = np.linalg.lstsq(X, mark[keep], rcond=None)
resid = mark[keep] - X @ coef
s2 = resid @ resid / (keep.sum() - 2)
return coef[0], s2 * np.linalg.inv(X.T @ X)[0, 0]
print(" bandwidth pupils used estimated jump standard error")
for h in (2.5, 5, 10, 20, 30):
near = np.abs(score - 70) <= h
(right, v1), (left, v0) = side_fit(award & near), side_fit(~award & near)
print(f" {h:5.1f} {near.sum():5d} {right - left:5.2f} {np.sqrt(v1 + v0):.2f}")Comparing all award holders with everyone else gives 36.32 marks, a measure of how much better strong students do, not of the scholarship. The local fits tell the real story, and it is a story about the bias–variance trade-off. A narrow window (2.5 points either side) uses only 385 students: the estimate, 2.15, is nearly unbiased but its standard error is 1.01. A window of 10 points gives 3.50 with standard error 0.54. Wide windows look precise and are wrong: with all the data, the straight lines cannot follow the curve, and the estimate of 8.48 comes with a standard error of 0.33, confidently far from the truth of 3.
Two checks make an RDD credible. If people can manipulate the running variable (re-sitting until they pass, under-reporting income), a density test finds a suspicious pile-up just past the threshold. And covariates fixed before treatment, such as age, should show no jump at the cut-off. Modern practice chooses the bandwidth by a data-driven rule and reports bias-corrected confidence intervals.
From Fisher's fields to the 2021 Nobel prize
The ideas in this lesson come from disciplines that took decades to recognise each other. Jerzy Neyman wrote down potential outcomes for agricultural plots in 1923; Ronald Fisher made randomisation the foundation of experimental design at Rothamsted; the geneticist Sewall Wright drew the first path diagrams around 1920. Epidemiology learned to argue causally from observational data in the controversy over smoking and lung cancer, with Richard Doll and Austin Bradford Hill's studies of British doctors and Hill's 1965 considerations (strength, consistency, dose–response and others) for judging whether an association is causal.
Donald Rubin turned potential outcomes into a general framework for observational studies from 1974, and with Paul Rosenbaum introduced the propensity score in 1983. Judea Pearl, a computer scientist, developed causal diagrams and the do-calculus in the 1990s. The two frameworks largely agree on what can be identified, and modern practice uses both: diagrams to decide what to adjust for, potential outcomes to define and estimate effects.
In economics, the credibility revolution of the 1990s built research around natural experiments. Joshua Angrist and Alan Krueger used children's quarter of birth, which through school-entry and leaving-age rules shifts how long they stay in school, as an instrument for the effect of schooling on earnings; Card and Krueger's minimum-wage study used a state border. In 2021 the Sveriges Riksbank Prize in Economic Sciences went half to David Card "for his empirical contributions to labour economics" and half jointly to Joshua Angrist and Guido Imbens "for their methodological contributions to the analysis of causal relationships". Alan Krueger, Card's co-author on much of that work, had died in 2019; the prize is not awarded posthumously.
Causal inference in professional practice
Design before analysis. Write down the randomised trial you wish you could run (who is eligible, which treatment strategies, when follow-up starts, which outcome) and make the observational analysis imitate it. Miguel Hernán and James Robins call this target trial emulation; it prevents errors such as "immortal time bias", where the treated are, by construction, those who survived long enough to be treated. It is now standard with health records and registries, such as the Nordic national registries. Such data are about real people: access is governed by law and ethics review, and analyses run on de-identified data in secure environments.
Sensitivity analysis. Because unconfoundedness cannot be tested, careful studies ask how strong an unmeasured confounder would have to be to explain away the result. Tyler VanderWeele and Peng Ding's E-value expresses this as a single number on the risk-ratio scale: an observed risk ratio of 2 could be fully explained only by a confounder associated with both treatment and outcome by risk ratios of at least .
Modern difference-in-differences. When units adopt a policy at different times and effects change over time, the traditional "two-way fixed effects" regression can weight comparisons in ways that make no sense, even negatively. Since about 2020, papers by Goodman-Bacon, Callaway and Sant'Anna, de Chaisemartin and D'Haultfœuille and others have given estimators that compare each cohort only with valid controls. Synthetic control, which weights untreated regions to match a single treated one before the policy, is a close relative.
Software. The methods here are a few dozen lines of NumPy, but practitioners rely on maintained packages: in R, MatchIt and packages for instruments, DiD and RDD; in Python, DoWhy, which asks you to state the causal graph and then runs refutation checks; DAGitty for diagrams and adjustment sets; and Stata, still common in economics. The R snippet below (not run here) is the shape of a typical matching analysis.
# R, not run here: nearest-neighbour matching on a logistic propensity score, then a balance table
library(MatchIt)
m <- matchit(subscribed ~ schooling + area, data = farms, method = "nearest", distance = "glm")
summary(m) # standardised mean differences before and after
matched <- match.data(m)
lm(yield ~ subscribed + schooling + area, data = matched, weights = weights)Using machine learning to estimate propensity scores and outcome models flexibly is the subject of the frontier lesson Causal machine learning; randomised A/B tests at enormous scale are covered in the Data Science elective's Experiments in products.
Common misconceptions
Misconception
"Correlation never implies causation." As a warning it is wise; as a rule it is false. Under stated assumptions (randomisation, unconfoundedness given measured variables, a valid instrument, parallel trends, continuity at a threshold) associations in data do identify causal effects. The useful question is never "is it only a correlation?" but "which assumption turns this association into an effect, and how plausible is it?"
Misconception
"Controlling for more variables is always safer." Adjusting for a mediator removes part of the effect you want to measure; adjusting for a collider, or selecting the sample on one, creates bias where there was none; and adjusting for a variable that affects only the treatment (an instrument) can amplify the bias from any unmeasured confounder. What to adjust for is a causal question, answered by the diagram, not by how many columns the data set has.
Misconception
"Propensity score matching fixes confounding." It balances the covariates that went into the score, and nothing else. If an important confounder is unmeasured, matched groups remain unbalanced on it, and the estimate is biased exactly as a regression would be. Matching is a way of adjusting, not a way of discovering what to adjust for.
Misconception
"An instrumental-variable estimate is the effect of the treatment." With effects that vary between people, it is the effect for compliers, the people whose treatment the instrument changed. A lottery for school places estimates the effect for families who would take a place if offered one, which may differ from the effect of expanding the schools to everyone.
Exercises
1. Causal inference
Warm-upHumanA library in Kraków runs a summer reading club. With an imaginary God's-eye view, here are four children's reading-test scores in September without the club, , and with it, . Kofi and Lena joined; Mei and Nour did not.
| Child | Joined? | ||
|---|---|---|---|
| Kofi | 30 | 38 | yes |
| Lena | 34 | 40 | yes |
| Mei | 50 | 52 | no |
| Nour | 46 | 50 | no |
What is the average treatment effect (ATE) of the club for these four children, in points?
2. Causal inference
CoreHumanA study in Chile asks whether a twelve-week exercise programme () lowers blood pressure (). The analysts believe this causal diagram:
- age affects both joining the programme and blood pressure;
- the programme causes weight loss, and weight loss lowers blood pressure;
- both joining the programme and having low blood pressure make people more likely to attend the final check-up visit.
Which set of variables satisfies the backdoor criterion?
3. Causal inference
CoreHumanAn arts-and-science academy in Nairobi admits applicants who score highly in mathematics or in music (or both). Among its students, mathematics and music scores are clearly negatively correlated, although among all applicants they are unrelated. What is the best explanation?
4. Causal inference
CoreHumanCity P halves its bus fares in January; neighbouring city Q does not. Average weekly bus trips, in thousands:
| Before | After | |
|---|---|---|
| City P (fare cut) | 120 | 150 |
| City Q (no change) | 100 | 110 |
What is the difference-in-differences estimate of the effect of the fare cut on weekly trips in city P, in thousands?
5. Causal inference
CoreHumanPlaces on a job-training course in Lyon are allocated by lottery. Among lottery winners, 50% complete the course; among losers, 10% complete it anyway (they find a place elsewhere). A year later, winners earn on average EUR 60 more per month than losers. Assuming the lottery affects earnings only through training and there are no defiers, what is the instrumental-variable (Wald) estimate of the effect of completing the course on monthly earnings, in euros?
6. Causal inference
CoreComputationalWrite ipw_ate(y, t, e), which estimates the average treatment effect by normalised inverse probability weighting. The inputs are arrays (or lists) of outcomes y, treatment indicators t (0/1 or Booleans) and propensity scores e. Weight each treated unit by and each untreated unit by , compute the weighted mean outcome in each group (dividing by the sum of that group's weights), and return the difference as a float.
For example, ipw_ate([1, 2, 3, 4], [1, 1, 0, 0], [0.5, 0.25, 0.5, 0.75]) is .
7. Causal inference
StretchComputationalWrite rd_jump(x, y, cutoff, h) for a sharp regression discontinuity. Fit a least-squares straight line to the points with cutoff <= x <= cutoff + h, and another to the points with cutoff - h <= x < cutoff. Return, as a float, the height of the right-hand line at x = cutoff minus the height of the left-hand line there.
Use NumPy only (for example np.linalg.lstsq or np.polyfit). Centring x at the cut-off makes each intercept the height you need.
Further reading and resources
- Causal Inference: What If · Miguel A. Hernán and James M. RobinsA free book that builds causal inference from potential outcomes and causal diagrams, with worked code.
- Causal Inference: The Mixtape · Scott CunninghamA free, readable online book on the econometric designs (matching, instruments, difference-in-differences, regression discontinuity), with R, Stata and Python code.
- The Sveriges Riksbank Prize in Economic Sciences 2021 · The Nobel PrizeCard, Angrist and Imbens, for natural experiments and the analysis of causal relationships. The popular and advanced background documents are linked from this page.
- Statistics and causal inference · Paul W. Holland · 1986Names "the fundamental problem of causal inference".
- Estimating causal effects of treatments in randomized and nonrandomized studies · Donald B. Rubin · 1974
- Causal diagrams for empirical research · Judea Pearl · 1995The backdoor criterion and the do-calculus.
- The central role of the propensity score in observational studies for causal effects · Paul R. Rosenbaum and Donald B. Rubin · 1983
- Identification and estimation of local average treatment effects · Guido W. Imbens and Joshua D. Angrist · 1994
- Regression discontinuity designs: a guide to practice · Guido W. Imbens and Thomas Lemieux · 2008
- Difference-in-differences with variation in treatment timing · Andrew Goodman-Bacon · 2021Why the classic two-way fixed-effects regression can mislead when units adopt a policy at different times.
- Using big data to emulate a target trial when a randomized trial is not available · Miguel A. Hernán and James M. Robins · 2016
- Sensitivity analysis in observational research: introducing the E-value · Tyler J. VanderWeele and Peng Ding · 2017
- DAGittyA free browser tool (and R package) for drawing causal diagrams and finding adjustment sets.
- DoWhyAn open-source Python library that makes the causal assumptions of an analysis explicit and tests their robustness.