A promise about the next case
A team in Nairobi has trained a gradient-boosted model that predicts the yield of a maize plot from satellite and weather features. A team in Lyon has a neural network that reads a skin image and names one of seven conditions. A team in Seoul has a language model that answers questions about tax rules. All three produce a single best guess, and all three are asked the same question by the people who will use them: how wrong could this be?
The classical answers need a model you believe. A regression interval relies on normal, constant-variance errors and a correctly specified mean. A Bayesian posterior predictive needs a prior and a likelihood. A boosted tree or a transformer has neither, and its own "confidence" scores are often badly calibrated.
Conformal prediction gives a different kind of answer. Wrap any trained model in a short calibration step, and it returns, for each new input , a set — an interval for a number, a set of labels for a class — with the guarantee
for a level you choose, such as 90%. The guarantee holds in finite samples, for any model however poor, and for any data distribution, under a single assumption: the calibration data and the new case are exchangeable. Vladimir Vovk, Alexander Gammerman and Glenn Shafer developed the idea from the late 1990s (their monograph is Algorithmic Learning in a Random World; the standard modern tutorial is by Angelopoulos and Bates), and since about 2019 it has spread through machine learning, medicine and the engineering of language-model systems.
The rank argument
Everything in conformal prediction rests on one observation about ranks.
Suppose we have a model , already trained, and a fresh calibration set of labelled examples it has never seen. For each one, compute a nonconformity score that says how badly the example fits the model — for regression, the absolute residual . Now a new example arrives, with score that we cannot compute yet because we do not know its label.
If the examples are exchangeable, their scores are too, and so the new score is equally likely to be the smallest, the second smallest, … or the largest of the . Its rank is uniform. So the chance that it falls at or below the -th smallest calibration score is exactly (when there are no ties). Pick so that this is at least , call that score , and the set of labels whose score would be at most — for regression, the interval — covers the truth with probability at least .
No model of the data appears anywhere in the argument. That is the source of conformal prediction's generality, and also of its limits: the guarantee is about the rank of a score among exchangeable scores, and says nothing about any particular .
Split conformal prediction
Definition— Exchangeability
Random variables are exchangeable if their joint distribution is unchanged by any permutation: has the same distribution as for every permutation . Independent, identically distributed variables are exchangeable; so are draws without replacement from a finite population. A time series with a trend is not.
Definition— Split conformal prediction
Given a trained model, a score function (large means the pair fits the model badly), a calibration set not used in training, and a level :
- compute the calibration scores ;
- let and let be the -th smallest score (or if );
- for a new input , output .
Theorem— Marginal coverage of split conformal prediction
If the calibration examples and the new example are exchangeable, then
If, in addition, the scores are almost surely distinct, then the coverage is also at most .
Proof
The model was trained on separate data, so it is fixed, and the scores are exchangeable. Assume first that they are distinct. Then exactly when , that is, when at most calibration scores lie below : when the rank of among all scores is at most . By exchangeability every rank from to is equally likely, so the probability is . Since , this lies between and . Ties can only make more likely, so the lower bound survives them.
∎
Three remarks matter in practice. The is not pedantry: with 19 calibration points and , , the largest score; with 18 points, and the honest interval is infinite. Using the plain empirical 95% quantile of the calibration scores would quietly under-cover. Second, the score is where the modelling goes: any function will give valid coverage, but a good one gives small sets. Third, "split" means the calibration data must not have been used for training — reuse them and the scores are no longer exchangeable with a new one, because the model has seen them. Full conformal prediction avoids the split by refitting the model once for every candidate label, at great computational cost, and the jackknife+ and CV+ methods of Barber, Candès, Ramdas and Tibshirani sit in between, using leave-one-out or cross-validation fits with a slightly weaker guarantee.
Split conformal regression, checked
Here is the whole procedure on synthetic data whose noise grows with : train a model, calibrate on 500 fresh points, and check the coverage. Because we built the world, we can compute the exact coverage of any interval by integrating over the known noise distribution, with no test-set noise in the check.
import numpy as np
from scipy import stats
rng = np.random.default_rng(2026)
mean_fn = lambda x: 2 + 0.8 * x + 0.1 * x**2 * np.sin(x) # the true regression function
sd_fn = lambda x: 0.3 + 0.25 * x # noise grows with x
def simulate(m):
x = rng.uniform(0, 10, m)
return x, mean_fn(x) + rng.normal(0, sd_fn(x))
# 1. Fit any model on a training set. Here: a degree-7 polynomial by least squares.
x_tr, y_tr = simulate(1000)
coef = np.polyfit(x_tr, y_tr, deg=7)
predict = lambda x: np.polyval(coef, x)
# 2. Calibrate on fresh data. The score is the absolute residual.
alpha, n_cal = 0.10, 500
x_cal, y_cal = simulate(n_cal)
scores = np.abs(y_cal - predict(x_cal))
k = int(np.ceil((n_cal + 1) * (1 - alpha))) # use the k-th smallest score
q_hat = np.sort(scores)[k - 1]
print(f"k = {k} of n = {n_cal}; q_hat = {q_hat:.3f}, so the interval is prediction +/- {q_hat:.3f}")
# 3. Because we built the world, we can compute the exact coverage of any interval width q.
grid = np.linspace(0, 10, 4001)
def true_coverage(q):
err = mean_fn(grid) - predict(grid)
return np.mean(stats.norm.cdf((err + q) / sd_fn(grid)) - stats.norm.cdf((err - q) / sd_fn(grid)))
print(f"exact coverage of this interval: {true_coverage(q_hat):.4f} (target: at least {1 - alpha:.2f})")
# 4. The coverage depends on the calibration set, which is random. Recalibrate 2,000 times.
cov = np.array([true_coverage(np.sort(np.abs(yc - predict(xc)))[k - 1])
for xc, yc in (simulate(n_cal) for _ in range(2000))])
theory = stats.beta(k, n_cal + 1 - k) # the distribution conformal theory predicts
print(f"\nover 2,000 calibration sets: mean {cov.mean():.4f}, sd {cov.std():.4f}, "
f"5th-95th percentile {np.quantile(cov, 0.05):.3f}-{np.quantile(cov, 0.95):.3f}")
print(f"theory, Beta({k}, {n_cal + 1 - k}): mean {theory.mean():.4f}, sd {theory.std():.4f}, "
f"5th-95th percentile {theory.ppf(0.05):.3f}-{theory.ppf(0.95):.3f}")The theory behind the last line is worth knowing. Given the calibration set, the coverage of the interval is , where is the distribution function of the score. Since are independent uniforms and is the -th order statistic, the coverage has a distribution (Vovk, 2012). The guarantee "at least 90%" is an average over calibration sets; any one calibration set may give a little less. If you need a guarantee for your calibration set — "with probability 95%, my coverage is at least 88%" — you can read it off the Beta distribution and choose a slightly larger . With 500 calibration points the wobble is about percentage points; with 5,000 it is about .
Marginal is not conditional
The guarantee averages over new inputs as well as over calibration sets. It says that 90% of cases are covered, not that each case has a 90% chance. When the noise varies with , an interval of constant width is too wide where the data are quiet and too narrow where they are noisy, and the average can still be exactly right.
The fix is in the score. Divide the residual by an estimate of the local spread, , where is a second model fitted on the training data; the interval becomes . The coverage guarantee is unchanged — it never depended on the score — but the width now follows the noise.
import numpy as np
rng = np.random.default_rng(7)
mean_fn = lambda x: 2 + 0.8 * x + 0.1 * x**2 * np.sin(x)
sd_fn = lambda x: 0.3 + 0.25 * x
def simulate(m):
x = rng.uniform(0, 10, m)
return x, mean_fn(x) + rng.normal(0, sd_fn(x))
def conformal_quantile(scores, alpha):
k = int(np.ceil((scores.size + 1) * (1 - alpha)))
return np.sort(scores)[k - 1]
alpha = 0.10
x_tr, y_tr = simulate(1000)
coef = np.polyfit(x_tr, y_tr, 7)
mu = lambda x: np.polyval(coef, x) # model for the mean
coef_s = np.polyfit(x_tr, np.abs(y_tr - mu(x_tr)), 1) # second model, for the spread: |residual| on x
spread = lambda x: np.maximum(np.polyval(coef_s, x), 0.05)
x_cal, y_cal = simulate(1000)
q_abs = conformal_quantile(np.abs(y_cal - mu(x_cal)), alpha) # score 1: |residual|
q_norm = conformal_quantile(np.abs(y_cal - mu(x_cal)) / spread(x_cal), alpha) # score 2: |residual| / spread
x_te, y_te = simulate(50000)
res = np.abs(y_te - mu(x_te))
cover_abs, width_abs = res <= q_abs, np.full(x_te.size, 2 * q_abs)
cover_norm, width_norm = res <= q_norm * spread(x_te), 2 * q_norm * spread(x_te)
print(f"{'x range':>9} | {'|residual| score':^21} | {'normalised score':^21}")
print(f"{'':>9} | {'coverage':>9} {'width':>10} | {'coverage':>9} {'width':>10}")
for lo in range(0, 10, 2):
b = (x_te >= lo) & (x_te < lo + 2)
print(f"{lo:>4}-{lo + 2:<4} | {cover_abs[b].mean():9.3f} {width_abs[b].mean():10.2f} | "
f"{cover_norm[b].mean():9.3f} {width_norm[b].mean():10.2f}")
print(f"{'overall':>9} | {cover_abs.mean():9.3f} {width_abs.mean():10.2f} | "
f"{cover_norm.mean():9.3f} {width_norm.mean():10.2f}")Both intervals have marginal coverage of about 90%, as promised. But the constant-width interval covers 100% of cases with below 2 and only 77% of those above 8, while the normalised interval stays between 0.89 and 0.92 in every band — and is narrower on average, 5.11 against 6.08. Other good scores exist: conformalized quantile regression (Romano, Patterson and Candès) fits models for the 5% and 95% conditional quantiles and conformalises the distance outside them; it is a widely used default.
Prediction sets for classification
For a classifier that outputs probabilities , the simplest score is one minus the probability given to the true class, . The prediction set then contains every class whose probability is at least (Sadinle, Lei and Wasserman showed it gives the smallest sets on average among methods with the guarantee, if the probabilities are right). The model need not be calibrated or even good: conformal prediction turns whatever scores it produces into sets with the promised coverage.
The example classifies land cover — water, forest, cropland, urban — from two synthetic features of the kind derived from satellite images, with a deliberately crude classifier (Gaussian classes with a shared covariance, which the data do not have).
import numpy as np
rng = np.random.default_rng(42)
classes = ["water", "forest", "cropland", "urban"]
centres = np.array([[0.0, 0.0], [2.5, 0.5], [1.5, 2.2], [3.5, 2.8]])
scales = np.array([0.6, 1.0, 1.1, 0.8]) # some classes are more spread out than others
def simulate(m):
y = rng.integers(0, 4, m)
x = centres[y] + rng.normal(size=(m, 2)) * scales[y][:, None]
return x, y
# A simple, deliberately imperfect classifier: Gaussians with one shared covariance (LDA).
x_tr, y_tr = simulate(2000)
means = np.array([x_tr[y_tr == c].mean(axis=0) for c in range(4)])
cov = sum(np.cov(x_tr[y_tr == c].T) for c in range(4)) / 4
P = np.linalg.inv(cov)
def predict_proba(x):
logits = x @ P @ means.T - 0.5 * np.sum(means @ P * means, axis=1)
logits -= logits.max(axis=1, keepdims=True)
p = np.exp(logits)
return p / p.sum(axis=1, keepdims=True)
alpha = 0.10
x_cal, y_cal = simulate(1000)
p_cal = predict_proba(x_cal)
scores = 1 - p_cal[np.arange(y_cal.size), y_cal] # score: 1 - probability given to the true class
n = scores.size
q_hat = np.sort(scores)[int(np.ceil((n + 1) * (1 - alpha))) - 1]
print(f"q_hat = {q_hat:.3f}: a class is in the set when the model gives it probability >= {1 - q_hat:.3f}")
x_te, y_te = simulate(20000)
p_te = predict_proba(x_te)
sets = p_te >= 1 - q_hat # boolean matrix: row i = prediction set for point i
covered = sets[np.arange(y_te.size), y_te]
size = sets.sum(axis=1)
top1 = (p_te.argmax(axis=1) == y_te).mean()
print(f"top-1 accuracy of the classifier: {top1:.3f}")
print(f"coverage of the 90% sets: {covered.mean():.3f}; average set size: {size.mean():.2f}")
print("distribution of set sizes:", {s: round(float(np.mean(size == s)), 3) for s in range(0, 5)})
print("\ncoverage by true class (only the average over classes is guaranteed):")
for c, name in enumerate(classes):
print(f" {name:9s} {covered[y_te == c].mean():.3f} average set size {size[y_te == c].mean():.2f}")
# Two example inputs: one clear-cut, one ambiguous.
for x in [np.array([[-0.5, -0.4]]), np.array([[2.4, 1.6]])]:
p = predict_proba(x)[0]
chosen = [classes[c] for c in range(4) if p[c] >= 1 - q_hat]
print(f"\nx = {x[0]}: probabilities {np.round(p, 2)} -> set {chosen}")
# Class-conditional ("Mondrian") calibration: one threshold per class, from that class's scores only.
q_class = np.array([np.sort(s)[int(np.ceil((s.size + 1) * (1 - alpha))) - 1]
for s in (scores[y_cal == c] for c in range(4))])
sets_c = p_te >= 1 - q_class[None, :]
cov_c = sets_c[np.arange(y_te.size), y_te]
print("\nclass-conditional calibration:")
for c, name in enumerate(classes):
print(f" {name:9s} {cov_c[y_te == c].mean():.3f}")
print(f" average set size {sets_c.sum(axis=1).mean():.2f}")The classifier alone is right 80.1% of the time; its 90% sets cover 91.1% of cases, with an average size of 1.38. The set size is the uncertainty signal: two-thirds of inputs get a single label, and the ambiguous input at , where three classes overlap, gets three. A downstream user can act on a singleton and send a large set for human review.
The guarantee is marginal over classes as well as inputs, and the run shows the price: water, which is easy, is covered 99.6% of the time, cropland only 81.2%. Calibrating separately within each class — class-conditional or Mondrian conformal prediction, named by Vovk and colleagues after Piet Mondrian's grid paintings because it partitions the data — gives each class its own guarantee. The four per-class coverages become 0.905, 0.913, 0.906 and 0.876; with about 250 calibration points per class, a wobble of points is expected, as the Beta distribution of the previous section predicts. For many classes and images, the APS and RAPS scores (Romano, Sesia and Candès; Angelopoulos et al.) add the probabilities of classes in order until the true class is reached, which gives sets that adapt better to hard inputs.
When exchangeability fails
Exchangeability is a strong assumption in disguise. It fails when:
- the inputs shift (covariate shift): a model calibrated on one hospital's patients is used at another, or a yield model calibrated in wet years meets a drought;
- the labels shift: a disease becomes more common during an outbreak;
- the data are ordered in time and the world drifts;
- feedback links prediction and outcome, as when a model's predictions change who is treated.
Under covariate shift — the distribution of changes, but given does not — there is a repair. Tibshirani, Barber, Candès and Ramdas showed that weighting each calibration score by the likelihood ratio , and giving the test point's own weight to a score of , restores the guarantee. The ratio is rarely known, but it can be estimated with a classifier trained to tell calibration inputs from unlabelled deployment inputs, which are usually easy to collect. Below, inputs are centred at 3 during training and calibration and at 5 in deployment, where the noise is larger.
import numpy as np
from scipy import stats
rng = np.random.default_rng(11)
mean_fn = lambda x: 2 + 0.8 * x + 0.1 * x**2 * np.sin(x)
sd_fn = lambda x: 0.3 + 0.25 * np.abs(x)
def simulate(m, centre):
x = rng.normal(centre, 1.5, m) # where the inputs come from can shift
return x, mean_fn(x) + rng.normal(0, sd_fn(x))
alpha = 0.10
x_tr, y_tr = simulate(2000, 3.0) # training population: inputs centred at 3
coef = np.polyfit(x_tr, y_tr, 7)
mu = lambda x: np.polyval(coef, x)
def split_quantile(s, alpha):
return np.sort(s)[int(np.ceil((s.size + 1) * (1 - alpha))) - 1]
def weighted_quantiles(s_cal, w_cal, w_test, alpha):
"""Weighted conformal (Tibshirani et al., 2019): calibration scores weighted by the density
ratio, plus a point mass at +infinity carrying the test point's own weight."""
order = np.argsort(s_cal)
s_sorted, cum_w = s_cal[order], np.cumsum(w_cal[order])
need = (1 - alpha) * (cum_w[-1] + w_test) # weight needed at or below the quantile
j = np.searchsorted(cum_w, need)
return np.where(j < s_sorted.size, s_sorted[np.minimum(j, s_sorted.size - 1)], np.inf)
def estimate_ratio(x_cal, x_unlabelled):
"""Logistic regression telling calibration inputs (0) from unlabelled deployment inputs (1).
With equal sample sizes, its odds estimate the density ratio p_deploy(x) / p_cal(x)."""
Z = np.column_stack([np.ones(x_cal.size + x_unlabelled.size), np.r_[x_cal, x_unlabelled]])
t = np.r_[np.zeros(x_cal.size), np.ones(x_unlabelled.size)]
w = np.zeros(2)
for _ in range(25): # Newton's method
p = 1 / (1 + np.exp(-Z @ w))
w += np.linalg.solve(Z.T @ (Z * (p * (1 - p))[:, None]), Z.T @ (t - p))
return lambda x: np.exp(w[0] + w[1] * x)
true_ratio = lambda x: stats.norm.pdf(x, 5.0, 1.5) / stats.norm.pdf(x, 3.0, 1.5)
cov = {"ordinary split conformal": [], "weighted, true ratio": [], "weighted, estimated ratio": []}
for rep in range(200):
x_cal, y_cal = simulate(1000, 3.0) # calibration: same population as training
s_cal = np.abs(y_cal - mu(x_cal))
x_te, y_te = simulate(2000, 5.0) # deployment: inputs have shifted upwards
s_te = np.abs(y_te - mu(x_te))
est_ratio = estimate_ratio(x_cal, rng.normal(5.0, 1.5, 1000))
cov["ordinary split conformal"].append(np.mean(s_te <= split_quantile(s_cal, alpha)))
cov["weighted, true ratio"].append(np.mean(s_te <= weighted_quantiles(s_cal, true_ratio(x_cal), true_ratio(x_te), alpha)))
cov["weighted, estimated ratio"].append(np.mean(s_te <= weighted_quantiles(s_cal, est_ratio(x_cal), est_ratio(x_te), alpha)))
print("coverage in deployment, over 200 repetitions (target 0.90):")
for name, c in cov.items():
print(f" {name:26s} mean {np.mean(c):.3f} middle 90% of runs {np.quantile(c, 0.05):.3f}-{np.quantile(c, 0.95):.3f}")
w = true_ratio(x_cal)
print(f"effective calibration sample size after weighting (last run): {w.sum() ** 2 / (w ** 2).sum():.0f} of 1000")Ordinary conformal prediction, blind to the shift, covers only 76.3% of deployment cases. Weighting restores the average coverage to about 90.7%, and the estimated ratio does as well as the true one. But the runs vary much more, from 0.852 to 0.971: weighting concentrates the calibration on the few points that resemble deployment, leaving an effective sample size of about 169 out of 1,000. When the shift is so large that almost no calibration points resemble deployment, the honest weighted interval becomes infinite.
For data in time, adaptive conformal inference (Gibbs and Candès, 2021) adjusts online — widening after misses and narrowing after hits — and guarantees that the long-run miss rate approaches even under arbitrary drift, at the cost of saying less about any single time. Barber et al. bound how much coverage can be lost when exchangeability holds only approximately.
Conformal prediction in practice (2026)
Medicine. Clinical uses are attractive because a set is a natural clinical object: a differential diagnosis, a range for a laboratory value, a flag that says "uncertain — refer". Researchers have built conformal sets for dermatology and radiology classifiers, intervals for drug-response and length-of-stay predictions, and, for more than a decade, conformal predictors for compound activity in pharmaceutical chemistry. The open problems are statistical, not computational. Marginal coverage across patients does not mean coverage for this patient, and coverage can be uneven across age groups, sexes or ethnic groups unless those groups are calibrated separately; models move between hospitals, scanners and years, breaking exchangeability; and the calibration labels themselves (a pathologist's diagnosis, a coded outcome) contain errors. As of 2026, conformal methods appear mainly in research studies and in some decision-support prototypes. Their coverage guarantees complement prospective clinical validation; they do not replace it.
Language models. A language model's output is not a number or a label, so the first task is to define what "covered" means. Several ideas are now established. Conformal language modelling (Quach et al., 2023) samples answers until the set of samples contains an acceptable one with a guaranteed probability, using a calibrated stopping rule. Conformal factuality (Mohri and Hashimoto, 2024) splits a long answer into individual claims and removes the least confident ones until the remaining answer is entirely correct with probability . KnowNo (Ren et al., 2023) lets a robot's language-model planner form a conformal set of possible actions and ask a human for help when the set has more than one member. More generally, conformal risk control (Angelopoulos et al., 2022) and Learn then Test (Angelopoulos et al., 2021) calibrate thresholds to control other error rates, such as the fraction of missed tumour pixels in a segmentation or the rate of unsafe responses.
Each of these needs a labelled calibration set drawn from the prompts the system will actually face, and a definition of a correct or acceptable answer that someone has to supply and check. Prompt distributions drift quickly, so the exchangeability assumption needs particular care. The guarantees are real, and narrower than they sound in a press release.
Software. Conformal methods take a few lines of NumPy, as above. Open-source implementations include MAPIE in Python and the conformalInference package in R; deep-learning frameworks have their own community libraries. Whatever the tool, check that its calibration step uses data held out from training and applies the correction.
Common misconceptions
Misconception
"A 90% conformal interval contains this case's outcome with probability 0.9." The guarantee is marginal: averaged over new cases and over calibration sets. For a particular the coverage may be higher or lower — 100% or 77% in the bands of the second example. Conditional guarantees need group-wise calibration or extra assumptions.
Misconception
"Conformal prediction makes a bad model good." It makes a bad model honest. Coverage holds for any model, but a poor model has large scores and therefore wide intervals or large sets. Efficiency — small sets — still has to be earned by modelling.
Misconception
"Distribution-free means assumption-free." The method needs exchangeability between calibration and future data, and calibration data that the model was not trained on. Under shift or drift the guarantee can fail badly, as the 76% coverage above shows.
Misconception
"My calibration set gives exactly 90% coverage." Given the calibration set, coverage has a Beta distribution centred near ; with 500 points its standard deviation is about 1.3 percentage points. Use more calibration data, or a slightly more conservative , if you need a guarantee for the calibration set you actually have.
Exercises
1. Conformal prediction
Warm-upHumanYou have calibration residuals and want a 90% split conformal interval (). The half-width is the -th smallest residual. What is ?
2. Conformal prediction
CoreHumanFor a 95% split conformal interval (), what is the smallest number of calibration points for which the interval is finite, that is, for which ?
3. Conformal prediction
CoreHumanA hospital calibrates a 90% split conformal interval for patients' length of stay on 2,000 recent discharges from the same hospital. Which statement is justified by the conformal guarantee alone?
4. Conformal prediction
CoreHumanIn which situations does the split conformal guarantee not apply as stated? Select all that apply.
5. Conformal prediction
CoreComputationalWrite conformal_halfwidth(residuals, alpha) that returns the split conformal half-width from an array of absolute calibration residuals: the -th smallest residual with , or np.inf if .
The starter uses the plain empirical quantile, a common mistake; the tests will show why it is not enough.
6. Conformal prediction
StretchComputationalWrite prediction_sets(p_cal, y_cal, p_test, alpha) for conformal classification with the score .
p_cal: array of shape(n, K), the model's class probabilities on the calibration inputs;y_cal: integer array of shape(n,), their true classes;p_test: array of shape(m, K).
Return a boolean array of shape (m, K) whose entry [i, c] is True when class c belongs to the prediction set of test input i. If , every class belongs to every set.
7. Conformal prediction
Warm-upHumanConformal prediction's coverage guarantee needs only one assumption about the data: that the calibration examples and the new example are ______ (one word).
Further reading and resources
- A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification · Anastasios Angelopoulos and Stephen Bates · 2021The standard modern tutorial, with code and many worked applications.
- Algorithmic Learning in a Random World · Vladimir Vovk, Alexander Gammerman and Glenn Shafer · 2005The founding monograph of conformal prediction (second edition 2022).
- Distribution-Free Predictive Inference For Regression · Jing Lei, Max G'Sell, Alessandro Rinaldo, Ryan Tibshirani and Larry Wasserman · 2016Split conformal prediction for regression, analysed carefully.
- Conditional validity of inductive conformal predictors · Vladimir Vovk · 2012Coverage conditional on the calibration set, and the limits of conditional coverage.
- Conformalized Quantile Regression · Yaniv Romano, Evan Patterson and Emmanuel Candès · 2019
- Classification with Valid and Adaptive Coverage · Yaniv Romano, Matteo Sesia and Emmanuel Candès · 2020
- Least Ambiguous Set-Valued Classifiers with Bounded Error Levels · Mauricio Sadinle, Jing Lei and Larry Wasserman · 2016
- Uncertainty Sets for Image Classifiers using Conformal Prediction · Anastasios Angelopoulos, Stephen Bates, Jitendra Malik and Michael Jordan · 2020
- The limits of distribution-free conditional predictive inference · Rina Foygel Barber, Emmanuel Candès, Aaditya Ramdas and Ryan Tibshirani · 2019
- Conformal Prediction With Conditional Guarantees · Isaac Gibbs, John Cherian and Emmanuel Candès · 2023
- Conformal Prediction Under Covariate Shift · Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candès and Aaditya Ramdas · 2019
- Conformal prediction beyond exchangeability · Rina Foygel Barber, Emmanuel Candès, Aaditya Ramdas and Ryan Tibshirani · 2022
- Adaptive Conformal Inference Under Distribution Shift · Isaac Gibbs and Emmanuel Candès · 2021
- Predictive inference with the jackknife+ · Rina Foygel Barber, Emmanuel Candès, Aaditya Ramdas and Ryan Tibshirani · 2019
- Conformal Risk Control · Anastasios Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei and Tal Schuster · 2022
- Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control · Anastasios Angelopoulos, Stephen Bates, Emmanuel Candès, Michael Jordan and Lihua Lei · 2021
- Conformal Language Modeling · Victor Quach and others · 2023
- Language Models with Conformal Factuality Guarantees · Christopher Mohri and Tatsunori Hashimoto · 2024
- Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners · Allen Ren and others · 2023
- MAPIEAn open-source Python library of conformal methods, compatible with scikit-learn.
- conformalInference · Ryan Tibshirani and collaboratorsAn R package implementing conformal inference for regression.