Skip to content
LumenaryAcademy
Foundations 65 min

Estimation and confidence intervals

From a sample to a range of plausible values — estimators and their errors, what "95% confidence" really promises (checked by simulation), the bootstrap, and how many observations to collect.

Code tested · 6AI-draftedAwaiting review by a statistician

Your guides: Chloe · The Statistician, Margaret · The Simulator, Gideon · The Sceptic

MotivationH

A number and how far to trust it

A health ministry in Malawi wants to know what fraction of rural households have a solar panel, to plan a programme of support. A survey of 400 randomly chosen households finds 92 with a panel: 23%. Should the ministry plan for 23%?

It should plan for "about 23%", and the whole art of this lesson is to say how much "about" covers. From the previous lesson we know that a different sample of 400 would have given a different percentage, typically differing by about two percentage points. So a responsible report says: 23%, with a 95% confidence interval from 19% to 27%. The interval tells the reader how precise the survey was. A survey of only 40 households that found 9 with panels (22.5%) would report almost the same percentage, but its interval would run from about 12% to 38%; a planner who saw only the point estimates could not tell the two surveys apart.

Estimation is the craft of turning data into a best guess (a point estimate) together with an honest statement of its uncertainty (an interval estimate). It raises three questions that this lesson answers in turn. What makes one recipe for a guess better than another? What exactly does "95% confidence" promise? And how many observations do we need for a given precision?

IntuitionH

Recipes, targets and nets

An estimator is a recipe: a rule that turns any possible sample into a number. "Add up the values and divide by nn" is an estimator of the population mean; "sort the values and take the middle one" is another. Because samples vary, the recipe's output varies, and we judge a recipe by how its outputs behave over all the samples it might meet, its sampling distribution.

Two different things can go wrong, and the picture of a dartboard keeps them apart.

low bias,low variancelow bias,high variancehigh bias,low variancehigh bias,high variance
Bias and variance, pictured as ten throws at a target (the bullseye is the true value; each dot is the estimate from one sample). Bias is a systematic offset of the whole cloud; variance is its scatter. A good estimator has both small. The two can trade off against each other.

A confidence interval adds a second idea. Instead of one throw at the target, we cast a net: a range computed from the data, wide enough that the procedure catches the true value in a stated fraction of samples, such as 95%.

Analogy, not a formal construction

Think of a fisher who throws a net into a murky lake to catch one particular fish she cannot see. Her nets are made so that 95% of throws catch it. After a throw she cannot see whether this net holds the fish; she knows only that her method succeeds 95 times in 100. The "95%" belongs to her way of throwing, not to any particular net.

DefinitionsH

Definitions

Definition— estimand, estimator, estimate

The estimand is the quantity we want to know, such as a population mean μ\mu or proportion pp, written generally as θ\theta. An estimator θ^\hat\theta is a function of the sample, and therefore a random variable. An estimate is the value it takes on the data actually observed.

Definition— bias, standard error and mean squared error

The bias of θ^\hat\theta is E[θ^]−θE[\hat\theta] - \theta; the estimator is unbiased if this is 0 for every θ\theta. Its standard error is SD⁡(θ^)\operatorname{SD}(\hat\theta). Its mean squared error is MSE⁡(θ^)=E[(θ^−θ)2]\operatorname{MSE}(\hat\theta) = E\big[(\hat\theta - \theta)^2\big], the average squared distance from the truth; its square root is the RMSE.

Definition— consistency

An estimator is consistent if, as the sample size grows, it converges in probability to θ\theta: the chance of missing by more than any fixed amount tends to 0.

Definition— confidence interval

A (1−α)(1-\alpha) confidence interval for θ\theta is a pair of statistics (L,U)(L, U) such that P(L≤θ≤U)≥1−αP(L \le \theta \le U) \ge 1 - \alpha for every possible value of θ\theta. The probability is over repeated samples; it is called the coverage of the procedure. The value 1−α1 - \alpha, usually 95%, is the confidence level.

Theorem— Bias–variance decomposition

For any estimator, MSE⁡(θ^)=bias⁡(θ^)2+Var⁡(θ^)\operatorname{MSE}(\hat\theta) = \operatorname{bias}(\hat\theta)^2 + \operatorname{Var}(\hat\theta).

Proof

Write m=E[θ^]m = E[\hat\theta]. Then θ^−θ=(θ^−m)+(m−θ)\hat\theta - \theta = (\hat\theta - m) + (m - \theta). Square and take expectations: the cross term is 2(m−θ) E[θ^−m]=02(m - \theta)\,E[\hat\theta - m] = 0, leaving E[(θ^−m)2]+(m−θ)2=Var⁡(θ^)+bias⁡2E[(\hat\theta - m)^2] + (m - \theta)^2 = \operatorname{Var}(\hat\theta) + \operatorname{bias}^2.

∎

The theorem explains why "unbiased" is not the same as "best". An estimator with a little bias and much less variance can have a smaller total error. This trade-off runs through all of modern statistics and machine learning; Lumenary Math's Statistical inference treats estimators and maximum likelihood more formally.

ComputationC

Choosing an estimator: a simulation

Two classic choices show the ideas at work. First, the sample variance: should we divide the sum of squared deviations by nn or by n−1n - 1? Second, for the centre of a distribution, should we use the sample mean or the sample median? We simulate 100,000 samples and measure each estimator's bias, standard error and RMSE.

python
import numpy as np

rng = np.random.default_rng(3)
reps = 100_000

def report(name, estimates, truth):
    bias = estimates.mean() - truth
    sd = estimates.std()
    rmse = np.sqrt(np.mean((estimates - truth) ** 2))
    print(f"{name:34} bias {bias:+7.3f}   sd {sd:6.3f}   rmse {rmse:6.3f}")

# 1. Two estimators of a variance (true value 4) from samples of size 5.
x = rng.normal(10.0, 2.0, size=(reps, 5))
report("variance, divide by n - 1", x.var(axis=1, ddof=1), 4.0)
report("variance, divide by n", x.var(axis=1, ddof=0), 4.0)

# 2. Mean or median for the centre (true value 0) of samples of size 25?
normal = rng.normal(0.0, 1.0, size=(reps, 25))
heavy = rng.standard_t(3, size=(reps, 25))          # t with 3 df: occasional wild values
print()
report("normal data: sample mean", normal.mean(axis=1), 0.0)
report("normal data: sample median", np.median(normal, axis=1), 0.0)
report("heavy-tailed data: sample mean", heavy.mean(axis=1), 0.0)
report("heavy-tailed data: sample median", np.median(heavy, axis=1), 0.0)
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Dividing by n−1n - 1 (Bessel's correction) removes the bias: the average estimate is 4.004 against a true variance of 4, while dividing by nn is too low by 0.8 on average, exactly σ2/n\sigma^2/n. The deviations are measured from the sample mean, which sits closer to the data than the true mean does, so they are systematically too small, and n−1n - 1 corrects for it.

For the centre, the answer depends on the data. With normal data the mean wins (standard error 0.201 against 0.249 for the median). With heavy-tailed data, simulated here from a tt distribution with 3 degrees of freedom, occasional extreme values throw the mean around, and the median wins clearly (0.274 against 0.341). No estimator is best for every population, which is one reason to look at the data before choosing.

Worked examplesH

Intervals for means and proportions

The mean: Student's t interval

If the data are a random sample from a normal population, then T=Xˉ−μs/nT = \frac{\bar X - \mu}{s/\sqrt n} follows a tt distribution with n−1n - 1 degrees of freedom. It looks like the standard normal but has heavier tails, because ss itself is estimated and sometimes comes out too small. Solving −t∗≤T≤t∗-t^* \le T \le t^* for μ\mu, where t∗t^* is the 0.975-quantile of tn−1t_{n-1}, gives the 95% interval Xˉ±t∗sn.\bar X \pm t^* \frac{s}{\sqrt n}. William Sealy Gosset worked this out in 1908 as a brewer at Guinness in Dublin, where experiments on barley and yeast had small samples. Guinness let him publish only under a pen name, so it is still called Student's tt. As nn grows, t∗t^* approaches 1.96; for n=16n = 16 it is 2.131, and for n=5n = 5 it is 2.776.

Example— Teff yields in the Ethiopian highlands

An agronomist records the yield of teff on 16 randomly chosen plots: mean xˉ=1.62\bar x = 1.62 tonnes per hectare, standard deviation s=0.30s = 0.30. The standard error is 0.30/16=0.0750.30/\sqrt{16} = 0.075, and t∗=2.131t^* = 2.131 for 15 degrees of freedom, so the 95% interval is 1.62±2.131×0.075=1.62±0.160,that is (1.46, 1.78) t/ha.1.62 \pm 2.131 \times 0.075 = 1.62 \pm 0.160, \quad \text{that is } (1.46,\ 1.78) \text{ t/ha}. Using 1.96 instead of 2.131 would give (1.47,1.77)(1.47, 1.77): slightly too narrow, and with the coverage loss we will measure below.

The tt interval assumes a random sample, independent observations and a normal population. The normality assumption matters less as nn grows, thanks to the central limit theorem, but as the previous lesson showed, "grows" can mean a lot for skewed data.

The proportion: Wald and Wilson

For a proportion, p^=k/n\hat p = k/n has standard error p(1−p)/n\sqrt{p(1-p)/n}. Plugging in p^\hat p for pp gives the textbook Wald interval p^±1.96p^(1−p^)/n\hat p \pm 1.96\sqrt{\hat p(1 - \hat p)/n}. For the Malawi survey, p^=92/400=0.23\hat p = 92/400 = 0.23, the standard error is 0.021, and the interval is (0.189,0.271)(0.189, 0.271).

The Wald interval has a known flaw: when p^\hat p is near 0 or 1, its estimated standard error is too small, and at p^=0\hat p = 0 it collapses to the single point 0. A clinic that sees no side effects among 30 patients would report the interval [0,0][0, 0], which claims certainty from 30 people. The Wilson interval (Edwin Bidwell Wilson, 1927) inverts the test of pp instead of plugging in p^\hat p, and behaves much better: p^+z22n1+z2n  ±  z1+z2np^(1−p^)n+z24n2,z=1.96.\frac{\hat p + \frac{z^2}{2n}}{1 + \frac{z^2}{n}} \;\pm\; \frac{z}{1 + \frac{z^2}{n}}\sqrt{\frac{\hat p(1 - \hat p)}{n} + \frac{z^2}{4n^2}}, \qquad z = 1.96. For 92 of 400 it gives (0.191,0.274)(0.191, 0.274), nearly the same as Wald. For 0 of 30 it gives (0,0.114)(0, 0.114): zero side effects in 30 patients is consistent with a true rate as high as about 11%. A handy shortcut for zero events is the rule of three: with 0 events in nn independent trials, 3/n3/n is an approximate 95% upper bound for the rate.

ExplanationH

What 95% means, checked by simulation

The definition of a confidence interval is a promise about a procedure: in repeated samples, intervals built this way contain the true value 95% of the time. The figure shows the promise in action.

missesmissestrue mean 50303540455055606570
Twenty-five 95% t intervals, each from a different simulated sample of 10 values from a population with mean 50. Each interval is centred on its own sample mean and has its own width. Two of the 25 miss the true mean; in the long run, about 1 in 20 would.

Once a particular interval has been computed, such as (1.46,1.78)(1.46, 1.78) for teff, it either contains the true mean yield or it does not; we just do not know which. The 95% describes how often the method succeeds, and that is what makes it a statement we can check. A simulation can generate thousands of samples from a population whose truth we know and count how often the intervals succeed.

python
import numpy as np
from scipy import stats

rng = np.random.default_rng(8)
reps = 20_000

def coverage(draw, true_mean, n, use_t=True):
    """Fraction of 95% intervals  mean +/- crit * s / sqrt(n)  that contain the true mean."""
    x = draw((reps, n))
    m, s = x.mean(axis=1), x.std(axis=1, ddof=1)
    crit = stats.t.ppf(0.975, df=n - 1) if use_t else 1.96
    half = crit * s / np.sqrt(n)
    return np.mean((m - half <= true_mean) & (true_mean <= m + half))

normal = lambda size: rng.normal(50.0, 10.0, size)
skewed = lambda size: rng.lognormal(0.0, 1.0, size)     # true mean exp(0.5)
print("coverage of nominal 95% intervals for a mean")
print("  n   normal data, t   normal data, 1.96   skewed data, t")
for n in [5, 10, 40]:
    print(f"{n:3} {coverage(normal, 50.0, n):16.3f} {coverage(normal, 50.0, n, use_t=False):19.3f}"
          f" {coverage(skewed, np.exp(0.5), n):16.3f}")

# For a proportion the coverage can be computed exactly: add up P(X = k) over the k
# whose interval contains the true p.
def wald(k, n, z=1.96):
    p = k / n
    half = z * np.sqrt(p * (1 - p) / n)
    return p - half, p + half

def wilson(k, n, z=1.96):
    p = k / n
    centre = (p + z**2 / (2 * n)) / (1 + z**2 / n)
    half = z / (1 + z**2 / n) * np.sqrt(p * (1 - p) / n + z**2 / (4 * n**2))
    return centre - half, centre + half

def exact_coverage(method, n, p):
    k = np.arange(n + 1)
    lo, hi = method(k, n)
    return np.sum(stats.binom.pmf(k, n, p) * ((lo <= p) & (p <= hi)))

print("\nexact coverage of nominal 95% intervals for a proportion")
for n, p in [(50, 0.5), (50, 0.1), (50, 0.02), (500, 0.02)]:
    print(f"n = {n:3}, p = {p:4}:  Wald {exact_coverage(wald, n, p):.3f}   Wilson {exact_coverage(wilson, n, p):.3f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The table shows three things.

  1. The tt interval delivers what it promises when its assumptions hold. With normal data, coverage is 0.952 at every sample size, within simulation error of 0.95.
  2. Shortcuts cost coverage in small samples. Using 1.96 in place of t∗t^* gives 87.8% coverage at n=5n = 5; the interval claims 95% confidence while missing one time in eight.
  3. Broken assumptions cost coverage too. For lognormal data the tt interval covers only 82% of the time at n=5n = 5 and 89% at n=40n = 40. Skewness makes the sample mean and ss move together (samples that happen to include a large value have both a high mean and a large ss), and the symmetric interval cannot keep up.

For proportions, coverage can be computed exactly by adding up binomial probabilities. With n=50n = 50 and p=0.02p = 0.02, the Wald interval covers the truth only 63.5% of the time, and even at n=500n = 500 it falls short, at 92.8%; the Wilson interval stays close to 95%. Brown, Cai and DasGupta's 2001 review recommends against the Wald interval for exactly these reasons. (No interval for a discrete count hits 95% exactly; coverage zigzags as pp changes, which is why Wilson's numbers run a little above or below.)

ComputationC

The bootstrap: intervals by resampling

Formulas like s/ns/\sqrt n exist for a few statistics. What is the standard error of a median, a ratio, a trimmed mean, or a machine-learning model's accuracy? In 1979 Bradley Efron proposed a general answer, the bootstrap, named after the impossible feat of lifting yourself up by your own bootstraps.

The idea uses the plug-in principle. The sampling distribution depends on the unknown population. The sample is our best picture of that population. So we pretend the sample is the population and draw new samples from it: resample nn values with replacement, compute the statistic, and repeat thousands of times. The spread of these bootstrap statistics estimates the standard error, and their 2.5th and 97.5th percentiles give a percentile bootstrap interval.

python
import numpy as np

# Synthetic data: monthly mobile data use (GB) reported by 40 people. Skewed, with a few heavy users.
rng = np.random.default_rng(12)
data = np.round(rng.lognormal(mean=1.6, sigma=0.8, size=40), 1)
print("data:", np.sort(data))
print(f"sample mean {data.mean():.2f} GB, sample median {np.median(data):.2f} GB")

# The bootstrap: resample the data with replacement, recompute the statistic, repeat.
B = 10_000
resamples = rng.choice(data, size=(B, data.size), replace=True)
boot_medians = np.median(resamples, axis=1)
boot_means = resamples.mean(axis=1)

lo, hi = np.percentile(boot_medians, [2.5, 97.5])
print(f"bootstrap standard error of the median: {boot_medians.std(ddof=1):.2f} GB")
print(f"95% percentile bootstrap interval for the median: ({lo:.2f}, {hi:.2f}) GB")

# For the mean we can compare the bootstrap with the formula s / sqrt(n).
print(f"bootstrap SE of the mean {boot_means.std(ddof=1):.3f}   formula s/sqrt(n) {data.std(ddof=1) / np.sqrt(data.size):.3f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The 40 reported values are skewed: most people use 2 to 10 GB a month, and three use about 20. The bootstrap standard error of the median is 0.68 GB, and the 95% percentile interval for the population median runs from 3.55 to 6.60 GB. For the mean, where we can check, the bootstrap standard error (0.754) agrees closely with s/ns/\sqrt n (0.772).

The bootstrap is powerful, and it is easy to trust it too much.

  • It cannot create information. Every resample is built from the same 40 values. If the sample is small or unrepresentative, so is every resample. With n=10n = 10, percentile intervals tend to be too narrow.
  • Some statistics break it. The bootstrap fails for the maximum of a sample, and is unreliable for statistics that depend on a few extreme values.
  • The resampling must copy the design. Observations that come in clusters (pupils within schools) or in time order must be resampled as clusters or blocks, not one by one.

Better versions exist. The BCa (bias-corrected and accelerated) interval adjusts the percentiles for bias and skewness, and is the default in SciPy's scipy.stats.bootstrap. Tim Hesterberg's 2015 article in the references is a careful guide to what works. Resampling and simulation returns to the bootstrap and its relatives in depth.

Worked examplesH

How many observations?

The margin of error of a 95% interval for a mean is about 1.96 σ/n1.96\,\sigma/\sqrt n. Setting it equal to a target margin mm and solving for nn gives the planning formula n=(z σm)2,n=z2 p(1−p)m2 for a proportion,n = \left(\frac{z\,\sigma}{m}\right)^2, \qquad n = \frac{z^2\, p(1-p)}{m^2} \text{ for a proportion}, rounded up. The proportion formula needs a guess at pp; since p(1−p)p(1-p) is largest at p=0.5p = 0.5, using 0.5 gives a sample size that is enough whatever the truth.

python
import math
from scipy import stats

def n_for_mean(sigma, margin, conf=0.95):
    """Sample size so that a conf-level interval for a mean has half-width about `margin`."""
    z = stats.norm.ppf(0.5 + conf / 2)
    return math.ceil((z * sigma / margin) ** 2)

def n_for_proportion(margin, p=0.5, conf=0.95):
    """Sample size for a proportion; p = 0.5 is the worst (largest) case."""
    z = stats.norm.ppf(0.5 + conf / 2)
    return math.ceil(z**2 * p * (1 - p) / margin**2)

print("teff yields, sigma about 0.30 t/ha:")
for margin in [0.10, 0.05, 0.025]:
    print(f"  margin +/-{margin:5.3f} t/ha needs n = {n_for_mean(0.30, margin)}")
print("a proportion, worst case p = 0.5:")
for margin in [0.05, 0.03, 0.01]:
    print(f"  margin +/-{margin:.2f} needs n = {n_for_proportion(margin)}")
print("  margin +/-0.03 at 99% confidence needs n =", n_for_proportion(0.03, conf=0.99))
print("  margin +/-0.03 if p is near 0.1 needs n =", n_for_proportion(0.03, p=0.1))
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Each halving of the margin quadruples the sample, so a margin of ±1\pm 1 point costs nine times as many interviews as ±3\pm 3 points (9,604 against 1,068). Raising the confidence level from 95% to 99% raises the sample size by about 73%. These are minimums. In practice the planner inflates them for the share of people expected not to respond, and for the design effect of a clustered sample, and gets σ\sigma from a pilot study or a previous survey, which is itself uncertain. When the aim is to detect an effect rather than to measure a quantity, sample size is planned through power instead, in the next lesson.

ExplanationH

Intervals in professional practice

Report the interval, not just the estimate. Medical journals following the CONSORT reporting guidelines for clinical trials expect an effect estimate with its confidence interval, and national statistics offices publish intervals or margins of error with their survey estimates. An estimate without an interval cannot be judged.

Software. In Python, scipy.stats.t.interval and scipy.stats.binomtest(...).proportion_ci(method="wilson") give the intervals above, and scipy.stats.bootstrap does resampling. In R, the same intervals come from three built-in functions, shown here for comparison and not run:

r
# R, not run here
t.test(yield_t_ha)$conf.int            # Student's t interval for a mean
prop.test(92, 400)$conf.int            # Wilson interval (with a continuity correction by default)
binom.test(0, 30)$conf.int             # Clopper-Pearson "exact" interval

Other kinds of interval. A Bayesian credible interval answers the question many people wrongly think a confidence interval answers: given the data and a prior, where is the parameter, with 95% probability? It needs a prior distribution, and it is the subject of Bayesian inference. A prediction interval covers a single future observation rather than a mean, and is much wider. A tolerance interval covers a stated fraction of the population. Mixing these up is a common and consequential error in engineering reports.

History. Jerzy Neyman, a Polish statistician working in London, set out the theory of confidence intervals in 1937, including the interpretation used here. Ronald Fisher proposed a rival "fiducial" theory that never won general acceptance, and the debate between them was famously bitter. Efron's bootstrap arrived in 1979, just as computers became cheap enough to make it practical.

Common misconceptionsH

Common misconceptions

Misconception

"There is a 95% probability that the true value lies in this interval." In the frequentist framework the true value is fixed, so a computed interval either contains it or not. The 95% is the success rate of the method over repeated samples. (A Bayesian credible interval does support a probability statement of this kind, under its prior.)

Misconception

"95% of the data fall inside the confidence interval." The interval for a mean is much narrower than the spread of individual values: it shrinks like 1/n1/\sqrt n, while the data do not. For teff, individual plots ranged far beyond (1.46,1.78)(1.46, 1.78). A range meant to cover individual values is a prediction or tolerance interval.

Misconception

"An unbiased estimator is always better than a biased one." Total error is bias squared plus variance. The divide-by-nn variance estimate had a smaller RMSE than the unbiased one, and many of the best modern methods (shrinkage, regularisation) deliberately accept a little bias for a large cut in variance.

Misconception

"The bootstrap makes small samples reliable." The bootstrap reuses the sample you have; it cannot reveal what the sample missed. Its intervals are approximations that improve with nn, and they are often too narrow for small samples.

Misconception

"A wider interval means a worse study." A wider interval means a less precise one, but an honest wide interval is far more useful than a falsely narrow one. Intervals that are too narrow, from shortcuts or broken assumptions, are the dangerous ones, because they invite overconfident decisions.

Exercises

Exercises

1. Estimation and confidence intervals

Warm-upHuman

From a random sample of 16 plots, a 95% confidence interval for the mean teff yield is (1.46,1.78)(1.46, 1.78) tonnes per hectare. Which statement is correct?

Choose one

2. Estimation and confidence intervals

CoreHuman

A food scientist in Valencia measures the vitamin C content of 9 oranges from one grove: mean 20 mg per 100 g, sample standard deviation 6. The 0.975-quantile of the tt distribution with 8 degrees of freedom is 2.306. What is the 95% tt interval for the grove's mean?

Choose one

3. Estimation and confidence intervals

Warm-upHuman

An estimator of a river's mean flow is biased upwards by 3 m³/s and has a standard error of 4 m³/s. What is its mean squared error, in (m³/s)²?

4. Estimation and confidence intervals

CoreHuman

A national statistics office wants to estimate the share of adults with a bank account to within ±3\pm 3 percentage points at 95% confidence, with no prior idea of the answer. Assuming simple random sampling and using z=1.96z = 1.96, what is the minimum sample size?

5. Estimation and confidence intervals

CoreHuman

In a trial in Nairobi, 150 patients receive a new formulation of a medicine and none has a serious allergic reaction. Using the rule of three, what is an approximate 95% upper bound for the rate of serious reactions, in percent?

6. Estimation and confidence intervals

CoreComputational

The starter computes the Wald interval for a proportion, which collapses to (0,0)(0, 0) when k=0k = 0. Rewrite proportion_interval(k, n, z=1.96) to return the Wilson interval from the lesson instead, as a tuple (lo, hi) clipped to [0,1][0, 1]:

p^+z22n1+z2n  ±  z1+z2np^(1−p^)n+z24n2.\frac{\hat p + \frac{z^2}{2n}}{1 + \frac{z^2}{n}} \;\pm\; \frac{z}{1 + \frac{z^2}{n}}\sqrt{\frac{\hat p(1 - \hat p)}{n} + \frac{z^2}{4n^2}}.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

7. Estimation and confidence intervals

StretchComputational

Write bootstrap_ci(data, stat, B, seed, level=0.95), a percentile bootstrap interval:

  • create rng = np.random.default_rng(seed);
  • B times, resample data with replacement (same size as data) and apply the function stat (for example np.median) to the resample;
  • return the (1 - level)/2 and (1 + level)/2 quantiles of the B bootstrap statistics as a tuple of two floats.

The tests use 400 normal values, for which the theory predicts an interval for the mean of about xˉ±1.96 s/n\bar x \pm 1.96\, s/\sqrt n.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

8. Estimation and confidence intervals

CoreHuman

In which of these situations would a plain percentile bootstrap (resampling individual observations) be unreliable? Select all that apply.

Choose all that apply

Further reading and resources