The Lumenary · Early undergraduate

Statistical inference

Name: ______________________

Date: ______________________

You will learn to

Define estimators and assess their bias, variance and mean squared error; prove that the sample mean and the sample variance with $n - 1$ are unbiased
Write down likelihoods and derive maximum likelihood estimators, and state the Cramér–Rao bound
Construct confidence intervals and perform hypothesis tests, interpreting confidence levels and p-values correctly
Explain type I and type II errors and power, and why multiple testing needs a correction such as Bonferroni's
Fit linear regression by least squares through the normal equations $X^\top X\beta = X^\top y$ , and check its assumptions with residuals
Compare frequentist and Bayesian inference through the beta–binomial model

1.
A test of $H_0$ : "the new fertiliser has no effect on mean yield" gives $p = 0.03$ . Which statement is correct?
- A.There is a $3\%$ probability that the fertiliser has no effect.
- B.If the fertiliser had no effect, a difference at least as large as the one observed would occur in about $3\%$ of such experiments.
- C.The fertiliser increases the yield by $3\%$ .
- D.If the experiment is repeated, it will give a significant result $97\%$ of the time.
2.
A health clinic records the number of patients arriving in five consecutive hours: $3, 5, 2, 6, 4$ . Modelling the counts as independent Poisson $(\lambda)$ , what is the maximum likelihood estimate of $\lambda$ ?
3.
Compute the sample variance $S^2 = \frac{1}{n-1}\sum_i (x_i - \bar x)^2$ of the data $2, 4, 9$ .
4.
For an i.i.d. sample $X_1, \dots, X_n$ from the uniform distribution on $[0, \theta]$ , the MLE is $M = \max_i X_i$ , and $E[M] = \frac{n}{n+1}\theta$ . Which estimator is unbiased for $\theta$ ?
- A. $M$
- B. $\frac{n+1}{n}M$
- C. $\frac{n}{n+1}M$
- D. $M + 1$
5.
In the proof that the normal equations give the least-squares solution, we expand $\|y - X\beta\|^2 = \|r\|^2 + 2\,r^\top Xw + \|Xw\|^2$ with $r = y - X\hat\beta$ and $w = \hat\beta - \beta$ . Why does the cross term vanish?
- A. $r^\top Xw = (X^\top r)^\top w$ and $X^\top r = X^\top y - X^\top X\hat\beta = 0$ by the normal equations.
- B.Because $w$ is small when $\beta$ is close to $\hat\beta$ .
- C.Because the residuals always sum to zero.
- D.Because $X^\top X$ is invertible.
6.
A study tests $8$ hypotheses and obtains the p-values $0.0004,\ 0.003,\ 0.006,\ 0.0071,\ 0.011,\ 0.02,\ 0.04,\ 0.3$ . Using the Bonferroni correction with family-wise error rate $0.05$ , how many hypotheses are rejected?
7.
A small trial of a new treatment (20 patients) finds a $95\%$ confidence interval of $[-2, 15]$ days for the reduction in recovery time, with $p = 0.13$ . What is the best conclusion?
- A.The treatment has no effect.
- B.The data are compatible with no effect and also with a substantial benefit; a larger study is needed.
- C.The treatment is harmful, since the interval contains negative values.
- D.There is an $87\%$ chance that the treatment works.
8.
From $260$ of $400$ households, a $95\%$ confidence interval for the proportion $p$ is $[0.603, 0.697]$ , and with a uniform prior the Bayesian posterior is Beta $(261, 141)$ with $95\%$ credible interval $[0.602, 0.695]$ . Which statements are correct? (Choose all that apply.)
- A.In repeated surveys, about $95\%$ of intervals computed this way would contain the true $p$ .
- B.Frequentist theory says $P(0.603 \le p \le 0.697) = 0.95$ for this particular interval.
- C.Given the model and the uniform prior, $P(0.602 \le p \le 0.695 \mid \text{data}) = 0.95$ .
- D.The posterior mean is $261/402 \approx 0.649$ .
9.
Which Lean statement says that a solution b of the normal equations minimises the residual sum of squares? (Xᵀ is the transpose, *ᵥ multiplies a matrix by a vector, ⬝ᵥ is the dot product.)
- A.(hb : (Xᵀ * X) *ᵥ b = Xᵀ *ᵥ y) (β : p → ℝ) : (y - X *ᵥ b) ⬝ᵥ (y - X *ᵥ b) ≤ (y - X *ᵥ β) ⬝ᵥ (y - X *ᵥ β)
- B.(hb : X *ᵥ b = y) (β : p → ℝ) : (y - X *ᵥ b) ⬝ᵥ (y - X *ᵥ b) ≤ (y - X *ᵥ β) ⬝ᵥ (y - X *ᵥ β)
- C.(hb : (Xᵀ * X) *ᵥ b = Xᵀ *ᵥ y) (β : p → ℝ) : (y - X *ᵥ β) ⬝ᵥ (y - X *ᵥ β) ≤ (y - X *ᵥ b) ⬝ᵥ (y - X *ᵥ b)
- D.(hb : (X * Xᵀ) *ᵥ b = Xᵀ *ᵥ y) (β : p → ℝ) : (y - X *ᵥ b) ⬝ᵥ (y - X *ᵥ b) ≤ (y - X *ᵥ β) ⬝ᵥ (y - X *ᵥ β)
10.
Write fit_line(xs, ys) returning the least-squares (intercept, slope) for the model $y = \beta_0 + \beta_1 x$ , using $\hat\beta_1 = \frac{\sum (x_i - \bar x)(y_i - \bar y)}{\sum (x_i - \bar x)^2}$ and $\hat\beta_0 = \bar y - \hat\beta_1\bar x$ . Use Fraction so the answers are exact. The tests also check that the residuals sum to zero and are orthogonal to $x$ — the normal equations.
Computational — Practice

Answer key — Statistical inference

1.
The p-value is $P_{H_0}(T \ge t_{\text{obs}})$ : assuming no effect, how often would data at least this extreme arise? It is not $P(H_0 \mid \text{data})$ , not an effect size, and not a replication probability.
2.
$\ell'(\lambda) = \frac{\sum x_i}{\lambda} - n$ , which is zero at $\hat\lambda = \bar x = 20/5 = 4$ , and $\ell'' = -\sum x_i/\lambda^2 < 0$ , so this is the maximum.
3.
$\bar x = 5$ ; the squared deviations are $9, 1, 16$ , with sum $26$ ; $S^2 = 26/2 = 13$ . Dividing by $n = 3$ instead would give $26/3 \approx 8.7$ , which underestimates $\sigma^2$ on average (Bessel's correction).
4.
Since $E[cM] = c\frac{n}{n+1}\theta$ , the choice $c = \frac{n+1}{n}$ removes the bias. (Note also $2\bar X$ is unbiased, but $\frac{n+1}{n}M$ has much smaller variance, $\frac{\theta^2}{n(n+2)}$ against $\frac{\theta^2}{3n}$ . The Cramér–Rao bound does not apply here, because the support $[0, \theta]$ depends on $\theta$ .)
5.
$r^\top Xw = (X^\top r)^\top w$ , and $X^\top r = X^\top y - X^\top X\hat\beta = 0$ . Geometrically, the residual is perpendicular to the column space of $X$ , so $\|y - X\beta\|^2 = \|r\|^2 + \|Xw\|^2$ is Pythagoras' theorem.
6.
The threshold is $0.05/8 = 0.00625$ , and only $0.0004$ , $0.003$ and $0.006$ are below it: $3$ rejections. For comparison, the Benjamini–Hochberg thresholds $k \cdot 0.05/8$ are $0.00625, 0.0125, 0.01875, 0.025, 0.03125, 0.0375, 0.04375, 0.05$ ; the sorted p-values stay below them up to $k = 7$ , so BH rejects $7$ hypotheses while controlling the false discovery rate instead of the family-wise error rate.
7.
The confidence interval $[-2, 15]$ contains $0$ (so $p > 0.05$ ) but also large benefits. The trial is underpowered: it neither establishes an effect nor rules out an important one. Report the interval, and plan a larger study.
8.
The confidence level describes the procedure over repeated samples; the credible interval is a posterior probability statement. With a uniform prior the posterior is Beta $(261, 141)$ , with mean $261/402 \approx 0.649$ . The two intervals nearly coincide here because $n$ is large (Bernstein–von Mises), but they answer different questions.
9.
Statement (a). (b) has the wrong hypothesis (an exact fit), (c) reverses the inequality, and (d) uses $XX^\top$ , which does not even typecheck in general. The lesson's Lean module also proves the converse, minimizer_normal_equations.

10.

from fractions import Fraction

def fit_line(xs, ys):
    xs = [Fraction(x) for x in xs]
    ys = [Fraction(y) for y in ys]
    n = len(xs)
    mx, my = sum(xs) / n, sum(ys) / n
    sxy = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
    sxx = sum((x - mx) ** 2 for x in xs)
    slope = sxy / sxx
    return my - slope * mx, slope

The two tested properties are the normal equations: $\sum_i r_i = 0$ (orthogonality to the column of ones) and $\sum_i x_i r_i = 0$ (orthogonality to the column of $x$ 's).