Skip to content
LumenaryAcademy
Frontier 80 min

Case studies

Five projects from public health, agriculture, sport, energy and public services on four continents — the question, the data, the method, what went right and what went wrong — with synthetic re-creations you can run.

Code tested · 6AI-draftedAwaiting review by a data scientist

Your guides: Chloe · The Statistician, Margaret · The Simulator, Gideon · The Sceptic

MotivationH

Learning from other people's projects

Engineers study bridges that fell down; doctors study cases that went wrong. Data science has fewer formal traditions of this kind, but it has a growing record of projects documented well enough to learn from: the question that was asked, the data that were used, the method, and what happened when the result met the world. This lesson works through five, from five fields and four continents. Each is described here in our own words from published sources, which are linked so you can read the originals, and each comes with a small synthetic re-creation in code that isolates the mechanism behind its success or failure. The re-creations are not models of the real systems, and their numbers are not estimates of the real ones.

Two cautions before starting. First, published case studies are not a random sample: dramatic failures and clear successes get written up, quiet muddles rarely do. Second, hindsight is cheap. Every team in these stories was competent and acting in good faith, often under pressure; the point is to recognise the traps in time, not to feel superior to people who fell into them.

For each case, ask the same five questions: What was the question? What were the data? What was the method? What went right? What went wrong?

Worked examplesH

Public health: nowcasting flu from searches (United States, 2008–2015)

Question. Official influenza surveillance in the United States counts the share of doctor visits for influenza-like illness, but the figures arrive one to two weeks late. Could searches for flu-related terms estimate this week's level now?

Data and method. Researchers at Google and the US Centers for Disease Control and Prevention took weekly counts of 50 million candidate search queries and, for each, measured how well its time series matched five years of official figures across US regions (Ginsberg and colleagues, 2009). The best-matching 45 queries were combined into one predictor, and a simple regression turned it into an estimate. Google Flu Trends launched in 2008 and published estimates for many countries.

What went right. It tracked the official series closely in the years it was fitted to, a day after the week ended, and it inspired a whole field of "digital epidemiology".

What went wrong. In spring 2009 a new pandemic strain arrived out of season, and the original model underestimated it badly; it was revised that September (Cook and colleagues, 2011). In the 2012–13 season it estimated more than double the official level, after overestimating for much of the preceding period (Lazer and colleagues, 2014). Lazer and colleagues identified two traps. Big data hubris: with 50 million candidates and a few hundred weeks of data, some queries match flu simply because they share its season, so selecting by correlation picks up winter rather than illness. Algorithm dynamics: the search engine itself kept changing (suggested searches, for example), and so did how people searched, partly because of news coverage of flu. They also showed that simple models using the official figures from earlier weeks did as well or better, and that combining both sources worked better than either. Google stopped publishing the estimates in 2015.

The re-creation builds seven synthetic seasons and 5,000 synthetic search terms: 4,950 that merely follow winter, and 50 that follow flu symptoms (and, in the seventh season, worried readers of the news). It selects the 45 terms best correlated with flu in the first five seasons, as the original method did, and compares them with a model built only on the symptom terms.

python
import numpy as np

# Seven synthetic flu seasons of 52 weeks (week 0 = early July). Seasons 1-5 train the model.
rng = np.random.default_rng(2009)
weeks = np.arange(52)
winter = np.exp(-0.5 * ((weeks - 30) / 6) ** 2)            # the shape of winter, every year
flu, media = [], []
for season in range(7):
    peak = 30 if season != 5 else 12                       # season 6: an off-season pandemic wave
    size = rng.uniform(2, 5) if season != 5 else 4.0
    flu.append(1.0 + size * np.exp(-0.5 * ((weeks - peak) / 4) ** 2))
    media.append(np.full(52, 2.0 if season == 6 else 0.0))  # season 7: heavy news coverage of flu
flu, media = np.concatenate(flu), np.concatenate(media)
season_shape = np.tile(winter, 7)

# 5,000 candidate search terms: 4,950 merely seasonal (winter sports, holidays, heating...)
# and 50 driven by flu symptoms -- and, in season 7, by worried people reading the news.
T = len(flu)
seasonal = rng.uniform(0.5, 2, (4950, 1)) * season_shape + rng.normal(0, 0.3, (4950, T))
symptom = rng.uniform(0.2, 0.5, (50, 1)) * (flu + media * flu * 0.5) + rng.normal(0, 0.4, (50, T))
terms = np.vstack([seasonal, symptom])
is_symptom = np.r_[np.zeros(4950, bool), np.ones(50, bool)]

train = slice(0, 5 * 52)
z = (terms - terms[:, train].mean(1, keepdims=True)) / terms[:, train].std(1, keepdims=True)
corr = z[:, train] @ ((flu[train] - flu[train].mean()) / flu[train].std()) / (5 * 52)
top = np.argsort(corr)[-45:]                                # the 45 best-correlated terms
print(f"of the 45 terms chosen by correlation, {is_symptom[top].sum()} are about flu symptoms")

def nowcast(chosen):
    x = z[chosen].mean(0)
    slope, intercept = np.polyfit(x[train], flu[train], 1)
    return intercept + slope * x

for name, chosen in [("45 best-correlated terms", top), ("50 symptom terms only", np.flatnonzero(is_symptom))]:
    est = nowcast(chosen)
    print(f"\n{name}:")
    for s, label in [(4, "season 5 (training)"), (5, "season 6 (off-season wave)"), (6, "season 7 (news-driven)")]:
        w = slice(52 * s, 52 * (s + 1))
        print(f"  {label:27} true peak {flu[w].max():.1f}% in week {flu[w].argmax():2d}, "
              f"estimated peak {est[w].max():.1f}% in week {est[w].argmax():2d}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.
0%4%8%season 6: off-season waveseason 7: news-driven searchestrue level45 best-correlated termssymptom terms only
Seasons 6 and 7 of the synthetic re-creation. The model built from the 45 best-correlated terms (dashed) predicts a winter peak that never came and misses the spring wave entirely; the symptom-based model (gold) follows the spring wave but doubles the real level when news coverage drives searches.

None of the 45 best-correlated terms was about symptoms: in the training seasons, winter itself predicts flu as well as any symptom does, and there are 99 times as many winter terms. That model fits the training season well, then predicts a winter peak in the off-season year and sees nothing of the real wave in week 12. The symptom model catches the off-season wave but, when news coverage sends healthy people searching, estimates a peak of 6.9% against a true 3.5%. The two failures are different, and both are versions of one lesson: a correlation learnt in one regime says nothing about why it held, so it cannot tell you when it will stop.

Public-health surveillance has since added sources that measure the population more directly. Testing wastewater for viral RNA, for example, was shown early in the COVID-19 pandemic to track reported prevalence in Dutch cities (Medema and colleagues, 2020) and is now used in many countries; it has its own biases (dilution by rain, households not connected to sewers), which is why it complements rather than replaces clinical surveillance.

Worked examplesH

Agriculture and climate: insuring herds from space (northern Kenya, since 2010)

Question. In the drylands of northern Kenya and southern Ethiopia, droughts can kill a large share of a pastoralist family's animals, their main asset. Ordinary insurance, which sends an assessor to verify each loss, is far too expensive across such distances. Could insurance pay out on an index measured from satellites instead?

Data and method. Researchers at the International Livestock Research Institute and partner universities matched years of household surveys of herd losses to the normalised difference vegetation index (NDVI), a satellite measure of how green the pasture is, and fitted a model predicting the average livestock mortality in each area from it (Chantarat and colleagues, 2013). Index-based livestock insurance launched in Marsabit in 2010: when predicted mortality in an area passes a threshold, every policyholder there is paid, with no farm visits. The approach was later extended in Ethiopia and adopted by Kenya's government for a national programme.

What went right. Payouts can be computed from public satellite data within weeks, cheaply and without disputes about individual claims, and they arrive when a whole area is in trouble, which is when families most need cash to keep animals alive.

What went wrong, or rather what remains hard. A household's losses are only partly explained by its area's. Disease, theft, a bad choice of grazing route or simple luck strike families individually. The gap between what the index pays and what a household lost is called basis risk. A study of eight seasons of household data found that the insurance removed much of the area-wide risk, but that policyholders were still left with most of their exposure to large losses (Jensen, Barrett and Mude, 2016). Uptake has been modest, and basis risk is part of the reason: paying a premium and then receiving nothing after a heavy loss erodes trust. The re-creation shows why basis risk is inevitable when losses have a large household component.

python
import numpy as np

# Synthetic herders: 10 areas x 100 households x 16 seasons. Losses = area drought + household luck.
rng = np.random.default_rng(14)
areas, homes, seasons = 10, 100, 16
ndvi = rng.normal(0, 1, (areas, seasons))                         # vegetation anomaly (negative = dry)
area_loss = np.clip(0.10 - 0.11 * ndvi + rng.normal(0, 0.04, ndvi.shape), 0, 0.9)
own = rng.beta(0.5, 7, (areas, homes, seasons))                    # disease, theft, bad luck
loss = np.clip(area_loss[:, None, :] + own - own.mean(), 0, 1)     # share of the herd lost

# The index: predicted area loss from NDVI, fitted on the first 8 seasons; pay above a 15% strike.
fit = slice(0, 8)
slope, intercept = np.polyfit(ndvi[:, fit].ravel(), loss[:, :, fit].mean(1).ravel(), 1)
index = np.clip(intercept + slope * ndvi, 0, 1)
payout = np.maximum(index - 0.15, 0)                               # share of herd value paid
paid = np.broadcast_to(payout[:, None, :], loss.shape)
each = np.broadcast_to(index[:, None, :], loss.shape)              # the index each household sees

test = (slice(None), slice(None), slice(8, None))
big = loss[test] > 0.30
print(f"index vs area-average loss, correlation: {np.corrcoef(index[:, 8:].ravel(), loss[test].mean(1).ravel())[0, 1]:.2f}")
print(f"index vs household loss, correlation:    {np.corrcoef(each[test].ravel(), loss[test].ravel())[0, 1]:.2f}")
print(f"household-seasons with a loss above 30%: {big.mean():.1%}")
print(f"  of those, the share that received no payout: {(paid[test][big] == 0).mean():.1%}")
print(f"payouts made to households that lost under 10%: {((paid[test] > 0) & (loss[test] < 0.10)).sum()} "
      f"of {(paid[test] > 0).sum()} paid household-seasons")
excess = np.maximum(loss[test] - 0.15, 0)
left = np.maximum(excess - paid[test], 0)
print(f"losses above 15% of the herd still uncovered after payouts: {left.sum() / excess.sum():.0%}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The index is a good model of what it was built to predict: it correlates at 0.91 with area-average losses in seasons it was not fitted on. Measured against what households actually lost, the correlation falls to 0.67. Of household-seasons with a loss above 30% of the herd, 22.7% received nothing, while 378 of 2,800 payouts went to households that had lost less than 10%. And 59% of losses above the 15% threshold remained uncovered.

Worked examplesH

Sport: expected goals and the clinical finisher (Europe and beyond)

Question. How good is a striker at finishing, as opposed to getting into good positions? Goals alone mix the two, along with a great deal of luck.

Data and method. Expected goals (xG) models estimate the probability that a shot becomes a goal from its characteristics (distance, angle, body part, the kind of pass before it), fitted on tens of thousands of past shots, usually by logistic regression or gradient-boosted trees. Academic work on the probability of scoring from different situations goes back at least to the 1990s; in the 2010s, event data recorded for every professional match made xG standard in clubs and broadcasts. You can build one yourself from openly published event data, such as the data set of seven competitions released by Pappalardo and colleagues. A player's total xG says how many goals an average finisher would have scored from his or her chances; "goals minus xG" is then read as finishing skill.

What went right. xG describes the quality of chances far better than shot counts or goals, and because chance quality is more stable than finishing luck, it is widely used to judge whether a team's results are likely to last. It changed how clubs scout, how analysts talk about matches, and how coaches judge whether a result was deserved.

What went wrong. The most common misuse is reading one season's goals minus xG as a stable trait. Over roughly 70 shots, the luck in whether each one goes in is large compared with real differences in finishing, so the "most clinical finishers" of one season are mostly the luckiest, and such numbers are easily over-read in scouting and in the media. The re-creation gives 200 synthetic strikers a true finishing skill, modest differences between them, and two seasons of shots.

python
import numpy as np

# Synthetic strikers: 200 players, two seasons, about 70 shots each per season.
rng = np.random.default_rng(10)
players = 200
skill = rng.normal(0, 0.10, players)               # true finishing skill, on the log-odds scale

def season():
    goals, xg, edge = np.zeros(players), np.zeros(players), np.zeros(players)
    for i in range(players):
        distance = rng.uniform(5, 30, rng.poisson(70))            # metres from goal, per shot
        p = 1 / (1 + np.exp(-(1.2 - 0.17 * distance)))           # average finisher: the shot's xG
        p_i = 1 / (1 + np.exp(-(1.2 - 0.17 * distance + skill[i])))   # this player
        goals[i], xg[i], edge[i] = (rng.random(p.size) < p_i).sum(), p.sum(), (p_i - p).sum()
    return goals, xg, edge

g1, x1, e1 = season()
g2, x2, e2 = season()
over1, over2 = g1 - x1, g2 - x2                     # goals minus expected goals
print(f"mean per player per season: {x1.mean():.1f} xG, {g1.mean():.1f} goals")
print(f"correlation of (goals - xG) between the two seasons: {np.corrcoef(over1, over2)[0, 1]:.2f}")
best = np.argsort(over1)[-10:]
print(f"top 10 'clinical finishers' in season 1: {over1[best].mean():+.1f} goals above xG each")
print(f"the same 10 players in season 2:         {over2[best].mean():+.1f}")
print(f"what their true skill is worth:          {e2[best].mean():+.1f} per season")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

This is regression to the mean, and it appears wherever individuals are ranked on noisy measurements: hospitals by mortality, schools by one year's results, fund managers by one year's returns, sales staff by one quarter. The remedy is the same everywhere: estimate how much of the spread is noise (as here, from the correlation between periods), shrink individual estimates towards the average accordingly (the idea behind multilevel models), and report uncertainty with every ranking.

Worked examplesH

Energy: forecasting demand when behaviour changes overnight (France and India, 2020)

Question. Grid operators must balance supply and demand second by second, so they forecast electricity demand days, hours and minutes ahead, and schedule generation accordingly.

Data and method. Demand forecasting is one of the most mature applications of statistical modelling. Models combine temperature (heating and cooling), cloud cover, the calendar (time of day, weekday, holidays) and recent demand, using regression, generalised additive models and, increasingly, gradient boosting and neural networks. In stable conditions day-ahead national forecasts are typically accurate to within a few per cent.

France: lockdown. France's first lockdown began on 17 March 2020. The main forecasting model at the national electricity company, a generalised additive model, had been accurate to about 1% (mean absolute percentage error) before the lockdown and was off by about 5% in its first weeks, so experts had to correct forecasts by hand. Researchers there built adaptive versions whose effects update continuously from recent errors (with Kalman filters), borrowed information from Italy's lockdown, which had started earlier, and combined several models; the errors fell close to pre-pandemic levels (Obst, de Vilmarest and Goude, 2020). This is concept drift, met with the monitoring and adaptation described in data science in production.

India: nine minutes. On 5 April 2020 India's prime minister asked people to switch off their lights for nine minutes at 9 p.m. No historical data could predict the result. India's grid operator (then POSOCO) planned for a fall in demand of about 12 to 14 GW and arranged for hydro and gas plants to absorb the swing. The actual fall was about 31 GW, more than twice the plan, because many people switched off far more than lights; the grid held, with frequency kept within its normal band, because the plan had margins and flexible plants (IEEFA summary of the operator's report).

What went right and wrong. In both cases the models failed exactly as one should expect when behaviour changes abruptly, and in both cases the system coped: monitoring detected the error, people with domain knowledge intervened, and planning used scenarios and safety margins rather than a single point forecast. The lesson is less about forecasting than about operating a forecast: know when it is out of its depth, and design the decisions around it to survive being wrong.

Worked examplesH

Public services: grading without exams (England, 2020)

Question. When the COVID-19 pandemic cancelled A-level exams in England in 2020, the exam regulator, Ofqual, had to award grades that universities and employers could rely on, without exams and without the large grade inflation that teachers' optimistic estimates would cause.

Data and method. Teachers submitted a centre assessment grade for each student and a rank order of students within each subject. The regulator's standardisation model predicted each school's grade distribution in a subject from its results in previous years, adjusted for the prior attainment of this year's students, and then assigned grades by mapping the teachers' rank order onto that distribution. For small classes, where statistics are unreliable, teachers' grades were used as they were, with a sliding mix up to classes of about 15 (Ofqual's interim report).

What went right. In aggregate, the model did what it was designed to do: the national distribution of grades stayed close to previous years, where using teachers' grades would have raised the share of A and A* grades to about 38% instead of the 27.6% awarded (Ofqual).

What went wrong. About 39% of teachers' grades were lowered, most by one grade and some by two or more. A strong student in a school whose past results were weak could not receive a grade the school had rarely achieved, whatever their teachers expected. Small classes, more common in independent schools, received their teachers' more generous grades, so the method favoured them systematically. Routes of appeal against a statistical prediction were limited. After a public outcry the decision was reversed within days, and students were given their teachers' grades (Scotland had reversed a similar approach the week before). The re-creation is much simplified, but it reproduces the mechanism.

python
import numpy as np

# A simplified re-creation of grading without exams. Grades 0..6 stand for U, E, D, C, B, A, A*.
rng = np.random.default_rng(2020)
GRADES = ["U", "E", "D", "C", "B", "A", "A*"]

def exam_grade(ability):                     # the grade an exam would have given
    return np.clip(np.round(ability + rng.normal(0, 0.7, ability.size)), 0, 6).astype(int)

rows = []                                    # (cohort size, school level, exam, teacher, awarded)
for school in range(400):
    small = rng.random() < 0.3
    n = int(rng.integers(3, 13)) if small else int(rng.integers(18, 40))
    level = rng.normal(3.6 if small else 3.0, 0.7)          # the school's typical attainment
    history = np.concatenate([exam_grade(rng.normal(level, 1.2, n)) for _ in range(3)])
    ability = rng.normal(level, 1.2, n)
    exam = exam_grade(ability)
    teacher = np.clip(np.round(ability + 0.4 + rng.normal(0, 0.5, n)), 0, 6).astype(int)
    rank = np.argsort(-(ability + rng.normal(0, 0.4, n)))    # teachers' rank order, best first
    # Statistical grade: give the k-th ranked student the grade at the same position in the
    # school's historical distribution of grades.
    hist_sorted = np.sort(history)[::-1]
    stat = np.empty(n, int)
    stat[rank] = hist_sorted[((np.arange(n) + 0.5) / n * hist_sorted.size).astype(int)]
    w = 1.0 if n <= 5 else (0.0 if n >= 15 else (15 - n) / 10)   # weight on the teacher's grade
    awarded = np.round(w * teacher + (1 - w) * stat).astype(int)
    rows += [(n, level, e, t, a) for e, t, a in zip(exam, teacher, awarded)]

n, level, exam, teacher, awarded = map(np.array, zip(*rows))
top = lambda g: (g >= 5).mean()
print(f"students: {len(exam)}; share given A or A*: exam {top(exam):.1%}, "
      f"teacher {top(teacher):.1%}, algorithm {top(awarded):.1%}")
down = awarded < teacher
print(f"teacher grades lowered: {down.mean():.1%} (by two or more grades: {(teacher - awarded >= 2).mean():.1%})")
for label, mask in [("classes of 15 or more", n >= 15), ("classes of 5 or fewer", n <= 5)]:
    print(f"  {label:22} teacher grade lowered {down[mask].mean():5.1%}; compared with the exam grade: "
          f"higher {(awarded > exam)[mask].mean():5.1%}, lower {(awarded < exam)[mask].mean():5.1%}")
weak = (level < 2.5) & (n >= 15) & (exam >= 5)
print(f"A/A* students (by exam) in historically weak large schools: {weak.sum()}; "
      f"given B or lower: {(awarded[weak] <= 4).mean():.0%}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Here we can do what nobody could in 2020: compare with the grade a (synthetic) exam would have given. Nationally the algorithm looks excellent: it awards A or A* to 18.4% of students, against 18.1% from the exam and 24.4% from teachers. For individuals it is another story. In large classes it disagrees with the exam for well over half the students, upwards for 29.2% and downwards for 28.7%, because a school's past says little about which of this year's students will do well. Small classes, which kept their teachers' grades, were raised above their exam grade for 47.1% of students. And of the 60 students in historically weak large schools who would have earned an A or A* in the exam, 48% were given B or lower.

ExplanationH

What the cases have in common

CaseKind of questionWhat went wrong (or remains hard)General lesson
Flu from searchesPredictive (nowcast)Features chosen by correlation tracked the season; the data-generating process kept changingChoose features for a reason; test in the conditions that matter; combine with direct measurement
Livestock insurancePredictive, feeding a payout ruleAn area index cannot see household losses (basis risk)Evaluate where the errors are felt; be candid about what is not covered
Expected goalsEvaluating individualsSmall samples read as stable skillSeparate noise from signal; expect regression to the mean
Electricity demandPredictiveAbrupt changes in behaviour broke historical patternsMonitor, adapt, keep people in the loop, plan with margins
Grading without examsAllocating to individualsAccurate in aggregate, unfair to individuals; little room to appealDesign for the people affected; consult them; provide redress

Three patterns recur. Correlations learnt from the past are fragile when the world changes, whether the change is a new virus, a lockdown or a news cycle; the systems that coped had monitoring, domain experts and margins. The level of evaluation matters: area averages versus households, national distributions versus students, a season versus a career. A model can be excellent at the level its builders checked and poor at the level where people live. And legitimacy is part of the result: the projects that kept public trust explained themselves, measured their own errors openly, and gave people a way to object, the themes of responsible data science.

Many other documented projects reward the same five questions, in every field and country. When you read one, look for the evaluation that was not done, and ask who would have noticed first if it had gone wrong.

Common misconceptionsH

Common misconceptions

Misconception

"More data would have fixed Google Flu Trends." It had enormous amounts of data. The problem was searching 50 million candidates against a few hundred weeks of outcomes, which guarantees spurious matches, and then trusting them outside the conditions in which they were found.

Misconception

"The index insurance model was inaccurate." It predicted area losses well. Basis risk comes from losses that are not shared across an area, which no area index can see. Better models help at the margin; the design (and honesty about what is covered) matters more.

Misconception

"The grading algorithm failed because it was a bad statistical model." By its own target, national grade distributions, it did well. It failed because that target was not the question students, families and teachers were asking, and because a prediction about a school was applied to individuals with little chance of appeal.

Misconception

"A striker who beat xG by six goals is a proven finisher." Over one season, mostly luck. Only consistent over-performance across several seasons, shrunk towards the average, is evidence of skill.

Exercises

Exercises

1. Case studies

Warm-upHuman

In the lesson's re-creation, the model built from the 45 search terms best correlated with flu predicted a winter peak in the off-season year and missed the real spring wave. Why?

Choose one

2. Case studies

CoreHuman

A striker scored 25 goals from 15 xG this season. In this league the correlation of (goals − xG) between one season and the next is 0.2, and the average of (goals − xG) across strikers is 0. Assuming a similar number of chances next season, what is the best simple prediction of next season's goals − xG?

3. Case studies

CoreComputational

Write basis_risk(loss, index, strike=0.15, big=0.30) for an index insurance scheme. loss gives each household-season's share of the herd lost and index the area index that applied to it; each pays max(index - strike, 0). Return a dict with

  • "missed": the share of household-seasons with loss > big that received no payout;
  • "uncovered": the share of the total loss above the strike, ∑max⁡(loss−strike,0)\sum \max(\text{loss} - \text{strike}, 0), that remains after subtracting each household-season's payout (never below zero for any household-season).

Accept lists or NumPy arrays.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

4. Case studies

CoreComputational

Implement the rank-based step of the lesson's grading re-creation. allocate(history, n) takes a school's historical grades (integers, higher is better) and returns grades for n students ranked best first: the k-th ranked student (k = 0, 1, …, n − 1) gets the grade at position int((k + 0.5) / n * len(history)) of the historical grades sorted from best to worst. Return a list of n integers.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

5. Case studies

CoreHuman

Before the 2020 grades were published, which check would have been most likely to reveal the problem that caused the outcry?

Choose one

6. Case studies

CoreHuman

On 5 April 2020 India's demand fell by about 31 GW in minutes, more than twice the 12–14 GW the grid operator had planned for, yet the grid held. Which of these explain why? Select all that apply.

Choose all that apply

Further reading and resources