Skip to content
LumenaryAcademy
Advanced 75 min

Time series

Data in order — trend, seasonality and noise; autocorrelation and stationarity; AR, MA and ARIMA models; exponential smoothing; and how to judge a forecast honestly.

Code tested · 6AI-draftedAwaiting review by a statistician

Your guides: Chloe · The Statistician, Margaret · The Simulator, Gideon · The Sceptic

MotivationH

When the order of the data matters

A grid operator must decide, a day ahead, how much generation to schedule for every hour. Schedule too little and the frequency sags; schedule too much and expensive plants burn fuel for nothing. The operator's main tool is a forecast built from years of past demand, and the past demand is a time series: measurements of one quantity, taken in order, at regular intervals.

Time series are everywhere a quantity is tracked: monthly unemployment, daily hospital admissions, weekly sales of school uniforms, the carbon dioxide concentration measured at Mauna Loa since 1958, river levels, sunspot counts. They differ from the data of most earlier lessons in one decisive way. Earlier, we treated observations as independent draws, so shuffling the rows changed nothing. In a time series, order is information. This month's demand is close to last month's; a cold January is followed by a fairly cold February. That dependence is what makes forecasting possible, and it is also what breaks methods that assume independence: standard errors come out too small, two unrelated trending series appear strongly related, and a model tested on randomly chosen rows quietly peeks at the future.

This lesson covers the classic toolkit: decomposing a series into trend, seasonality and noise; measuring dependence with autocorrelation; the idea of stationarity; the AR, MA and ARIMA models of Box and Jenkins; exponential smoothing; and the part most often done badly, evaluating forecasts honestly.

IntuitionH

Trend, seasonality and noise

Most series that people forecast have three visible ingredients. The trend is the slow movement of the level: demand growing with population, a disease declining with vaccination. The seasonal component repeats with a fixed period: electricity peaks in winter in Warsaw and in summer in Riyadh; retail sales peak before festivals; hospital admissions follow the week. The remainder is whatever is left. An additive decomposition writes yt=Tt+St+Rt,y_t = T_t + S_t + R_t, and suits series whose seasonal swings stay the same size as the level changes. When the swings grow in proportion to the level, as airline passenger numbers or sales of a growing product often do, a multiplicative decomposition yt=Tt×St×Rty_t = T_t \times S_t \times R_t fits better, and taking logarithms turns it into an additive one.

The classical way to estimate the parts is simple. A moving average spanning exactly one season smooths the seasonal pattern away and leaves the trend. Subtracting the trend leaves seasonality plus noise, and averaging that by calendar month gives the seasonal indices. The code below does this for eight years of synthetic monthly electricity demand for a mid-sized city, built so that we know the true components.

python
import numpy as np

rng = np.random.default_rng(42)
n = 96                                                   # eight years of months, January 2017 onwards
t = np.arange(n)
trend = 300 + 1.2 * t                                    # GWh per month, slowly growing
season = 30 * np.cos(2 * np.pi * t / 12) + 12 * np.cos(4 * np.pi * t / 12)   # winter peak, small summer bump
noise = np.zeros(n)
for i in range(1, n):                                    # AR(1) noise: today's surprise lingers
    noise[i] = 0.5 * noise[i - 1] + rng.normal(0, 8)
y = trend + season + noise

# Classical additive decomposition.
# 1. Trend: a centred 2x12 moving average (weights 1/24, 1/12, ..., 1/12, 1/24).
w = np.r_[0.5, np.ones(11), 0.5] / 12
trend_hat = np.full(n, np.nan)
trend_hat[6:-6] = np.convolve(y, w, mode="valid")
# 2. Seasonal indices: average detrended value for each calendar month, centred to sum to zero.
detrended = y - trend_hat
idx = np.array([np.nanmean(detrended[m::12]) for m in range(12)])
idx -= idx.mean()
# 3. Remainder.
remainder = y - trend_hat - np.tile(idx, n // 12)

months = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
print("month   estimated index   true seasonal effect")
for m in range(12):
    print(f" {months[m]}        {idx[m]:+6.1f}            {season[m]:+6.1f}")
r = remainder[~np.isnan(remainder)]
print(f"remainder: {r.size} months, standard deviation {r.std():.1f} GWh")

def acf(x, max_lag):
    x = x - x.mean()
    return np.array([np.sum(x[k:] * x[:len(x) - k]) / np.sum(x * x) for k in range(1, max_lag + 1)])

print("autocorrelation of the remainder, lags 1-6:", np.round(acf(r, 6), 2),
      f"  (white-noise band: +/-{1.96 / np.sqrt(r.size):.2f})")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The estimated indices track the truth to within a few GWh: January is 43.8 above the trend (truth 42.0), September 22.0 below (truth 21.0). Two details deserve attention. The moving average needs six months on each side, so the trend is missing for the first and last half-year, exactly where a forecaster most wants it; production methods handle the ends with forecasts or local regression. And the remainder is not pure noise: its lag-1 autocorrelation of 0.38 is outside the white-noise band of ±0.21, because we built the noise so that each month's surprise carries into the next. That leftover dependence is what the models later in the lesson capture.

25030035040045020172018201920202021202220232024demand (GWh per month) and its 2×12 moving-average trendJFMAMJJASOND-20+20+40estimated seasonal effect of each month (GWh)
The synthetic electricity series with its 2×12 moving-average trend (top), and the twelve estimated seasonal indices (bottom). The trend line stops half a year short of each end, where the centred average runs out of data.
DefinitionsH

Autocorrelation and stationarity

Definition— autocorrelation function

For a series y1,…,yny_1, \ldots, y_n with mean yˉ\bar y, the sample autocorrelation at lag kk is rk=∑t=k+1n(yt−yˉ)(yt−k−yˉ)∑t=1n(yt−yˉ)2,r_k = \frac{\sum_{t=k+1}^{n} (y_t - \bar y)(y_{t-k} - \bar y)}{\sum_{t=1}^{n} (y_t - \bar y)^2}, the correlation of the series with itself shifted by kk steps. Plotted against kk it is the autocorrelation function (ACF), or correlogram.

If the series is white noise (independent, identically distributed values), each rkr_k is approximately normal with mean 0 and standard deviation 1/n1/\sqrt n, so about 95% of lags fall within ±1.96/n\pm 1.96/\sqrt n. Spikes outside the band are evidence of dependence, though with 20 lags plotted, one spike outside the band is expected by chance alone.

Definition— weak stationarity

A series is (weakly) stationary if its mean E[yt]=μE[y_t] = \mu and variance are the same at every time, and its autocovariance Cov(yt,yt−k)=γk\text{Cov}(y_t, y_{t-k}) = \gamma_k depends only on the lag kk, not on tt.

Stationarity is what lets us learn from the past: if the rules of dependence stayed the same, last decade's data describe next year. A series with a trend or a seasonal pattern is not stationary (its mean changes with time), and neither is a random walk, yt=yt−1+εty_t = y_{t-1} + \varepsilon_t, whose variance grows without bound. A random walk can be made stationary by differencing, ∇yt=yt−yt−1=εt\nabla y_t = y_t - y_{t-1} = \varepsilon_t; seasonal patterns by seasonal differencing, yt−yt−12y_t - y_{t-12}. A series that becomes stationary after dd differences is said to be integrated of order dd. Formal unit-root tests (the augmented Dickey–Fuller and KPSS tests) help decide how many differences are needed.

python
import numpy as np

rng = np.random.default_rng(1974)

def acf(x, max_lag):
    x = x - x.mean()
    return np.array([np.sum(x[k:] * x[:len(x) - k]) / np.sum(x * x) for k in range(1, max_lag + 1)])

n = 200
e = rng.normal(0, 1, n)
walk = np.cumsum(e)                                      # random walk: y_t = y_{t-1} + e_t
ar = np.zeros(n)
for i in range(1, n):
    ar[i] = 0.7 * ar[i - 1] + e[i]                       # stationary AR(1)
print("lags:                1     2     3     4     5    10")
for name, x in [("white noise", e), ("AR(1), phi = 0.7", ar), ("random walk", walk), ("differenced walk", np.diff(walk))]:
    r = acf(x, 10)
    print(f"{name:18}" + "".join(f"{v:6.2f}" for v in r[[0, 1, 2, 3, 4, 9]]))

# Spurious regression: regress one random walk on another, completely independent one.
def slope_t(x, y):
    X = np.column_stack([np.ones(len(x)), x])
    coef, *_ = np.linalg.lstsq(X, y, rcond=None)
    resid = y - X @ coef
    s2 = resid @ resid / (len(y) - 2)
    return coef[1] / np.sqrt(s2 * np.linalg.inv(X.T @ X)[1, 1])

reps = 2000
t_levels, t_diffs = np.empty(reps), np.empty(reps)
for r in range(reps):
    a, b = np.cumsum(rng.normal(size=n)), np.cumsum(rng.normal(size=n))
    t_levels[r], t_diffs[r] = slope_t(a, b), slope_t(np.diff(a), np.diff(b))
print(f"independent random walks: 'significant' slope (|t| > 1.97) in {np.mean(np.abs(t_levels) > 1.97):.0%} of {reps} regressions")
print(f"their differences:        'significant' slope in {np.mean(np.abs(t_diffs) > 1.97):.0%}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.
white noiselag 1 to 121.00.5AR(1), φ = 0.7lag 1 to 12random walklag 1 to 12
Sample autocorrelations at lags 1 to 12 for three simulated series of length 200 (the first three rows printed above). The dashed lines are the white-noise band ±1.96/√200 ≈ ±0.14. Stationary dependence decays quickly; a random walk's autocorrelations decay so slowly that the plot is itself a warning sign.

The second half of the program is one of the most important experiments in applied statistics. Two random walks generated independently have no relationship, yet regressing one on the other gives a "significant" slope at the 5% level in 83% of 2,000 tries. Clive Granger and Paul Newbold called this spurious regression in 1974. The ordinary standard errors assume independent residuals; with two wandering series, the residuals wander too, and the effective sample size is far smaller than 200. Regress the differences instead and the false-positive rate returns to the advertised 5%. This is why a correlation between two trending national statistics, however strong, is weak evidence of anything.

DefinitionsH

AR, MA and ARIMA models

Definition— autoregressive and moving-average models

With εt\varepsilon_t white noise of variance σ2\sigma^2:

  • an autoregressive model of order pp, AR(pp), is yt=c+ϕ1yt−1+⋯+ϕpyt−p+εty_t = c + \phi_1 y_{t-1} + \cdots + \phi_p y_{t-p} + \varepsilon_t: today is a weighted sum of recent values plus a fresh shock;
  • a moving-average model of order qq, MA(qq), is yt=μ+εt+θ1εt−1+⋯+θqεt−qy_t = \mu + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \cdots + \theta_q \varepsilon_{t-q}: today is a weighted sum of recent shocks;
  • ARMA(p,qp, q) combines both, and ARIMA(p,d,qp, d, q) is an ARMA(p,qp, q) model for the series differenced dd times.

Theorem— The AR(1) process

If yt=c+ϕyt−1+εty_t = c + \phi y_{t-1} + \varepsilon_t with ∣ϕ∣<1|\phi| \lt 1, the stationary process has mean μ=c/(1−ϕ)\mu = c/(1-\phi), variance γ0=σ2/(1−ϕ2)\gamma_0 = \sigma^2/(1-\phi^2) and autocorrelations ρk=ϕk\rho_k = \phi^k.

Proof

Taking expectations of both sides with a constant mean gives μ=c+ϕμ\mu = c + \phi\mu. For the variance, yt−1y_{t-1} depends only on shocks up to time t−1t-1, so it is uncorrelated with εt\varepsilon_t, and γ0=ϕ2γ0+σ2\gamma_0 = \phi^2\gamma_0 + \sigma^2. For k≥1k \ge 1, multiply the centred equation yt−μ=ϕ(yt−1−μ)+εty_t - \mu = \phi(y_{t-1} - \mu) + \varepsilon_t by yt−k−μy_{t-k} - \mu and take expectations; εt\varepsilon_t is uncorrelated with the past, so γk=ϕγk−1\gamma_k = \phi\gamma_{k-1}. Hence γk=ϕkγ0\gamma_k = \phi^k\gamma_0 and ρk=ϕk\rho_k = \phi^k. If ∣ϕ∣≥1|\phi| \ge 1 the variance equation has no positive solution: the process wanders or explodes.

∎

So an AR process has an ACF that decays geometrically (or as a damped wave), while an MA(qq) process, which only remembers qq shocks, has an ACF that cuts off after lag qq. The mirror-image tool is the partial autocorrelation function (PACF), the correlation at lag kk after removing the effect of lags 1,…,k−11, \ldots, k-1, which cuts off after lag pp for an AR(pp). George Box and Gwilym Jenkins built a whole method (identify, estimate, check) around reading these two plots, in their 1970 book. Today the order is usually chosen by an information criterion, AIC or BIC, from Likelihood and model selection.

An AR model is a regression of the series on its own past, so it can be fitted by least squares. Below, an AR(2) with ϕ1=1.3\phi_1 = 1.3 and ϕ2=−0.6\phi_2 = -0.6 generates a series with a pseudo-cycle: its characteristic roots are complex, so shocks echo as damped oscillations. Udny Yule fitted exactly this kind of model to sunspot numbers in 1927, the first autoregression.

python
import numpy as np

rng = np.random.default_rng(1927)
n = 300
y = np.zeros(n)
for t in range(2, n):                                    # AR(2) with a pseudo-cycle
    y[t] = 10 + 1.3 * y[t - 1] - 0.6 * y[t - 2] + rng.normal(0, 2)
y = y[50:]                                               # drop the start-up stretch
n = len(y)

def fit_ar(y, p, start):
    """Least-squares AR(p): regress y_t on 1, y_{t-1}, ..., y_{t-p}, using t = start, ..., n-1."""
    X = np.column_stack([np.ones(len(y) - start)] + [y[start - k:len(y) - k] for k in range(1, p + 1)])
    target = y[start:]
    coef, *_ = np.linalg.lstsq(X, target, rcond=None)
    resid = target - X @ coef
    return coef, resid, X

# Choose the order by AIC, fitting every candidate on the same stretch of data.
print(" p    AIC")
for p in range(1, 6):
    coef, resid, X = fit_ar(y, p, start=5)
    m = len(resid)
    print(f" {p}  {m * np.log(resid @ resid / m) + 2 * (p + 1):7.1f}")

coef, resid, X = fit_ar(y, 2, start=2)
s2 = resid @ resid / (len(resid) - 3)
se = np.sqrt(np.diag(s2 * np.linalg.inv(X.T @ X)))
print("AR(2) estimates (true 10, 1.3, -0.6):", np.round(coef, 3), " standard errors:", np.round(se, 3))
c, phi1, phi2 = coef
roots = np.roots([1, -phi1, -phi2])                      # z^2 - phi1 z - phi2 = 0
print(f"root modulus {abs(roots[0]):.3f} (< 1, so stationary); implied cycle {2 * np.pi / abs(np.angle(roots[0])):.1f} steps")
print(f"long-run mean {c / (1 - phi1 - phi2):.2f}, observed mean {y.mean():.2f}, residual sd {np.sqrt(s2):.2f}")

# Forecast 12 steps ahead by feeding forecasts back in.
hist = list(y[-2:])
for h in range(12):
    hist.append(c + phi1 * hist[-1] + phi2 * hist[-2])
print("last two values:", np.round(y[-2:], 1), " forecasts:", np.round(hist[2:], 1))
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

AIC drops sharply from order 1 (458.5) to order 2 (351.8) and then creeps up, so it picks the true order. The estimates, 1.240 and −0.594, are within 1.2 standard errors of the truth. The implied cycle of 9.9 steps is close to the true 10.9, and the forecasts show what every stationary model does in the long run: starting from 30.2, they overshoot slightly to 34.0 and then settle at the long-run mean of about 33. The further ahead we look, the less the past tells us, and the forecast becomes the mean.

ExplanationH

Exponential smoothing

A different tradition, born in operational research in the 1950s with Robert Brown's and Charles Holt's work on inventory forecasting, starts from a simple idea: forecast with a weighted average of past observations, giving recent ones more weight. Simple exponential smoothing keeps a level ℓt\ell_t and updates it after each observation: ℓt=ℓt−1+α (yt−ℓt−1),y^t+1=ℓt,\ell_t = \ell_{t-1} + \alpha\,(y_t - \ell_{t-1}), \qquad \hat y_{t+1} = \ell_t, with 0<α≤10 \lt \alpha \le 1. Unrolled, the forecast is αyt+α(1−α)yt−1+α(1−α)2yt−2+⋯\alpha y_t + \alpha(1-\alpha) y_{t-1} + \alpha(1-\alpha)^2 y_{t-2} + \cdots: weights decaying geometrically into the past. A large α\alpha reacts fast and is noisy; a small one is smooth and slow.

Holt's method adds a smoothed slope, so forecasts continue a trend, and Holt–Winters (Peter Winters, 1960) adds a smoothed seasonal index for each position in the season. These methods were long regarded as useful heuristics, until Rob Hyndman and colleagues showed around 2002 that each corresponds to a statistical state-space model (the ETS family, for error, trend and seasonal), with a likelihood, information criteria and proper prediction intervals. Some coincide with ARIMA models; simple exponential smoothing, for instance, gives the optimal forecasts for an ARIMA(0, 1, 1). The two traditions are now two views of the same territory, and good software offers both.

ComputationC

Judging forecasts honestly

A model's fit to the data it was estimated on says little about its forecasts; with enough parameters, any series can be fitted closely. The honest test imitates real use. Pick a forecast origin, fit using only data up to that point, forecast the next value(s), record the error, move the origin forward one step and repeat. This rolling-origin evaluation (also called time-series cross-validation) never lets a model see the future it is judged on. Randomly splitting months into training and test sets, as in ordinary cross-validation, would let a model "forecast" March 2020 from February and April 2020 on either side.

Errors need a scale. The mean absolute error (MAE) is in the data's units; the root mean squared error (RMSE) penalises large misses more; and the mean absolute scaled error (MASE) divides the MAE by the in-sample MAE of a simple benchmark, so that MASE below 1 means "better than the benchmark", whatever the series' units. Percentage errors (MAPE) are popular but break down when values are near zero and treat over- and under-forecasts unequally.

Every evaluation should include benchmarks: the naive forecast (next month equals this month), the seasonal naive (next January equals this January), and perhaps the mean. Sophisticated methods fail to beat them more often than one would hope. The code below runs a rolling-origin comparison on the electricity series: 36 one-month-ahead forecasts, re-tuning each method's smoothing parameters at every origin using only the data available then.

python
import itertools
import numpy as np

# The same synthetic electricity series as before: eight years of monthly demand in GWh.
rng = np.random.default_rng(42)
n = 96
t = np.arange(n)
noise = np.zeros(n)
for i in range(1, n):
    noise[i] = 0.5 * noise[i - 1] + rng.normal(0, 8)
y = 300 + 1.2 * t + 30 * np.cos(2 * np.pi * t / 12) + 12 * np.cos(4 * np.pi * t / 12) + noise

def ses_one_step(x, alpha):
    """Simple exponential smoothing: one-step forecasts for x[1:], and the forecast after the end."""
    level, f = x[0], []
    for v in x[1:]:
        f.append(level)
        level += alpha * (v - level)
    return np.array(f), level

def hw_one_step(x, a, b, g, m=12):
    """Additive Holt-Winters. Initialised from the first two seasons; returns one-step forecasts
    for x[m:], and the forecast after the end."""
    level = x[:m].mean()
    slope = (x[m:2 * m].mean() - x[:m].mean()) / m
    season = list(x[:m] - level)
    f = []
    for i in range(m, len(x)):
        s = season[i - m]
        f.append(level + slope + s)
        new_level = a * (x[i] - s) + (1 - a) * (level + slope)
        slope = b * (new_level - level) + (1 - b) * slope
        season.append(g * (x[i] - new_level) + (1 - g) * s)
        level = new_level
    return np.array(f), level + slope + season[len(x) - m]

def best(fn, x, grid, skip):
    """Smoothing parameters that minimise the in-sample one-step squared error."""
    return min(grid, key=lambda p: np.sum((fn(x, *p)[0] - x[skip:]) ** 2))

methods = {"naive (last month)": [], "seasonal naive": [], "exponential smoothing": [], "Holt-Winters": []}
hw_grid = list(itertools.product([0.1, 0.3, 0.5], [0.01, 0.1], [0.1, 0.3, 0.5]))
for origin in range(60, n):                              # rolling origin: train on y[:origin], forecast y[origin]
    train, actual = y[:origin], y[origin]
    alpha = best(ses_one_step, train, [(a,) for a in np.arange(0.05, 1.0, 0.05)], 1)
    params = best(hw_one_step, train, hw_grid, 12)
    methods["naive (last month)"].append(train[-1] - actual)
    methods["seasonal naive"].append(train[-12] - actual)
    methods["exponential smoothing"].append(ses_one_step(train, *alpha)[1] - actual)
    methods["Holt-Winters"].append(hw_one_step(train, *params)[1] - actual)

scale = np.mean(np.abs(y[12:60] - y[:48]))               # in-sample MAE of the seasonal naive method
print(f"{len(methods['naive (last month)'])} one-month-ahead forecasts, origins from month 60 to 95")
print("method                   MAE    RMSE   MASE")
for name, err in methods.items():
    err = np.array(err)
    print(f"{name:22} {np.mean(np.abs(err)):6.1f}  {np.sqrt(np.mean(err ** 2)):6.1f}  {np.mean(np.abs(err)) / scale:5.2f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Holt–Winters wins clearly, with an MAE of 6.4 GWh against 11.0 for the best benchmark, and a MASE of 0.41. The ranking of the benchmarks is instructive too. The seasonal naive forecast, which one might expect to do well on a seasonal series, is the worst here (MAE 14.2), because copying last January ignores a whole year of growth: about 12×1.2≈1412 \times 1.2 \approx 14 GWh. Simple exponential smoothing barely beats the naive forecast, since it models neither trend nor season. A method that captures all three components does best, as it should, because the simulation was built from exactly those components.

The large forecasting competitions organised by Spyros Makridakis and colleagues turned such comparisons into a science. In the M4 competition (100,000 series), the most accurate entries combined several statistical methods or blended them with machine learning, and simple benchmarks were hard to beat; in M5 (hierarchical retail sales), gradient-boosted trees trained across many related series did best. The durable lessons: always compare against benchmarks, evaluate out of sample, and report uncertainty, not just point forecasts.

Worked examplesH

Structural breaks

Every method above assumes the future resembles the past. Sometimes the rules change: a new tariff changes electricity demand, a change in a survey's questionnaire shifts the unemployment rate, a lockdown in 2020 changes hospital admissions, traffic and retail sales within days. A structural break is a change in the level, trend, seasonal pattern or dependence of a series at some point in time.

Example— Detecting a break

If the date is known in advance (the day a law took effect), Gregory Chow's 1960 test compares a model fitted separately before and after with one fitted to the whole series, using an FF statistic. If the date is unknown, search over candidate dates for the one that most reduces the residual sum of squares. Because the search picks the best of many dates, its test statistic must be compared with a distribution that accounts for the search, not with the ordinary FF table; Jushan Bai and Pierre Perron's 1998 method extends this to several breaks. The exercises ask you to write the search for a single shift in the mean.

What to do about a break depends on its cause. A one-off level shift can be modelled with an indicator variable; a lasting change may mean that only data since the break are relevant, at the cost of less data; and a temporary disruption, such as the pandemic months, is often modelled as an intervention so that it does not distort the seasonal pattern estimated for normal years. Statistical agencies faced exactly this choice in 2020 when seasonally adjusting their headline series.

ExplanationH

Time series in professional practice

National statistical offices publish seasonally adjusted series using programs such as the US Census Bureau's X-13ARIMA-SEATS and the European Statistical System's JDemetra+, which combine ARIMA models, outlier detection and calendar effects (Easter moves, months have different numbers of working days). Central banks, energy system operators and retailers forecast thousands of series automatically. In Python, statsmodels provides ARIMA, exponential smoothing and unit-root tests; in R, the fable family accompanies Hyndman and Athanasopoulos's free textbook. The snippet below (not run here; statsmodels is not loaded in the browser) fits a seasonal ARIMA and a Holt–Winters model to a monthly series.

python
# Python with statsmodels, not run here
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.holtwinters import ExponentialSmoothing

sarima = ARIMA(y, order=(1, 0, 0), seasonal_order=(0, 1, 1, 12), trend="t").fit()
hw = ExponentialSmoothing(y, trend="add", seasonal="add", seasonal_periods=12).fit()
print(sarima.forecast(12), hw.forecast(12))

Since about 2023, research groups have released pretrained foundation models for time series, transformer networks trained on very large collections of series and applied to new ones without re-training: examples include Google Research's TimesFM and Amazon's Chronos. As of 2026 the evidence on them is mixed and active: on some benchmarks they match tuned statistical methods with no tuning at all, on others well-tuned exponential smoothing or gradient boosting is as good or better, and their performance after structural breaks is a live question. The evaluation discipline of this lesson, rolling origins, honest benchmarks and calibrated intervals, is exactly what separates real progress from leaderboard noise. Probabilistic forecasts, judged with the methods of Conformal prediction and proper scoring rules, are increasingly the standard.

Common misconceptionsH

Common misconceptions

Misconception

"Ordinary cross-validation works for time series." Shuffling observations into folds lets the model learn from the future and from neighbours of the very points it is tested on, which are almost copies of them. Errors come out optimistic, sometimes dramatically. Use rolling-origin evaluation, with a gap between training and test data if dependence is long.

Misconception

"A strong correlation between two series over time shows they are linked." Any two series that both trend, up or down, will be strongly correlated. Before interpreting the association, difference or detrend the series, or test for cointegration.

Misconception

"A stationary series has no pattern." Stationarity means the rules of dependence do not change, not that there is no dependence. The AR(2) series above has a clear ten-step cycle and is stationary; a random walk with no pattern at all to the eye is not.

Misconception

"The model with the best in-sample fit forecasts best." In-sample fit always improves with more parameters. Forecast accuracy usually peaks at a modest level of complexity, and combinations of simple methods are often among the most accurate forecasts available.

Exercises

Exercises

1. Time series

Warm-upHuman

Which of these series is (weakly) stationary?

Choose one

2. Time series

Warm-upHuman

Daily deviations of a reservoir's water level from its seasonal norm follow an AR(1) model with ϕ=0.8\phi = 0.8. What is the autocorrelation between deviations three days apart, as a percentage rounded to the nearest whole number?

3. Time series

CoreHuman

A pharmacy in Accra forecasts weekly sales of a cough syrup by simple exponential smoothing with α=0.5\alpha = 0.5. The current level is 100 bottles. The next three weeks' sales are 140, 100 and 130. What is the forecast for the week after, in bottles?

4. Time series

CoreHuman

A stationary series of 400 observations has a sample ACF with one large spike at lag 1 (r1=−0.45r_1 = -0.45) and all other lags inside the band ±0.10\pm 0.10. Its partial autocorrelations decay gradually, alternating in sign. Which model is the natural first candidate?

Choose one

5. Time series

CoreHuman

A team forecasting daily hospital admissions in Manila splits five years of days at random into 80% training and 20% test data, fits a model and reports an excellent test error. Why should a reviewer distrust the result?

Choose one

6. Time series

CoreComputational

Write fit_ar1(y), which fits the model yt=c+ϕ yt−1+εty_t = c + \phi\,y_{t-1} + \varepsilon_t by least squares, regressing y2,…,yny_2, \ldots, y_n on a constant and y1,…,yn−1y_1, \ldots, y_{n-1}. Return the pair (c, phi) as Python floats.

For example, fit_ar1([1, 2, 3, 4, 5]) returns (1.0, 1.0), since each value is 1 plus the previous one.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

7. Time series

StretchComputational

A sensor's readings may have shifted to a new mean at some unknown time. Write best_break(y, min_size=5), which returns the index k that minimises the total sum of squared deviations when the series is split into y[:k] and y[k:], each segment around its own mean. Both segments must have at least min_size values. Return k as a Python int.

For example, best_break([0.0] * 10 + [5.0] * 10) is 10.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Further reading and resources