Mathematical modelling with functions

MotivationH

Three questions, three functions

A taxi company in Nairobi charges a fixed fee plus an amount per kilometre. What will a $12$ km trip cost?
A bus driver in Mumbai sees a child step into the road. How far does the bus travel before it stops, and how much farther at twice the speed?
A family in Lima saves in an account that pays interest every year. When will their savings double?

Each question is about the real world, and each has an answer that comes from a function: linear for the taxi, quadratic for the stopping distance, exponential for the savings. But no formula is a taxi, a bus or a bank account. To get from the situation to a formula we must simplify: decide what matters, what to ignore and what to assume. To trust the answer we must check it against reality. This lesson is about that whole journey, which is called mathematical modelling.

IntuitionH

The modelling cycle

The modelling cycle. Simplify the situation and formulate a model; solve the mathematics; interpret the results in the situation; validate them against reality. If the predictions fail, revise the assumptions and go round again.

Simplify. Decide which features matter and state your assumptions ("the fare is the same at every time of day", "the road is dry", "the interest rate stays the same").
Formulate. Choose variables and a function that links them.
Solve. Do the mathematics: evaluate, solve an equation, find a maximum.
Interpret. Translate the answer back: units, sensible ranges, what it means.
Validate. Compare predictions with observations the model was not built from. If they disagree, revise the assumptions and go round again.

The statistician George Box put it memorably in 1979: "all models are wrong, but some are useful". A model leaves things out on purpose, and it is judged by whether it serves its purpose, not by whether it is "true".

DefinitionsH

Definitions

Definition— mathematical model

A mathematical model is a simplified mathematical description of a real situation, together with its assumptions. It relates variables (quantities that change, such as time or distance) through a function whose parameters (constants such as a rate or a starting value) are chosen to suit the situation.

Definition— fit, residual and validation

Fitting a model means choosing its parameters to match data. The residual of a data point is observed value minus predicted value. Validation compares a model's predictions with observations, preferably ones not used in fitting.

Definition— interpolation and extrapolation

Using a model inside the range of the data used to build it is interpolation; using it outside that range is extrapolation.

ProofH

Fingerprints of three families

Suppose a quantity is measured at equally spaced times $0, 1, 2, \dots$ , giving values $u_0, u_1, u_2, \dots$ How can a table tell you which kind of function to try?

Theorem— Constant differences mean linear

If $u_{n+1} - u_n = d$ for every $n$ , then $u_n = u_0 + nd$ for every $n$ .

Proof

Induction: true for $n = 0$ , and if $u_n = u_0 + nd$ then $u_{n+1} = u_n + d = u_0 + (n + 1)d$ .

∎

Theorem— Constant ratios mean exponential

If $u_{n+1} = b\,u_n$ for every $n$ , then $u_n = u_0\, b^n$ for every $n$ .

Proof

Induction: $u_{n+1} = b\,u_n = b\,u_0 b^n = u_0 b^{n+1}$ .

∎

Theorem— Constant second differences mean quadratic

If the differences of the differences are constant, $u_{n+2} - 2u_{n+1} + u_n = c$ for every $n$ , then

u_n = u_0 + n\,(u_1 - u_0) + c\,\frac{n(n-1)}{2},

a quadratic function of $n$ .

Proof

We prove the formula for $n$ and for $n + 1$ together, by induction on $n$ . For $n = 0$ both are immediate. If the formula holds for $u_n$ and $u_{n+1}$ , then $u_{n+2} = c + 2u_{n+1} - u_n$ , and substituting the two formulas and simplifying gives the formula for $n + 2$ .

∎

Example— Reading a table

$n$	0	1	2	3	4
A	7	10	13	16	19
B	2	5	10	17	26
C	3	6	12	24	48

Column A has constant differences $3$ : linear, $u_n = 7 + 3n$ . Column B has differences $3, 5, 7, 9$ and constant second differences $2$ : quadratic, $u_n = 2 + 3n + n(n - 1) = n^2 + 2n + 2$ . Column C has constant ratio $2$ : exponential, $u_n = 3 \cdot 2^n$ . Real data are noisy, so in practice you look for differences or ratios that are roughly constant.

Theorem— Two points determine a line

If $x_1 \ne x_2$ , there is exactly one line $y = mx + c$ through $(x_1, y_1)$ and $(x_2, y_2)$ , namely $m = \dfrac{y_2 - y_1}{x_2 - x_1}$ and $c = y_1 - m x_1$ .

Proof

These values work: $mx_1 + c = y_1$ , and $mx_2 + c = y_1 + m(x_2 - x_1) = y_2$ . If another line $m'x + c'$ passes through both points, subtracting the two equations gives $(m - m')(x_1 - x_2) = 0$ , so $m = m'$ , and then $c = c'$ .

∎

Theorem— Doubling time

A quantity $A e^{rt}$ with $A > 0$ and continuous growth rate $r > 0$ doubles in time $T = \dfrac{\ln 2}{r} \approx \dfrac{0.693}{r}$ , whatever its starting value. With yearly compounding at rate $r$ the doubling time is $\dfrac{\ln 2}{\ln(1 + r)}$ , which is at least as long.

Proof

$Ae^{rT} = 2A \iff e^{rT} = 2 \iff rT = \ln 2$ . For yearly compounding, $(1 + r)^T = 2 \iff T = \tfrac{\ln 2}{\ln(1 + r)}$ . Since $1 + r \le e^r$ , we have $\ln(1 + r) \le r$ , and dividing $\ln 2$ by a smaller positive number gives a larger answer.

∎

At $r = 7\%$ the doubling time is about $0.693 / 0.07 \approx 9.9$ years. This is the "rule of 70": divide $70$ by the percentage rate.

Worked examplesH

Building models

Example— Linear: the taxi fare (illustrative numbers)

Suppose a $4$ km trip costs $500$ shillings and a $10$ km trip costs $950$ . Assuming the fare is a fixed fee plus a constant amount per kilometre, the fare is linear in the distance $x$ . By the two-point theorem the slope is $m = \tfrac{950 - 500}{10 - 4} = 75$ shillings per km, and the intercept is $c = 500 - 75 \cdot 4 = 200$ shillings. So $F(x) = 200 + 75x$ , and a $12$ km trip costs $F(12) = 1100$ shillings.

Interpret: $75$ is the price per kilometre and $200$ is the fixed fee. $F(0) = 200$ is the price of getting in and going nowhere. The model is meaningful only for $x \ge 0$ , and it ignores waiting time, surge pricing and tolls.

Example— Quadratic: stopping distance

A vehicle's stopping distance has two parts. During the driver's reaction time $t_r$ the vehicle keeps its speed $v$ and travels $v\,t_r$ . While braking with constant deceleration $a$ it travels $\tfrac{v^2}{2a}$ . With the assumptions $t_r = 1$ s and $a = 7\ \text{m/s}^2$ (a dry road),

d(v) = v + \frac{v^2}{14} \quad \text{metres, for } v \text{ in m/s}.

At $50$ km/h ( $13.9$ m/s), $d \approx 13.9 + 13.8 = 27.7$ m. At $100$ km/h ( $27.8$ m/s), $d \approx 27.8 + 55.1 = 82.9$ m: doubling the speed doubles the reaction distance but quadruples the braking distance. On a wet road ( $a \approx 4\ \text{m/s}^2$ , say) the braking distance would be much longer. Changing an assumption changes the prediction, and a good modeller reports both.

Example— Exponential: savings and a warning from biology

Savings growing at $5\%$ a year follow $S(t) = S_0 \cdot 1.05^t$ and double in $\tfrac{\ln 2}{\ln 1.05} \approx 14.2$ years.

Exponential models can be spectacularly right for a while and then spectacularly wrong. In ideal laboratory conditions a bacterium such as E. coli can divide about every $20$ minutes. If that continued for $48$ hours, one cell would become $2^{144} \approx 2 \times 10^{43}$ cells, weighing far more than the Earth. In reality nutrients run out and growth slows. A better model then is the logistic model, in which the growth rate falls as the population approaches a limit (see mathematical biology).

ProofH

Fitting a line to data: least squares

Real data never lie exactly on a line. Which line fits "best"? The standard answer, due to Legendre and Gauss, is the line that makes the sum of squared residuals

S(m, c) = \sum_{i=1}^{n} \big(y_i - (m x_i + c)\big)^2

as small as possible. Squaring treats overshoots and undershoots alike and punishes large misses heavily.

Theorem— The best intercept is set by the means

Fix a slope $m$ and put $r_i = y_i - m x_i$ , with mean $\bar r = \tfrac1n \sum r_i = \bar y - m \bar x$ . Then for every $c$ ,

\sum_{i} (r_i - c)^2 = \sum_{i} (r_i - \bar r)^2 + n\,(c - \bar r)^2.

So, for a given slope, the sum of squared residuals is smallest exactly when $c = \bar y - m\bar x$ : the best line passes through the point of means $(\bar x, \bar y)$ .

Proof

Write $r_i - c = (r_i - \bar r) + (\bar r - c)$ and expand:

\sum (r_i - c)^2 = \sum (r_i - \bar r)^2 + 2(\bar r - c)\sum (r_i - \bar r) + n(\bar r - c)^2.

The middle sum is $\sum r_i - n\bar r = 0$ : deviations from the mean add up to zero. The last term is never negative and is zero only when $c = \bar r$ .

∎

Now substitute $c = \bar y - m\bar x$ . Writing $S_{xx} = \sum (x_i - \bar x)^2$ , $S_{xy} = \sum (x_i - \bar x)(y_i - \bar y)$ and $S_{yy} = \sum (y_i - \bar y)^2$ , the sum of squares becomes

\sum \big((y_i - \bar y) - m(x_i - \bar x)\big)^2 = S_{yy} - 2m S_{xy} + m^2 S_{xx},

a quadratic in $m$ with positive leading coefficient $S_{xx}$ (if the $x_i$ are not all equal). By the vertex formula of the quadratics lesson, its minimum is at

m = \frac{S_{xy}}{S_{xx}}, \qquad c = \bar y - m \bar x.

That is the least-squares line. To fit an exponential $P = A e^{rt}$ , take logarithms: $\ln P = \ln A + rt$ is linear in $t$ , so fit a line to the points $(t, \ln P)$ . This minimises errors on the logarithmic scale, which means roughly relative errors.

ExplanationH

Validation: where models fail

A model earns trust by predicting what it was not built from. Three warnings:

Fitting the past is not forecasting the future. In the computation section, a linear and an exponential model both fit world population for 1950–1980 closely, yet their forecasts for 2020 differ by more than $2.5$ billion people, and both miss.
A perfect fit proves nothing. Through any $n$ data points with different $x$ -values there is a polynomial of degree less than $n$ passing exactly through all of them. A model with as many parameters as data points will always fit perfectly, and it typically forecasts nonsense. This is called overfitting.
Data cannot decide extrapolation. The theorem below shows that infinitely many models agree exactly with any finite set of data and yet predict any value you like at a new point. What justifies an extrapolation is not the data alone but the reasons behind the model: a mechanism (such as Newton's law of cooling) or a well-tested assumption.

Theorem— Extrapolation is not determined by data

Let $f$ be any model and $x_1, \dots, x_n$ the points where we have data. For every number $k$ , the model

g(x) = f(x) + k\,(x - x_1)(x - x_2)\cdots(x - x_n)

agrees with $f$ at every data point. At any new point $x_0$ (different from all $x_i$ ), choosing $k = \dfrac{v - f(x_0)}{(x_0 - x_1)\cdots(x_0 - x_n)}$ makes $g(x_0) = v$ , for any value $v$ we like.

Proof

At $x = x_i$ the product contains the factor $x_i - x_i = 0$ , so $g(x_i) = f(x_i)$ . At $x_0$ the product is a product of non-zero numbers, hence non-zero, so we may divide by it; substituting the chosen $k$ gives $g(x_0) = f(x_0) + (v - f(x_0)) = v$ .

∎

Formal mathematicsF

Modelling facts, checked by Lean

Lean cannot check that a model suits a situation. That is a judgement about the world. But it can check the mathematics inside the modelling cycle. The fingerprints of the three families:

linear_of_const_diffLean 4Proved here from basic factsFormally verified

/-- **Constant differences mean a linear model.** If a quantity measured at times
`0, 1, 2, …` always changes by the same amount `d`, then `u n = u 0 + n d`. Induction on `n`. -/
theorem linear_of_const_diff (u : ℕ → ℝ) (d : ℝ) (h : ∀ n, u (n + 1) - u n = d) (n : ℕ) :
    u n = u 0 + n * d := by
  induction n with
  | zero => simp
  | succ n ih =>
    have := h n
    push_cast
    linarith