When the formula runs out
School mathematics is full of problems with tidy answers: the quadratic formula, integrals you can look up, differential equations whose solutions are exponentials. Real problems mostly are not like that. There is no formula for the roots of most equations, no antiderivative for , and no closed-form solution for a ball flying through air that resists it. Yet engineers need the numbers, and they get them with numerical methods: algorithms that approximate the answer to any accuracy you ask for.
MATLAB's strength here is that the best general-purpose methods are already built in, tested for decades and wrapped in a uniform interface. They are often called function functions, because you hand them a function: a root finder needs the function whose zero it seeks, an integrator the integrand, an ODE solver the right-hand side of the equation. You met the first of them, fzero, with function handles in lesson 4.
Using them well needs three habits that run through this lesson. Know roughly how the method works, so you can predict when it struggles. Understand tolerances, the accuracy the solver aims for, and set them deliberately. And check results against something you know: an exact solution, a conservation law, a convergence rate.
A few terms
Definition— Tolerance
The accuracy a numerical routine aims for, usually as an absolute tolerance (an error size, such as 1e-10) and a relative tolerance (a fraction of the answer's size, such as 1e-6, about six significant digits). Solvers stop when their own estimate of the error is below the tolerance. Options are set with optimset for root finders and minimisers, name-value pairs for integral, and odeset for ODE solvers.
Definition— Order of convergence
A method is of order p if halving its step size divides the error by about . The trapezoid rule is order 2; the method inside ode45 is order 5 (with an order-4 companion used to estimate the error).
Definition— Initial value problem
An ordinary differential equation together with a starting value . y may be a vector, so a system of equations, and any higher-order equation can be rewritten as a first-order system by adding derivatives as extra unknowns.
Roots and minima
% 1. Polynomial roots: x^3 - 2x - 5 = 0, coefficients from the highest power.
c = [1 0 -2 -5];
r = roots(c);
disp(r)
xr = r(abs(imag(r)) < 1e-12);
fprintf('real root %.10f, check %.1e\n', xr, polyval(c, xr));
% 2. fzero: Kepler's equation M = E - e*sin(E) for an orbit of eccentricity 0.967.
e = 0.967;
kepler = @(E, M) E - e * sin(E) - M;
for M = [0.1 1 3]
E = fzero(@(E) kepler(E, M), [0 2*pi]);
fprintf('mean anomaly %.1f -> eccentric anomaly %.6f\n', M, E);
end
% fzero from a single starting guess searches for a sign change first.
fprintf('cos(x) = x at %.10f\n', fzero(@(x) cos(x) - x, 1));
% 3. fminbnd: the 330 ml can with the least metal (radius r in cm).
V = 330;
area = @(r) 2*pi*r.^2 + 2*V ./ r;
[rBest, aBest] = fminbnd(area, 1, 10);
fprintf('best radius %.4f cm (calculus says %.4f), area %.2f cm^2\n', rBest, (V/(2*pi))^(1/3), aBest);
% 4. fminsearch: Rosenbrock's banana-shaped valley, minimum at (1, 1).
rosen = @(p) (1 - p(1))^2 + 100 * (p(2) - p(1)^2)^2;
opts = optimset('TolX', 1e-10, 'TolFun', 1e-10);
[p, fval] = fminsearch(rosen, [-1.2 1], opts);
fprintf('minimum near (%.6f, %.6f), value %.1e\n', p, fval); 2.0946 + 0i
-1.0473 + 1.1359i
-1.0473 - 1.1359i
real root 2.0945514815, check 2.0e-14
mean anomaly 0.1 -> eccentric anomaly 0.780228
mean anomaly 1.0 -> eccentric anomaly 1.911437
mean anomaly 3.0 -> eccentric anomaly 3.069578
cos(x) = x at 0.7390851332
best radius 3.7450 cm (calculus says 3.7449), area 264.36 cm^2
minimum near (1.000000, 1.000000), value 5.0e-22rootstakes a polynomial's coefficients and returns all its roots, complex ones included. It does so with linear algebra from the last lesson: the roots are the eigenvalues of a companion matrix built from the coefficients. This cubic is the one Isaac Newton used to demonstrate his method; it has one real root and a complex pair.polyvalevaluates the polynomial, confirming the root to rounding level. (MATLAB prints the complex column as2.0946 + 0.0000i; only the layout differs.)fzerofinds a zero of any continuous function of one variable. Given a bracket, an interval where the function changes sign, it is guaranteed to succeed: it combines bisection, which always works, with faster interpolation steps. (MATLAB's version follows the classic algorithm of Dekker and Brent; Octave's follows a later refinement by Alefeld, Potra and Shi.) Kepler's equation links time to position along an elliptical orbit; with an eccentricity of 0.967, roughly that of Halley's comet, it is far from linear, andfzerohandles it without fuss.fzeroneeds a function of one variable, so the anonymous function@(E) kepler(E, M)fixesM, capturing its current value as you learned in lesson 4. With a single starting point instead of a bracket,fzerofirst searches outwards for a sign change, which usually works but is not guaranteed.fminbndfinds a minimum of a function of one variable on an interval. The answer, 3.7450 cm, differs from the calculus answer in the fourth decimal becausefminbndstops once it knows the minimiser to within its defaultTolXof1e-4. That is not an error, it is the accuracy you asked for by not asking. (The best can is exactly as tall as it is wide; real cans are taller, for reasons of handling and labels.)fminsearchminimises a function of several variables with the Nelder–Mead simplex method, which needs no derivatives. Rosenbrock's function has its minimum at the bottom of a long, curved, flat valley, a classic stress test for optimisers. Tightened tolerances, set withoptimset, get the answer to six decimals.
For systems of nonlinear equations there is fsolve. It is built into Octave but belongs to the Optimization Toolbox in MATLAB, a reminder that "it works in Octave" does not always mean "it works in every MATLAB installation". Use optimset for options: it works in both programs, whereas MATLAB's newer optimoptions does not exist in Octave 11.
Integrals of functions and of data
There are two different situations. When you can evaluate the integrand anywhere, use integral, which chooses its own points adaptively. When you only have measurements at fixed points, use trapz, which joins them with straight lines.
% 1. integral: adaptive quadrature of a function, even over infinite ranges.
fprintf('%.10f (sqrt(pi) = %.10f)\n', integral(@(x) exp(-x.^2), -Inf, Inf), sqrt(pi));
% 1/sqrt(x) is infinite at 0, but the area under it from 0 to 1 is exactly 2.
fprintf('%.10f default tolerances\n', integral(@(x) 1 ./ sqrt(x), 0, 1));
fprintf('%.10f RelTol 1e-10\n', integral(@(x) 1 ./ sqrt(x), 0, 1, 'RelTol', 1e-10));
% 2. trapz and cumtrapz: data sampled at known times.
% A solar panel's power output (kW), measured every hour from 06:00 to 18:00.
hours = 6:18;
power = [0 0.3 1.1 2.0 2.8 3.3 3.5 3.4 3.0 2.3 1.4 0.5 0];
energy = trapz(hours, power);
fprintf('energy over the day: %.2f kWh\n', energy);
soFar = cumtrapz(hours, power);
fprintf('half of it was produced by %d:00\n', hours(find(soFar >= energy / 2, 1)));
% 3. How the trapezoid rule's error shrinks: integral of sin from 0 to pi is 2.
fprintf('%5s %12s %8s\n', 'n', 'error', 'ratio');
previous = NaN;
for n = [10 20 40 80 160]
x = linspace(0, pi, n + 1);
err = abs(trapz(x, sin(x)) - 2);
fprintf('%5d %12.3e %8.2f\n', n, err, previous / err);
previous = err;
end1.7724538509 (sqrt(pi) = 1.7724538509)
1.9999998579 default tolerances
2.0000000000 RelTol 1e-10
energy over the day: 23.60 kWh
half of it was produced by 13:00
n error ratio
10 1.648e-02 NaN
20 4.114e-03 4.00
40 1.028e-03 4.00
80 2.570e-04 4.00
160 6.426e-05 4.00integraluses adaptive Gauss–Kronrod quadrature: it evaluates the function at cleverly chosen points on each piece of the interval, estimates the error by comparing two rules, and subdivides only the pieces where the estimate is too large. It accepts infinite limits, and the bell-curve integral matches to every digit shown.- The integrable spike of at zero is harder. With the default tolerances (a relative tolerance of
1e-6and an absolute tolerance of1e-10, the same in both programs) Octave's answer is correct to about seven digits, exactly the accuracy requested. Asking for more with'RelTol'gets more. MATLAB's implementation differs in its details and may print other trailing digits; both are right to within the tolerance. - The integrand must accept a vector of points and return a vector of values, so write it with
.*,./and.^. Forgetting the dot is the most commonintegralbug. trapz(x, y)integrates sampled data: here power in kilowatts over hours gives energy in kilowatt-hours.cumtrapzreturns the running integral, which answers questions like "by when was half the energy produced?".- The table shows the trapezoid rule is second order: doubling the number of intervals divides the error by exactly 4. Measuring this ratio is the standard way to check that a numerical method is working as designed, the same check the sparse rod made in the last lesson.
Double and triple integrals have integral2 and integral3, available in both programs.
Differential equations with ode45
[t, y] = ode45(f, tspan, y0) solves from . The function f takes a scalar time and a column vector state and returns a column vector of derivatives. If tspan has two elements, [t0 tEnd], the solver returns the points it chose itself; if it has more, it returns the solution at exactly those times. Each row of the output y is the state at the matching time in t.
% 1. One equation: Newton's law of cooling for a cup of tea in a 20 C room.
k = 0.08; % per minute
cooling = @(t, T) -k * (T - 20);
[t, T] = ode45(cooling, [0 5 10 20 30], 90);
exact = 20 + 70 * exp(-k * t);
fprintf('%4s %9s %9s %10s\n', 't', 'ode45', 'exact', 'error');
fprintf('%4d %9.4f %9.4f %10.2e\n', [t, T, exact, abs(T - exact)]');
% 2. A system: the SIR epidemic model, as fractions of the population.
beta = 0.3; % infections per infectious person per day
gamma = 0.1; % recoveries per day: 10 days ill on average
sir = @(t, y) [-beta * y(1) * y(2)
beta * y(1) * y(2) - gamma * y(2)
gamma * y(2)];
opts = odeset('RelTol', 1e-8, 'AbsTol', 1e-10);
[t, y] = ode45(sir, 0:0.5:300, [0.999; 0.001; 0], opts);
[peak, k] = max(y(:, 2));
fprintf('R0 = %.1f; infections peak at day %.1f with %.1f%% ill at once\n', beta / gamma, t(k), 100 * peak);
fprintf('after 300 days: %.2f%% were never infected\n', 100 * y(end, 1));
fprintf('S + I + R stays %.12f\n', sum(y(end, :)));
% The final size also solves r = 1 - s0*exp(-R0*r): a root-finding problem.
R0 = beta / gamma;
rFinal = fzero(@(r) r - (1 - 0.999 * exp(-R0 * r)), [0.5 1]);
fprintf('final-size equation: %.2f%% infected in total; ODE: %.2f%%\n', 100 * rFinal, 100 * y(end, 3)); t ode45 exact error
0 90.0000 90.0000 0.00e+00
5 66.9224 66.9224 1.11e-07
10 51.4530 51.4530 1.64e-06
20 34.1328 34.1328 4.18e-06
30 26.3503 26.3503 3.73e-06
R0 = 3.0; infections peak at day 38.5 with 30.1% ill at once
after 300 days: 5.94% were never infected
S + I + R stays 1.000000000000
final-size equation: 94.06% infected in total; ODE: 94.06%- The cooling equation has an exact solution, so the program can measure the error: a few millionths of a degree with the default tolerances (
RelTol1e-3,AbsTol1e-6). Solving a problem with a known answer first is the quickest way to make sure you have set a solver up correctly. - The SIR model splits a population into susceptible, infectious and recovered fractions. It is three coupled equations, written as one anonymous function returning a 3-by-1 column (the line breaks inside the brackets start new rows). The output
yhas one column per variable, soy(:, 2)is the infectious fraction over time. - Two independent checks are built in. The three equations' right-hand sides add up to zero, so S + I + R must stay 1, and it does to twelve decimals. And epidemiology offers a separate equation for the final size of an epidemic, solved here with
fzero; it agrees with the ODE to the digits shown. - The model's numbers (a basic reproduction number of 3, ten days of illness) are illustrative, not a forecast of any real disease. Real epidemic models add age groups, contacts, vaccination and much more, and are solved with exactly these tools.
Inside, ode45 is the Dormand–Prince method, an explicit Runge–Kutta pair: at each step it computes two estimates of orders 4 and 5 from six function evaluations, uses their difference as an error estimate, and shrinks or grows the step to keep that estimate within tolerance. ode23 is a cheaper, lower-order pair, useful with loose tolerances. Octave implements the same methods as MATLAB, but the two programs make slightly different choices about step sizes, so their results agree to within the tolerances rather than digit for digit. MATLAB's deval, which evaluates a solution structure at new times, is not available in Octave 11; ask for the times you need in tspan instead.
Stiff problems
Some equations mix very fast and very slow behaviour: a chemical reaction where one species reacts in microseconds and another over hours, or a circuit with a tiny capacitor. Such problems are called stiff. An explicit method such as ode45 must take tiny steps throughout, not for accuracy but because larger steps make it numerically unstable. Implicit methods solve a small system of equations at every step (with the linear algebra of the last lesson) and can take steps as large as accuracy allows.
% A stiff equation: y is pulled very hard (rate 1000) towards cos(t).
f = @(t, y) -1000 * (y - cos(t));
exact = (1e6 * cos(2) + 1000 * sin(2)) / (1e6 + 1); % the true y(2)
opts = odeset('Refine', 1); % one output per step
[t1, y1] = ode45(f, [0 2], 1, opts);
[t2, y2] = ode15s(f, [0 2], 1, opts);
[t3, y3] = ode23s(f, [0 2], 1, opts);
fprintf('ode45: %4d steps, error at t = 2: %.1e\n', numel(t1) - 1, abs(y1(end) - exact));
fprintf('ode15s: %4d steps, error at t = 2: %.1e\n', numel(t2) - 1, abs(y2(end) - exact));
fprintf('ode23s: %4d steps, error at t = 2: %.1e\n', numel(t3) - 1, abs(y3(end) - exact));
fprintf('average ode45 step %.2e\n', mean(diff(t1)));ode45: 609 steps, error at t = 2: 5.3e-05
ode15s: 21 steps, error at t = 2: 2.6e-06
ode23s: 57 steps, error at t = 2: 1.3e-05
average ode45 step 3.28e-03The solution simply follows smoothly, yet ode45 needs about 30 times as many steps as ode15s. Its average step, 0.0033, is no accident: it sits right at the stability limit, about 3.3/1000, that the rate 1000 imposes on this explicit method, however smooth the solution. (Refine is set to 1 because by default ode45 reports four output points per step.) The rule of thumb in both programs: start with ode45; if it is unexpectedly slow, the problem is probably stiff, so try ode15s, or ode23s with loose tolerances. Step counts like these vary between programs and versions; the large ratio does not.
One Octave-specific note. Octave's ode15s is built on the SUNDIALS library, and in testing this lesson it failed to take its first step when the same problem started from y(0) = 0 (an IDASolve failed error), where the solution changes extremely fast at the start. Passing odeset('InitialStep', 1e-6) fixed it.
Worked example: how far does a ball fly?
A school physics book says a ball launched at 30 m/s flies furthest at 45 degrees, landing = 91.7 m away. That ignores air. With air drag proportional to the square of the speed there is no formula, so here is the numerical answer. The program combines a second-order system written as four first-order equations, an event function that stops the solver when the ball lands, and fminbnd wrapped around the ODE solver to find the best angle:
1;
function dy = flight(~, y, c)
% FLIGHT A ball with quadratic air drag. State y = [x; height; vx; vy];
% c is the drag coefficient per unit mass, in 1/m.
g = 9.81;
speed = hypot(y(3), y(4));
dy = [y(3)
y(4)
-c * speed * y(3)
-g - c * speed * y(4)];
end
function [value, isterminal, direction] = landed(~, y)
% LANDED Event function: height crosses zero while falling.
value = y(2);
isterminal = 1; % stop the integration there
direction = -1; % only count downward crossings
end
function [range, duration] = throwBall(angle, speed, c)
% THROWBALL Range (m) and flight time (s) for a launch angle in degrees.
y0 = [0; 0; speed * cosd(angle); speed * sind(angle)];
opts = odeset('Events', @landed, 'RelTol', 1e-8, 'AbsTol', 1e-10);
[~, ~, te, ye] = ode45(@(t, y) flight(t, y, c), [0 60], y0, opts);
range = ye(end, 1);
duration = te(end);
end
v0 = 30; % launch speed, m/s
cDrag = 0.005; % drag per unit mass, 1/m: a modest amount
fprintf('%5s %10s %10s %10s\n', 'angle', 'vacuum', 'with drag', 'flight');
for angle = [30 40 45 50]
rVac = throwBall(angle, v0, 0);
[rDrag, tDrag] = throwBall(angle, v0, cDrag);
fprintf('%5d %9.2fm %9.2fm %9.2fs\n', angle, rVac, rDrag, tDrag);
end
fprintf('textbook vacuum range at 45 degrees: %.2f m\n', v0^2 / 9.81);
% Optimise through the ODE solver: the best launch angle with drag.
best = fminbnd(@(a) -throwBall(a, v0, cDrag), 20, 60, optimset('TolX', 1e-6));
fprintf('best angle with drag: %.2f degrees, range %.2f m\n', best, throwBall(best, v0, cDrag));angle vacuum with drag flight
30 79.45m 62.75m 2.86s
40 90.35m 68.50m 3.63s
45 91.74m 68.60m 3.98s
50 90.35m 66.90m 4.29s
textbook vacuum range at 45 degrees: 91.74 m
best angle with drag: 42.77 degrees, range 68.78 mHow it fits together:
- First-order form. Newton's second law is second order in position. Making the velocities part of the state,
y = [x; height; vx; vy], turns it into four first-order equations: the derivative of position is velocity, and the derivative of velocity is the acceleration from gravity and drag. This is how every mechanical system is fed to an ODE solver. - Events. We do not know in advance when the ball lands, so the program integrates up to a generous 60 seconds and lets the event function stop it. The solver watches
value(the height) for a sign change in the givendirection, locates the crossing precisely, and, becauseisterminalis 1, stops there. The extra outputsteandyeare the time and state at the event. Octave's ODE solvers support events with the same interface as MATLAB's. - A check first. With
c = 0the solver reproduces the textbook formula to the centimetre, and ranges at 40 and 50 degrees are equal, as the vacuum theory predicts. Only then are the drag results worth believing. - Solvers inside solvers.
fminbndminimises the negative range, and every evaluation it makes runs a whole ODE solution. Note thatthrowBallreturns two outputs, but inside the anonymous function only the first is used. Drag lowers the best angle below 45 degrees, here to about 42.8, a real effect known to anyone who has thrown things outdoors. - The local functions use
~for the time argument they ignore, andhypot(a, b), which computes without overflow.
How practitioners work with solvers
Trust, but verify. Professional numerical work always includes checks: a version of the problem with a known solution, a conserved quantity, a second solver or tolerance that gives the same digits, or a convergence table. When a solver gives a surprising answer, halve the tolerance and run it again; if the answer moves, it was not converged.
Set tolerances on purpose. Defaults are chosen for speed with moderate accuracy (ode45 aims for about three significant digits by default). A result you will publish or build on deserves tighter tolerances and a check that tightening them further changes nothing that matters. Remember that AbsTol matters when a solution component passes through or near zero.
Write vectorised, side-effect-free functions. Solvers may call your function thousands of times, at points of their choosing, in any order. It should depend only on its inputs, never print, and, for integral and plotting, accept arrays.
Know what you are calling. fzero needs a sign change, fminsearch finds a local minimum near its starting point, and fminbnd assumes one minimum in the interval. Starting an optimiser from several points is cheap insurance.
Do not write your own Runge–Kutta for production work, but do write one once (the exercises of a numerical analysis course will ask you to), because seeing a fixed-step method go unstable on a stiff problem is the best way to understand why the library solvers work as they do. Interpolation (interp1, spline, pchip) and polynomial fitting (polyfit) complete the everyday toolkit and behave the same in both programs.
Common misconceptions
Misconception
"The solver's answer is exact to all the digits it prints." It is accurate to roughly the tolerance, which by default is far looser than the 16 digits a double can show. Print fewer digits, or tighten the tolerance until the digits you need stop changing.
Misconception
"ode45 returns the solution at evenly spaced times." With tspan = [t0 tEnd] it returns the times it chose itself (plus refined points), which are uneven. Pass a vector of times, such as 0:0.5:300, to get the solution exactly there.
Misconception
"A slow ODE solve means the problem needs more accuracy." Often it means the problem is stiff, and an explicit solver is being forced into tiny steps for stability. Switching to ode15s or ode23s can be a hundred times faster.
Misconception
"fminsearch finds the minimum." It finds a local minimum near its starting point, and may stop early on flat or badly scaled problems. Try several starting points and check the result.
Exercises
1. Numerical methods: roots, integrals and ODEs
Warm-upComputationalWhat numbers does disp(sort(roots([1 -5 6]))') show? Write them separated by a single space.
2. Numerical methods: roots, integrals and ODEs
CoreComputationalWith f = @(x) x.^2 - 4, which call finds the root 2 with a bracket that fzero accepts?
3. Numerical methods: roots, integrals and ODEs
Warm-upComputationalWhat does disp(trapz(0:2:4, [1 3 2])) display?
4. Numerical methods: roots, integrals and ODEs
CoreHumanYou test a new integration routine. With step size h its error is 3.2e-3; with h/2 it is 2.0e-4; with h/4 it is 1.25e-5. What is the order of the method?
5. Numerical methods: roots, integrals and ODEs
CoreComputationalTo solve the oscillator equation y'' = −y with ode45, you use the state y = [position; velocity]. Which right-hand side is correct?
6. Numerical methods: roots, integrals and ODEs
CoreComputationalWhat does this display? Write the two numbers separated by a space.
[t, y] = ode45(@(t, y) -y, [0 0.5 1 1.5 2], [1; 2]);
disp(size(y))
7. Numerical methods: roots, integrals and ODEs
Warm-upComputationalintegral(@(x) x^2, 0, 1) stops with an error in Octave (and in MATLAB). Why?
8. Numerical methods: roots, integrals and ODEs
StretchHumanA chemical kinetics model has reactions on time scales from microseconds to hours. ode45 takes millions of tiny steps, although the concentrations you plot change smoothly over hours. What is the best first move?
Further reading and resources
- GNU Octave manual: Nonlinear Equationsfzero and fsolve.
- GNU Octave manual: Minimizers
- GNU Octave manual: Numerical Integration
- GNU Octave manual: Ordinary Differential Equations
- MathWorks documentationSearch for "Choose an ODE Solver" for MATLAB's guide to its solver suite.
- The MATLAB ODE Suite · Lawrence F. Shampine and Mark W. Reichelt · 1997The paper describing the design of ode45, ode23s, ode15s and their relatives.
- A family of embedded Runge-Kutta formulae · J. R. Dormand and P. J. Prince · 1980The method inside ode45.