Skip to content
LumenaryAcademy
Professional 65 min

Numerical methods: roots, integrals and ODEs

Finding roots and minima, integrating functions and data, and solving differential equations with ode45 and the stiff solvers, with tolerances you understand.

Code tested · 5AI-draftedAwaiting review by a computer scientist
MotivationH

When the formula runs out

School mathematics is full of problems with tidy answers: the quadratic formula, integrals you can look up, differential equations whose solutions are exponentials. Real problems mostly are not like that. There is no formula for the roots of most equations, no antiderivative for e−x2e^{-x^2}, and no closed-form solution for a ball flying through air that resists it. Yet engineers need the numbers, and they get them with numerical methods: algorithms that approximate the answer to any accuracy you ask for.

MATLAB's strength here is that the best general-purpose methods are already built in, tested for decades and wrapped in a uniform interface. They are often called function functions, because you hand them a function: a root finder needs the function whose zero it seeks, an integrator the integrand, an ODE solver the right-hand side of the equation. You met the first of them, fzero, with function handles in lesson 4.

Using them well needs three habits that run through this lesson. Know roughly how the method works, so you can predict when it struggles. Understand tolerances, the accuracy the solver aims for, and set them deliberately. And check results against something you know: an exact solution, a conservation law, a convergence rate.

DefinitionsH

A few terms

Definition— Tolerance

The accuracy a numerical routine aims for, usually as an absolute tolerance (an error size, such as 1e-10) and a relative tolerance (a fraction of the answer's size, such as 1e-6, about six significant digits). Solvers stop when their own estimate of the error is below the tolerance. Options are set with optimset for root finders and minimisers, name-value pairs for integral, and odeset for ODE solvers.

Definition— Order of convergence

A method is of order p if halving its step size divides the error by about 2p2^p. The trapezoid rule is order 2; the method inside ode45 is order 5 (with an order-4 companion used to estimate the error).

Definition— Initial value problem

An ordinary differential equation y′=f(t,y)y' = f(t, y) together with a starting value y(t0)=y0y(t_0) = y_0. y may be a vector, so a system of equations, and any higher-order equation can be rewritten as a first-order system by adding derivatives as extra unknowns.

Worked examplesH

Roots and minima

matlab
% 1. Polynomial roots: x^3 - 2x - 5 = 0, coefficients from the highest power.
c = [1 0 -2 -5];
r = roots(c);
disp(r)
xr = r(abs(imag(r)) < 1e-12);
fprintf('real root %.10f, check %.1e\n', xr, polyval(c, xr));

% 2. fzero: Kepler's equation M = E - e*sin(E) for an orbit of eccentricity 0.967.
e = 0.967;
kepler = @(E, M) E - e * sin(E) - M;
for M = [0.1 1 3]
  E = fzero(@(E) kepler(E, M), [0 2*pi]);
  fprintf('mean anomaly %.1f -> eccentric anomaly %.6f\n', M, E);
end

% fzero from a single starting guess searches for a sign change first.
fprintf('cos(x) = x at %.10f\n', fzero(@(x) cos(x) - x, 1));

% 3. fminbnd: the 330 ml can with the least metal (radius r in cm).
V = 330;
area = @(r) 2*pi*r.^2 + 2*V ./ r;
[rBest, aBest] = fminbnd(area, 1, 10);
fprintf('best radius %.4f cm (calculus says %.4f), area %.2f cm^2\n', rBest, (V/(2*pi))^(1/3), aBest);

% 4. fminsearch: Rosenbrock's banana-shaped valley, minimum at (1, 1).
rosen = @(p) (1 - p(1))^2 + 100 * (p(2) - p(1)^2)^2;
opts = optimset('TolX', 1e-10, 'TolFun', 1e-10);
[p, fval] = fminsearch(rosen, [-1.2 1], opts);
fprintf('minimum near (%.6f, %.6f), value %.1e\n', p, fval);
text
   2.0946 +      0i
  -1.0473 + 1.1359i
  -1.0473 - 1.1359i
real root 2.0945514815, check 2.0e-14
mean anomaly 0.1 -> eccentric anomaly 0.780228
mean anomaly 1.0 -> eccentric anomaly 1.911437
mean anomaly 3.0 -> eccentric anomaly 3.069578
cos(x) = x at 0.7390851332
best radius 3.7450 cm (calculus says 3.7449), area 264.36 cm^2
minimum near (1.000000, 1.000000), value 5.0e-22
  1. roots takes a polynomial's coefficients and returns all its roots, complex ones included. It does so with linear algebra from the last lesson: the roots are the eigenvalues of a companion matrix built from the coefficients. This cubic is the one Isaac Newton used to demonstrate his method; it has one real root and a complex pair. polyval evaluates the polynomial, confirming the root to rounding level. (MATLAB prints the complex column as 2.0946 + 0.0000i; only the layout differs.)
  2. fzero finds a zero of any continuous function of one variable. Given a bracket, an interval where the function changes sign, it is guaranteed to succeed: it combines bisection, which always works, with faster interpolation steps. (MATLAB's version follows the classic algorithm of Dekker and Brent; Octave's follows a later refinement by Alefeld, Potra and Shi.) Kepler's equation links time to position along an elliptical orbit; with an eccentricity of 0.967, roughly that of Halley's comet, it is far from linear, and fzero handles it without fuss. fzero needs a function of one variable, so the anonymous function @(E) kepler(E, M) fixes M, capturing its current value as you learned in lesson 4. With a single starting point instead of a bracket, fzero first searches outwards for a sign change, which usually works but is not guaranteed.
  3. fminbnd finds a minimum of a function of one variable on an interval. The answer, 3.7450 cm, differs from the calculus answer in the fourth decimal because fminbnd stops once it knows the minimiser to within its default TolX of 1e-4. That is not an error, it is the accuracy you asked for by not asking. (The best can is exactly as tall as it is wide; real cans are taller, for reasons of handling and labels.)
  4. fminsearch minimises a function of several variables with the Nelder–Mead simplex method, which needs no derivatives. Rosenbrock's function has its minimum at the bottom of a long, curved, flat valley, a classic stress test for optimisers. Tightened tolerances, set with optimset, get the answer to six decimals.

For systems of nonlinear equations there is fsolve. It is built into Octave but belongs to the Optimization Toolbox in MATLAB, a reminder that "it works in Octave" does not always mean "it works in every MATLAB installation". Use optimset for options: it works in both programs, whereas MATLAB's newer optimoptions does not exist in Octave 11.

Worked examplesH

Integrals of functions and of data

There are two different situations. When you can evaluate the integrand anywhere, use integral, which chooses its own points adaptively. When you only have measurements at fixed points, use trapz, which joins them with straight lines.

matlab
% 1. integral: adaptive quadrature of a function, even over infinite ranges.
fprintf('%.10f  (sqrt(pi) = %.10f)\n', integral(@(x) exp(-x.^2), -Inf, Inf), sqrt(pi));
% 1/sqrt(x) is infinite at 0, but the area under it from 0 to 1 is exactly 2.
fprintf('%.10f  default tolerances\n', integral(@(x) 1 ./ sqrt(x), 0, 1));
fprintf('%.10f  RelTol 1e-10\n', integral(@(x) 1 ./ sqrt(x), 0, 1, 'RelTol', 1e-10));

% 2. trapz and cumtrapz: data sampled at known times.
% A solar panel's power output (kW), measured every hour from 06:00 to 18:00.
hours = 6:18;
power = [0 0.3 1.1 2.0 2.8 3.3 3.5 3.4 3.0 2.3 1.4 0.5 0];
energy = trapz(hours, power);
fprintf('energy over the day: %.2f kWh\n', energy);
soFar = cumtrapz(hours, power);
fprintf('half of it was produced by %d:00\n', hours(find(soFar >= energy / 2, 1)));

% 3. How the trapezoid rule's error shrinks: integral of sin from 0 to pi is 2.
fprintf('%5s %12s %8s\n', 'n', 'error', 'ratio');
previous = NaN;
for n = [10 20 40 80 160]
  x = linspace(0, pi, n + 1);
  err = abs(trapz(x, sin(x)) - 2);
  fprintf('%5d %12.3e %8.2f\n', n, err, previous / err);
  previous = err;
end
text
1.7724538509  (sqrt(pi) = 1.7724538509)
1.9999998579  default tolerances
2.0000000000  RelTol 1e-10
energy over the day: 23.60 kWh
half of it was produced by 13:00
    n        error    ratio
   10    1.648e-02      NaN
   20    4.114e-03     4.00
   40    1.028e-03     4.00
   80    2.570e-04     4.00
  160    6.426e-05     4.00
  • integral uses adaptive Gauss–Kronrod quadrature: it evaluates the function at cleverly chosen points on each piece of the interval, estimates the error by comparing two rules, and subdivides only the pieces where the estimate is too large. It accepts infinite limits, and the bell-curve integral matches π\sqrt{\pi} to every digit shown.
  • The integrable spike of 1/x1/\sqrt{x} at zero is harder. With the default tolerances (a relative tolerance of 1e-6 and an absolute tolerance of 1e-10, the same in both programs) Octave's answer is correct to about seven digits, exactly the accuracy requested. Asking for more with 'RelTol' gets more. MATLAB's implementation differs in its details and may print other trailing digits; both are right to within the tolerance.
  • The integrand must accept a vector of points and return a vector of values, so write it with .*, ./ and .^. Forgetting the dot is the most common integral bug.
  • trapz(x, y) integrates sampled data: here power in kilowatts over hours gives energy in kilowatt-hours. cumtrapz returns the running integral, which answers questions like "by when was half the energy produced?".
  • The table shows the trapezoid rule is second order: doubling the number of intervals divides the error by exactly 4. Measuring this ratio is the standard way to check that a numerical method is working as designed, the same check the sparse rod made in the last lesson.

Double and triple integrals have integral2 and integral3, available in both programs.

Worked examplesH

Differential equations with ode45

[t, y] = ode45(f, tspan, y0) solves y′=f(t,y)y' = f(t, y) from y(t0)=y0y(t_0) = y_0. The function f takes a scalar time and a column vector state and returns a column vector of derivatives. If tspan has two elements, [t0 tEnd], the solver returns the points it chose itself; if it has more, it returns the solution at exactly those times. Each row of the output y is the state at the matching time in t.

matlab
% 1. One equation: Newton's law of cooling for a cup of tea in a 20 C room.
k = 0.08;                                  % per minute
cooling = @(t, T) -k * (T - 20);
[t, T] = ode45(cooling, [0 5 10 20 30], 90);
exact = 20 + 70 * exp(-k * t);
fprintf('%4s %9s %9s %10s\n', 't', 'ode45', 'exact', 'error');
fprintf('%4d %9.4f %9.4f %10.2e\n', [t, T, exact, abs(T - exact)]');

% 2. A system: the SIR epidemic model, as fractions of the population.
beta = 0.3;                                % infections per infectious person per day
gamma = 0.1;                               % recoveries per day: 10 days ill on average
sir = @(t, y) [-beta * y(1) * y(2)
                beta * y(1) * y(2) - gamma * y(2)
                gamma * y(2)];
opts = odeset('RelTol', 1e-8, 'AbsTol', 1e-10);
[t, y] = ode45(sir, 0:0.5:300, [0.999; 0.001; 0], opts);
[peak, k] = max(y(:, 2));
fprintf('R0 = %.1f; infections peak at day %.1f with %.1f%% ill at once\n', beta / gamma, t(k), 100 * peak);
fprintf('after 300 days: %.2f%% were never infected\n', 100 * y(end, 1));
fprintf('S + I + R stays %.12f\n', sum(y(end, :)));

% The final size also solves r = 1 - s0*exp(-R0*r): a root-finding problem.
R0 = beta / gamma;
rFinal = fzero(@(r) r - (1 - 0.999 * exp(-R0 * r)), [0.5 1]);
fprintf('final-size equation: %.2f%% infected in total; ODE: %.2f%%\n', 100 * rFinal, 100 * y(end, 3));
text
   t     ode45     exact      error
   0   90.0000   90.0000   0.00e+00
   5   66.9224   66.9224   1.11e-07
  10   51.4530   51.4530   1.64e-06
  20   34.1328   34.1328   4.18e-06
  30   26.3503   26.3503   3.73e-06
R0 = 3.0; infections peak at day 38.5 with 30.1% ill at once
after 300 days: 5.94% were never infected
S + I + R stays 1.000000000000
final-size equation: 94.06% infected in total; ODE: 94.06%
  • The cooling equation has an exact solution, so the program can measure the error: a few millionths of a degree with the default tolerances (RelTol 1e-3, AbsTol 1e-6). Solving a problem with a known answer first is the quickest way to make sure you have set a solver up correctly.
  • The SIR model splits a population into susceptible, infectious and recovered fractions. It is three coupled equations, written as one anonymous function returning a 3-by-1 column (the line breaks inside the brackets start new rows). The output y has one column per variable, so y(:, 2) is the infectious fraction over time.
  • Two independent checks are built in. The three equations' right-hand sides add up to zero, so S + I + R must stay 1, and it does to twelve decimals. And epidemiology offers a separate equation for the final size of an epidemic, solved here with fzero; it agrees with the ODE to the digits shown.
  • The model's numbers (a basic reproduction number R0R_0 of 3, ten days of illness) are illustrative, not a forecast of any real disease. Real epidemic models add age groups, contacts, vaccination and much more, and are solved with exactly these tools.

Inside, ode45 is the Dormand–Prince method, an explicit Runge–Kutta pair: at each step it computes two estimates of orders 4 and 5 from six function evaluations, uses their difference as an error estimate, and shrinks or grows the step to keep that estimate within tolerance. ode23 is a cheaper, lower-order pair, useful with loose tolerances. Octave implements the same methods as MATLAB, but the two programs make slightly different choices about step sizes, so their results agree to within the tolerances rather than digit for digit. MATLAB's deval, which evaluates a solution structure at new times, is not available in Octave 11; ask for the times you need in tspan instead.

Worked examplesH

Stiff problems

Some equations mix very fast and very slow behaviour: a chemical reaction where one species reacts in microseconds and another over hours, or a circuit with a tiny capacitor. Such problems are called stiff. An explicit method such as ode45 must take tiny steps throughout, not for accuracy but because larger steps make it numerically unstable. Implicit methods solve a small system of equations at every step (with the linear algebra of the last lesson) and can take steps as large as accuracy allows.

matlab
% A stiff equation: y is pulled very hard (rate 1000) towards cos(t).
f = @(t, y) -1000 * (y - cos(t));
exact = (1e6 * cos(2) + 1000 * sin(2)) / (1e6 + 1);   % the true y(2)
opts = odeset('Refine', 1);                           % one output per step
[t1, y1] = ode45(f, [0 2], 1, opts);
[t2, y2] = ode15s(f, [0 2], 1, opts);
[t3, y3] = ode23s(f, [0 2], 1, opts);
fprintf('ode45:  %4d steps, error at t = 2: %.1e\n', numel(t1) - 1, abs(y1(end) - exact));
fprintf('ode15s: %4d steps, error at t = 2: %.1e\n', numel(t2) - 1, abs(y2(end) - exact));
fprintf('ode23s: %4d steps, error at t = 2: %.1e\n', numel(t3) - 1, abs(y3(end) - exact));
fprintf('average ode45 step %.2e\n', mean(diff(t1)));
text
ode45:   609 steps, error at t = 2: 5.3e-05
ode15s:   21 steps, error at t = 2: 2.6e-06
ode23s:   57 steps, error at t = 2: 1.3e-05
average ode45 step 3.28e-03

The solution simply follows cos⁡t\cos t smoothly, yet ode45 needs about 30 times as many steps as ode15s. Its average step, 0.0033, is no accident: it sits right at the stability limit, about 3.3/1000, that the rate 1000 imposes on this explicit method, however smooth the solution. (Refine is set to 1 because by default ode45 reports four output points per step.) The rule of thumb in both programs: start with ode45; if it is unexpectedly slow, the problem is probably stiff, so try ode15s, or ode23s with loose tolerances. Step counts like these vary between programs and versions; the large ratio does not.

One Octave-specific note. Octave's ode15s is built on the SUNDIALS library, and in testing this lesson it failed to take its first step when the same problem started from y(0) = 0 (an IDASolve failed error), where the solution changes extremely fast at the start. Passing odeset('InitialStep', 1e-6) fixed it.

ComputationC

Worked example: how far does a ball fly?

A school physics book says a ball launched at 30 m/s flies furthest at 45 degrees, landing v02/gv_0^2/g = 91.7 m away. That ignores air. With air drag proportional to the square of the speed there is no formula, so here is the numerical answer. The program combines a second-order system written as four first-order equations, an event function that stops the solver when the ball lands, and fminbnd wrapped around the ODE solver to find the best angle:

matlab
1;

function dy = flight(~, y, c)
  % FLIGHT  A ball with quadratic air drag. State y = [x; height; vx; vy];
  % c is the drag coefficient per unit mass, in 1/m.
  g = 9.81;
  speed = hypot(y(3), y(4));
  dy = [y(3)
        y(4)
        -c * speed * y(3)
        -g - c * speed * y(4)];
end

function [value, isterminal, direction] = landed(~, y)
  % LANDED  Event function: height crosses zero while falling.
  value = y(2);
  isterminal = 1;      % stop the integration there
  direction = -1;      % only count downward crossings
end

function [range, duration] = throwBall(angle, speed, c)
  % THROWBALL  Range (m) and flight time (s) for a launch angle in degrees.
  y0 = [0; 0; speed * cosd(angle); speed * sind(angle)];
  opts = odeset('Events', @landed, 'RelTol', 1e-8, 'AbsTol', 1e-10);
  [~, ~, te, ye] = ode45(@(t, y) flight(t, y, c), [0 60], y0, opts);
  range = ye(end, 1);
  duration = te(end);
end

v0 = 30;          % launch speed, m/s
cDrag = 0.005;    % drag per unit mass, 1/m: a modest amount
fprintf('%5s %10s %10s %10s\n', 'angle', 'vacuum', 'with drag', 'flight');
for angle = [30 40 45 50]
  rVac = throwBall(angle, v0, 0);
  [rDrag, tDrag] = throwBall(angle, v0, cDrag);
  fprintf('%5d %9.2fm %9.2fm %9.2fs\n', angle, rVac, rDrag, tDrag);
end
fprintf('textbook vacuum range at 45 degrees: %.2f m\n', v0^2 / 9.81);

% Optimise through the ODE solver: the best launch angle with drag.
best = fminbnd(@(a) -throwBall(a, v0, cDrag), 20, 60, optimset('TolX', 1e-6));
fprintf('best angle with drag: %.2f degrees, range %.2f m\n', best, throwBall(best, v0, cDrag));
text
angle     vacuum  with drag     flight
   30     79.45m     62.75m      2.86s
   40     90.35m     68.50m      3.63s
   45     91.74m     68.60m      3.98s
   50     90.35m     66.90m      4.29s
textbook vacuum range at 45 degrees: 91.74 m
best angle with drag: 42.77 degrees, range 68.78 m

How it fits together:

  • First-order form. Newton's second law is second order in position. Making the velocities part of the state, y = [x; height; vx; vy], turns it into four first-order equations: the derivative of position is velocity, and the derivative of velocity is the acceleration from gravity and drag. This is how every mechanical system is fed to an ODE solver.
  • Events. We do not know in advance when the ball lands, so the program integrates up to a generous 60 seconds and lets the event function stop it. The solver watches value (the height) for a sign change in the given direction, locates the crossing precisely, and, because isterminal is 1, stops there. The extra outputs te and ye are the time and state at the event. Octave's ODE solvers support events with the same interface as MATLAB's.
  • A check first. With c = 0 the solver reproduces the textbook formula to the centimetre, and ranges at 40 and 50 degrees are equal, as the vacuum theory predicts. Only then are the drag results worth believing.
  • Solvers inside solvers. fminbnd minimises the negative range, and every evaluation it makes runs a whole ODE solution. Note that throwBall returns two outputs, but inside the anonymous function only the first is used. Drag lowers the best angle below 45 degrees, here to about 42.8, a real effect known to anyone who has thrown things outdoors.
  • The local functions use ~ for the time argument they ignore, and hypot(a, b), which computes a2+b2\sqrt{a^2 + b^2} without overflow.
ExplanationH

How practitioners work with solvers

Trust, but verify. Professional numerical work always includes checks: a version of the problem with a known solution, a conserved quantity, a second solver or tolerance that gives the same digits, or a convergence table. When a solver gives a surprising answer, halve the tolerance and run it again; if the answer moves, it was not converged.

Set tolerances on purpose. Defaults are chosen for speed with moderate accuracy (ode45 aims for about three significant digits by default). A result you will publish or build on deserves tighter tolerances and a check that tightening them further changes nothing that matters. Remember that AbsTol matters when a solution component passes through or near zero.

Write vectorised, side-effect-free functions. Solvers may call your function thousands of times, at points of their choosing, in any order. It should depend only on its inputs, never print, and, for integral and plotting, accept arrays.

Know what you are calling. fzero needs a sign change, fminsearch finds a local minimum near its starting point, and fminbnd assumes one minimum in the interval. Starting an optimiser from several points is cheap insurance.

Do not write your own Runge–Kutta for production work, but do write one once (the exercises of a numerical analysis course will ask you to), because seeing a fixed-step method go unstable on a stiff problem is the best way to understand why the library solvers work as they do. Interpolation (interp1, spline, pchip) and polynomial fitting (polyfit) complete the everyday toolkit and behave the same in both programs.

Common misconceptionsH

Common misconceptions

Misconception

"The solver's answer is exact to all the digits it prints." It is accurate to roughly the tolerance, which by default is far looser than the 16 digits a double can show. Print fewer digits, or tighten the tolerance until the digits you need stop changing.

Misconception

"ode45 returns the solution at evenly spaced times." With tspan = [t0 tEnd] it returns the times it chose itself (plus refined points), which are uneven. Pass a vector of times, such as 0:0.5:300, to get the solution exactly there.

Misconception

"A slow ODE solve means the problem needs more accuracy." Often it means the problem is stiff, and an explicit solver is being forced into tiny steps for stability. Switching to ode15s or ode23s can be a hundred times faster.

Misconception

"fminsearch finds the minimum." It finds a local minimum near its starting point, and may stop early on flat or badly scaled problems. Try several starting points and check the result.

Exercises

Exercises

1. Numerical methods: roots, integrals and ODEs

Warm-upComputational

What numbers does disp(sort(roots([1 -5 6]))') show? Write them separated by a single space.

2. Numerical methods: roots, integrals and ODEs

CoreComputational

With f = @(x) x.^2 - 4, which call finds the root 2 with a bracket that fzero accepts?

Choose one

3. Numerical methods: roots, integrals and ODEs

Warm-upComputational

What does disp(trapz(0:2:4, [1 3 2])) display?

4. Numerical methods: roots, integrals and ODEs

CoreHuman

You test a new integration routine. With step size h its error is 3.2e-3; with h/2 it is 2.0e-4; with h/4 it is 1.25e-5. What is the order of the method?

5. Numerical methods: roots, integrals and ODEs

CoreComputational

To solve the oscillator equation y'' = −y with ode45, you use the state y = [position; velocity]. Which right-hand side is correct?

Choose one

6. Numerical methods: roots, integrals and ODEs

CoreComputational

What does this display? Write the two numbers separated by a space.

[t, y] = ode45(@(t, y) -y, [0 0.5 1 1.5 2], [1; 2]);
disp(size(y))

7. Numerical methods: roots, integrals and ODEs

Warm-upComputational

integral(@(x) x^2, 0, 1) stops with an error in Octave (and in MATLAB). Why?

Choose one

8. Numerical methods: roots, integrals and ODEs

StretchHuman

A chemical kinetics model has reactions on time scales from microseconds to hours. ode45 takes millions of tiny steps, although the concentrations you plot change smoothly over hours. What is the best first move?

Choose one

Further reading and resources