Skip to content
LumenaryAcademy
Signals 60 min

Signals, systems and convolution

What a signal is, what makes a system linear and time-invariant, and how one measured response, the impulse response, predicts its answer to anything.

Code tested · 5AI-draftedAwaiting review by an electrical engineer

Your guides: Chloe · The Physicist, Margaret · The Bench Engineer, Gideon · The Design Reviewer

MotivationH

One measurement to predict everything

Clap your hands in a cathedral and you hear the clap, then a long, blurred tail of echoes. Speak a sentence in the same place and every syllable gets the same treatment. Here is a remarkable fact: if you record the clap carefully, you can predict, by calculation alone, exactly what any sentence, song or orchestra will sound like in that building, without ever playing them there. Audio engineers do this every day; it is called convolution reverb.

The same idea runs through all of electrical engineering. In Capacitors and inductors you found how an RC circuit responds to a step of voltage, and in Frequency response and filters how it responds to sinusoids. Those were two special inputs. This lesson shows that for a large and important class of systems, the linear time-invariant (LTI) ones, a single response, to an infinitely short kick called an impulse, determines the response to every possible input. The operation that turns "impulse response plus input" into "output" is convolution, and it is the first great tool of this level.

The rest of the Signals level builds on this foundation. Fourier analysis will show that convolution becomes simple multiplication when signals are written as sums of sinusoids; Sampling and the FFT moves everything onto computers; Digital filters designs impulse responses on purpose; and The Laplace transform and Feedback control use the same ideas to make systems stable and well behaved.

DefinitionsH

Signals: continuous, discrete, energy and power

A signal is a quantity that varies with time (or with position, as in an image). The voltage across a microphone, the current into a motor, the temperature of a furnace and the daily output of a solar farm are all signals.

Definition— Continuous-time and discrete-time signals

A continuous-time signal x(t)x(t) has a value for every real time tt. A discrete-time signal x[n]x[n] has a value only at integer indices nn, usually because a continuous signal was sampled every TT seconds, so that x[n]=x(nT)x[n] = x(nT). Parentheses mean continuous time and square brackets mean discrete time throughout Lumenary EE.

Signals in the physical world are continuous; signals inside computers are discrete (and also quantised to a finite number of levels, which Sampling and the FFT treats). Almost every idea in this lesson has a continuous version, with integrals, and a discrete version, with sums. We will move between them freely, and the code will always be discrete.

How "big" is a signal? Two measures are standard. They borrow their names from circuits: a voltage v(t)v(t) across a 1 Ω1\ \Omega resistor dissipates power v2v^2, so ∣x(t)∣2|x(t)|^2 is called the instantaneous power of a signal even when it is not a voltage.

Definition— Energy and average power

The energy of a signal is E=∫−∞∞∣x(t)∣2 dtorE=∑n=−∞∞∣x[n]∣2.E = \int_{-\infty}^{\infty} |x(t)|^2 \, dt \qquad \text{or} \qquad E = \sum_{n=-\infty}^{\infty} |x[n]|^2 . Its average power is the energy per unit time over a long interval, P=lim⁡T0→∞12T0∫−T0T0∣x(t)∣2 dt.P = \lim_{T_0 \to \infty} \frac{1}{2T_0} \int_{-T_0}^{T_0} |x(t)|^2 \, dt . An energy signal has finite energy (so zero average power): a clap, a pulse, a decaying transient. A power signal has finite, non-zero average power (so infinite energy): a sinusoid that goes on for ever, a steady DC voltage, sustained noise.

Example— A pulse, a decay and a sinusoid

A rectangular pulse of height 2 V lasting 3 ms has energy 22×0.003=0.012 V2 s2^2 \times 0.003 = 0.012\ \mathrm{V^2\,s}.

The decaying exponential x(t)=e−t/τx(t) = e^{-t/\tau} for t≥0t \ge 0 (and zero before) has energy ∫0∞e−2t/τ dt=τ/2\int_0^\infty e^{-2t/\tau}\,dt = \tau/2: finite, so it is an energy signal.

The sinusoid x(t)=Acos⁡(ωt+ϕ)x(t) = A\cos(\omega t + \phi) has ∣x∣2=A22(1+cos⁡(2ωt+2ϕ))|x|^2 = \tfrac{A^2}{2}\bigl(1 + \cos(2\omega t + 2\phi)\bigr). The second term averages to zero, so P=A2/2P = A^2/2. That is why the root-mean-square value of a sinusoid is A/2A/\sqrt 2, as you saw in AC and phasors.

The elementary signals

A handful of simple signals serve as building blocks, as test inputs, and as the vocabulary in which we describe systems.

tunit step u(t)1tunit impulse δ(t)area 1tdecaying exponential e−at u(t)1tsinusoid cos(ω₀t)
The elementary signals. The unit step switches on at t = 0 and stays on. The unit impulse is an idealised kick at t = 0 with area 1, drawn as an arrow. The decaying exponential is the natural response of a first-order circuit. The sinusoid is the steady AC signal of earlier lessons.
  • The unit step u(t)u(t) is 0 for t<0t < 0 and 1 for t≥0t \ge 0: a switch closing at time zero. Its discrete partner u[n]u[n] is 0 for n<0n < 0 and 1 for n≥0n \ge 0.
  • The unit impulse. In discrete time this is easy: δ[n]\delta[n] is 1 at n=0n = 0 and 0 everywhere else. In continuous time, δ(t)\delta(t) is the limit of a pulse of width ε\varepsilon and height 1/ε1/\varepsilon as ε→0\varepsilon \to 0: infinitely short, with area 1. Its defining property is the sifting property, ∫x(t) δ(t−t0) dt=x(t0)\int x(t)\,\delta(t - t_0)\,dt = x(t_0): integrating against a shifted impulse picks out one value. The step is the running integral of the impulse, u(t)=∫−∞tδ(τ) dτu(t) = \int_{-\infty}^t \delta(\tau)\,d\tau, and δ[n]=u[n]−u[n−1]\delta[n] = u[n] - u[n-1].
  • Exponentials este^{st}, where s=σ+jωs = \sigma + j\omega may be complex. With ss real they grow or decay; with s=jωs = j\omega they are the complex sinusoids ejωt=cos⁡ωt+jsin⁡ωte^{j\omega t} = \cos\omega t + j\sin\omega t of Euler's formula (see Complex numbers); in general they spiral, a sinusoid inside a growing or shrinking envelope. These turn out to be the most important signals of all, for reasons the last section of this lesson explains.
  • Sinusoids Acos⁡(ωt+ϕ)A\cos(\omega t + \phi), with amplitude AA, angular frequency ω=2πf\omega = 2\pi f and phase ϕ\phi. A real sinusoid is the sum of two complex ones: cos⁡ωt=12(ejωt+e−jωt)\cos\omega t = \tfrac12(e^{j\omega t} + e^{-j\omega t}).
DefinitionsH

Systems and their properties

A system takes an input signal and produces an output signal: y=T{x}y = T\{x\}. An RC low-pass filter takes a voltage and produces a voltage; a loudspeaker takes a current and produces air pressure; a moving-average program takes a list of daily temperatures and produces a smoother list. Five properties decide how much we can say about a system without knowing its insides.

Definition— Linear, time-invariant, causal, memoryless, stable

A system is

  • linear if it obeys superposition: for any inputs x1,x2x_1, x_2 and constants a,ba, b, the input ax1+bx2a x_1 + b x_2 produces the output ay1+by2a y_1 + b y_2. (Scaling and adding commute with the system.)
  • time-invariant if delaying the input delays the output by the same amount and changes nothing else: if x(t)x(t) gives y(t)y(t), then x(t−t0)x(t - t_0) gives y(t−t0)y(t - t_0). The system behaves the same on Tuesday as on Monday.
  • causal if the output at any time depends only on the input at that time and earlier. A real-time system cannot react to the future.
  • memoryless if the output at time tt depends only on the input at the same time tt. A resistor is memoryless; a capacitor, whose voltage depends on all the charge it has received, is not.
  • BIBO stable (bounded input, bounded output) if every bounded input, ∣x∣≤M|x| \le M for all time, produces a bounded output.

A system that is both linear and time-invariant is called LTI. To show that a property fails, one counterexample is enough. To show that it holds, you need an argument that covers every input. The program below does the first kind of work: for each candidate system it tries random inputs and reports whether the property survived.

python
import numpy as np

rng = np.random.default_rng(7)
N = 64

def delay(x, k):
    """Shift a signal right by k samples (x[n - k]), filling with zeros."""
    return np.concatenate([np.zeros(k), x[:len(x) - k]])

# Each system maps a whole input array x[0..N-1] to an output array.
systems = {
    "y[n] = 3 x[n]":              lambda x: 3 * x,
    "y[n] = 3 x[n] + 1":          lambda x: 3 * x + 1,
    "y[n] = x[n]^2":              lambda x: x ** 2,
    "y[n] = x[n] + 0.5 x[n-1]":   lambda x: x + 0.5 * delay(x, 1),
    "y[n] = n x[n]":              lambda x: np.arange(len(x)) * x,
    "y[n] = sum of x[0..n]":      lambda x: np.cumsum(x),
    "y[n] = x[n+1]":              lambda x: np.concatenate([x[1:], [0.0]]),
}

def linear(T):
    x1, x2 = rng.standard_normal(N), rng.standard_normal(N)
    a, b = 2.0, -0.7
    return np.allclose(T(a * x1 + b * x2), a * T(x1) + b * T(x2))

def time_invariant(T, k=5):
    x = np.concatenate([rng.standard_normal(N - 10), np.zeros(10)])  # room to shift
    return np.allclose(T(delay(x, k))[k:], delay(T(x), k)[k:])

def causal(T, n0=30):
    # Change the input only after time n0; a causal output cannot change before n0.
    x = rng.standard_normal(N)
    x2 = x.copy()
    x2[n0 + 1:] += rng.standard_normal(N - n0 - 1)
    return np.allclose(T(x)[:n0 + 1], T(x2)[:n0 + 1])

def memoryless(T, n0=30):
    # Change the input everywhere except at n0; a memoryless output at n0 cannot change.
    x = rng.standard_normal(N)
    x2 = x + rng.standard_normal(N)
    x2[n0] = x[n0]
    return np.isclose(T(x)[n0], T(x2)[n0])

yes = lambda b: "yes" if b else "no"
print(f"{'system':28} {'linear':>7} {'TI':>4} {'causal':>7} {'memoryless':>11}")
for name, T in systems.items():
    print(f"{name:28} {yes(linear(T)):>7} {yes(time_invariant(T)):>4} "
          f"{yes(causal(T)):>7} {yes(memoryless(T)):>11}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Three results deserve comment.

y=3x+1y = 3x + 1 is not linear, even though its graph is a straight line. Feed it zero and it outputs 1; a linear system must map zero to zero, since T{0}=T{0⋅x}=0⋅T{x}=0T\{0\} = T\{0 \cdot x\} = 0 \cdot T\{x\} = 0. Engineers call such a system affine, and they make it linear by measuring signals from the operating point: this is exactly what the small-signal models of Transistor amplifiers do.

y[n]=n x[n]y[n] = n\,x[n] is linear but not time-invariant: its gain grows with the clock, so the same input applied later is amplified more. Time-varying systems are real and important (a mobile phone's radio channel changes as you walk), but they are much harder to analyse.

The accumulator y[n]=∑k≤nx[k]y[n] = \sum_{k \le n} x[k] is LTI but has memory, and, as we will see, it is not stable: a constant input of 1 produces an output that grows without bound. It is the discrete twin of a capacitor charged by a constant current.

IntuitionH

The impulse response and convolution

Now the central argument. Take any LTI system and suppose you have measured its impulse response: the output h[n]h[n] when the input is δ[n]\delta[n]. Here is how that one measurement predicts everything.

Step 1: every signal is a sum of shifted impulses. A discrete signal x[n]x[n] is a list of numbers, and each number can be written as a scaled, shifted impulse: the value x[k]x[k] sitting at time kk is x[k] δ[n−k]x[k]\,\delta[n - k]. So x[n]=∑k=−∞∞x[k] δ[n−k].x[n] = \sum_{k=-\infty}^{\infty} x[k]\, \delta[n - k]. This looks almost too obvious to write down, but it is the key.

Step 2: time invariance. The shifted impulse δ[n−k]\delta[n - k] produces the shifted response h[n−k]h[n - k].

Step 3: linearity. The scaled sum of impulses produces the same scaled sum of responses.

Theorem— LTI systems are convolutions

If an LTI system has impulse response h[n]h[n], then its response to any input x[n]x[n] is the convolution y[n]=(x∗h)[n]=∑k=−∞∞x[k] h[n−k].y[n] = (x * h)[n] = \sum_{k=-\infty}^{\infty} x[k]\, h[n - k]. In continuous time, y(t)=(x∗h)(t)=∫−∞∞x(τ) h(t−τ) dτy(t) = (x * h)(t) = \int_{-\infty}^{\infty} x(\tau)\, h(t - \tau)\, d\tau.

Proof

Write x[n]=∑kx[k] δ[n−k]x[n] = \sum_k x[k]\,\delta[n - k]. By time invariance, T{δ[n−k]}=h[n−k]T\{\delta[n - k]\} = h[n - k]. By linearity, applied to this sum of scaled inputs, T{x}[n]=∑kx[k] T{δ[n−k]}=∑kx[k] h[n−k]T\{x\}[n] = \sum_k x[k]\, T\{\delta[n-k]\} = \sum_k x[k]\, h[n - k]. (For infinite sums, one also needs the system to be continuous in a suitable sense, which every physical system we meet is.) The continuous version follows the same way from the sifting property x(t)=∫x(τ) δ(t−τ) dτx(t) = \int x(\tau)\,\delta(t - \tau)\,d\tau, which writes xx as a continuous "sum" of impulses.

∎

There are two ways to read the formula, and both are worth having.

The input view: every sample launches an echo. Each input sample x[k]x[k] starts its own copy of the impulse response, scaled by x[k]x[k] and starting at time kk; the output is all those copies added up. This is the cathedral: every syllable you speak sets off its own tail of reverberation, and you hear them overlapping.

x[0]·h[n] = 1·h[n]10.50.25x[1]·h[n−1] = 2·h[n−1]210.5x[2]·h[n−2] = 3·h[n−2]31.50.75sum: y[n]12.54.2520.75n = 012345
Convolving x = [1, 2, 3] with h = [1, 0.5, 0.25]. Each input sample launches a copy of h, scaled by the sample and shifted to start at its time (red). The output is their sum, column by column (blue).

The output view: flip and slide. To compute one output sample y[n]y[n], flip hh back to front, slide it so that its origin lies at nn, multiply it point by point with xx, and add. The flipped hh is a window that says how much each past input still matters now: h[0]h[0] weighs the present input, h[1]h[1] the input one step ago, and so on.

Example— A convolution by hand

Let x=[1,2,3]x = [1, 2, 3] and h=[1,0.5,0.25]h = [1, 0.5, 0.25], both starting at n=0n = 0. Using y[n]=∑kx[k] h[n−k]y[n] = \sum_k x[k]\,h[n-k]:

  • y[0]=x[0]h[0]=1y[0] = x[0]h[0] = 1
  • y[1]=x[0]h[1]+x[1]h[0]=0.5+2=2.5y[1] = x[0]h[1] + x[1]h[0] = 0.5 + 2 = 2.5
  • y[2]=x[0]h[2]+x[1]h[1]+x[2]h[0]=0.25+1+3=4.25y[2] = x[0]h[2] + x[1]h[1] + x[2]h[0] = 0.25 + 1 + 3 = 4.25
  • y[3]=x[1]h[2]+x[2]h[1]=0.5+1.5=2y[3] = x[1]h[2] + x[2]h[1] = 0.5 + 1.5 = 2
  • y[4]=x[2]h[2]=0.75y[4] = x[2]h[2] = 0.75

Notice the pattern: the indices in every product add up to nn. A signal of length NN convolved with one of length MM has length N+M−1N + M - 1, here 3+3−1=53 + 3 - 1 = 5. If you have ever multiplied polynomials, this is the same calculation: (1+2z+3z2)(1+0.5z+0.25z2)=1+2.5z+4.25z2+2z3+0.75z4(1 + 2z + 3z^2)(1 + 0.5z + 0.25z^2) = 1 + 2.5z + 4.25z^2 + 2z^3 + 0.75z^4. That is no coincidence, and Digital filters builds the z-transform on it.

The code below implements convolution in the input view, literally "add a scaled, shifted copy of hh for each input sample", and checks it against NumPy's np.convolve. It then models a room with a direct path and two echoes and checks the algebraic properties of convolution.

python
import numpy as np

def convolve(x, h):
    """y[n] = sum over k of x[k] h[n - k]: each input sample launches a scaled, shifted copy of h."""
    y = np.zeros(len(x) + len(h) - 1)
    for k, xk in enumerate(x):          # input sample x[k] arrives at time k ...
        for m, hm in enumerate(h):      # ... and its echo h[m] lands at time k + m
            y[k + m] += xk * hm
    return y

x = np.array([1.0, 2.0, 3.0])
h = np.array([1.0, 0.5, 0.25])
print("x       =", x)
print("h       =", h)
print("by hand =", convolve(x, h))
print("numpy   =", np.convolve(x, h))

# A room that answers a clap with the clap itself plus two fainter echoes.
fs = 1000                                   # samples per second
h_room = np.zeros(301)
h_room[0], h_room[120], h_room[300] = 1.0, 0.6, 0.3     # echoes after 120 ms and 300 ms
word = np.sin(2 * np.pi * 50 * np.arange(80) / fs) * np.hanning(80)   # an 80 ms "word"
heard = convolve(word, h_room)
print(f"\nword: {len(word)} samples, room: {len(h_room)} samples, heard: {len(heard)} samples")
print("agrees with np.convolve:", np.allclose(heard, np.convolve(word, h_room)))

# Commutative, associative, distributive -- checked on random signals.
rng = np.random.default_rng(1)
a, b, c = rng.standard_normal(20), rng.standard_normal(7), rng.standard_normal(7)
print("\nx*h == h*x:            ", np.allclose(convolve(a, b), convolve(b, a)))
print("(x*h1)*h2 == x*(h1*h2):", np.allclose(convolve(convolve(a, b), c), convolve(a, convolve(b, c))))
print("x*(h1+h2) == x*h1+x*h2:", np.allclose(convolve(a, b + c), convolve(a, b) + convolve(a, c)))
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The three algebraic properties have physical meanings. Commutativity says the input and the system can swap roles. Associativity says two systems in cascade (one feeding the next) act like one system whose impulse response is h1∗h2h_1 * h_2, and so the order of a cascade of LTI systems does not matter. Distributivity says two systems in parallel, with their outputs added, act like one system with impulse response h1+h2h_1 + h_2. Together they make block diagrams of LTI systems into algebra, which Feedback control uses constantly.

ComputationC

A circuit as a convolution

Convolution is not only for sampled signals. Take the RC low-pass filter from Capacitors and inductors: input voltage x(t)x(t) through a resistor RR into a capacitor CC, output y(t)y(t) across the capacitor. Its differential equation is RC y′(t)+y(t)=x(t)RC\,y'(t) + y(t) = x(t). With τ=RC\tau = RC, its impulse response, the output when the input is δ(t)\delta(t) and the capacitor starts uncharged, is h(t)=1τ e−t/τ u(t).h(t) = \frac{1}{\tau}\, e^{-t/\tau}\, u(t). (The impulse dumps a charge onto the capacitor instantly, raising its voltage to 1/τ1/\tau; then it discharges through the resistor.) Its step response, which you derived earlier by solving the differential equation, is 1−e−t/τ1 - e^{-t/\tau} for t≥0t \ge 0. Convolution should give the same answer: y(t)=∫−∞∞u(σ) h(t−σ) dσ=∫0t1τe−(t−σ)/τ dσ=1−e−t/τ,t≥0.y(t) = \int_{-\infty}^{\infty} u(\sigma)\, h(t - \sigma)\, d\sigma = \int_0^t \frac{1}{\tau} e^{-(t-\sigma)/\tau}\, d\sigma = 1 - e^{-t/\tau}, \qquad t \ge 0 . It does. Computers do the integral as a sum: sample both signals every Δt\Delta t seconds and multiply the discrete convolution by Δt\Delta t, the width of each slice of the integral. The program uses R=10 kΩR = 10\ \mathrm{k\Omega} and C=100 nFC = 100\ \mathrm{nF}, so τ=1\tau = 1 ms.

python
import numpy as np

R, C = 10e3, 100e-9           # 10 kilohms and 100 nanofarads
tau = R * C                    # time constant: 1 ms

def step_response_by_convolution(dt, t_end=5e-3):
    t = np.arange(0, t_end, dt)
    h = np.exp(-t / tau) / tau            # impulse response of the RC low-pass, in 1/s
    u = np.ones_like(t)                   # a 1 V step applied at t = 0
    y = np.convolve(u, h)[:len(t)] * dt   # the integral, approximated by a sum
    return t, y

print("time step    largest error vs 1 - exp(-t/tau)")
for dt in (1e-4, 1e-5, 1e-6):
    t, y = step_response_by_convolution(dt)
    exact = 1 - np.exp(-t / tau)
    print(f"{dt * 1e6:6.0f} us    {np.max(np.abs(y - exact)):.4f} V")

t, y = step_response_by_convolution(1e-5)
print("\nstep response (dt = 10 us), one row per 0.25 ms:")
for i in range(0, 401, 25):
    bar = "#" * int(round(40 * y[i]))
    print(f"t = {t[i] * 1e3:4.2f} ms  {y[i]:.3f} V  {bar}")

area = np.sum(np.abs(np.exp(-t / tau) / tau)) * 1e-5
print(f"\nintegral of |h| over 5 ms: {area:.3f}  (finite, so the circuit is BIBO stable)")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The error shrinks in proportion to Δt\Delta t (0.1, 0.01, 0.001 V for steps of 100, 10 and 1 μs): the sum is a rectangle-rule approximation of the integral, and its worst error is at t=0t = 0, where it already counts one full slice Δt/τ\Delta t/\tau of the impulse response. The curve is the familiar charging shape, reaching 63.9% (exactly 1−e−1≈63.2%1 - e^{-1} \approx 63.2\%, plus the discretisation error) after one time constant.

Stability from the impulse response

The last line of the output connects to stability. A convolution adds up the past input weighted by hh, so if the weights have finite total size, a bounded input cannot produce an unbounded output.

Theorem— BIBO stability of an LTI system

An LTI system is BIBO stable if and only if its impulse response is absolutely summable (in continuous time, absolutely integrable): ∑n=−∞∞∣h[n]∣<∞or∫−∞∞∣h(t)∣ dt<∞.\sum_{n=-\infty}^{\infty} |h[n]| < \infty \qquad \text{or} \qquad \int_{-\infty}^{\infty} |h(t)|\,dt < \infty .

Proof

If: suppose ∣x[n]∣≤M|x[n]| \le M for all nn and S=∑k∣h[k]∣S = \sum_k |h[k]| is finite. Then ∣y[n]∣=∣∑kh[k] x[n−k]∣≤∑k∣h[k]∣ M=MS|y[n]| = \bigl|\sum_k h[k]\, x[n-k]\bigr| \le \sum_k |h[k]|\,M = MS, a bound that holds for every nn.

Only if: suppose ∑k∣h[k]∣\sum_k |h[k]| is infinite. Choose the bounded input x[k]=sign⁡(h[−k])x[k] = \operatorname{sign}(h[-k]), which is ±1\pm 1 or 0. Then y[0]=∑kx[k] h[−k]=∑k∣h[−k]∣y[0] = \sum_k x[k]\, h[-k] = \sum_k |h[-k]|, which is infinite, so this bounded input does not give a bounded output.

∎

The RC circuit's h(t)=e−t/τ/τh(t) = e^{-t/\tau}/\tau integrates to exactly 1, so it is stable. The accumulator's impulse response is u[n]u[n], whose sum diverges: unstable, as the constant input showed. And here is a subtler case: h[n]=1/(n+1)h[n] = 1/(n+1) for n≥0n \ge 0 dies away to zero, yet its sum is the harmonic series, which diverges. An impulse response that fades out is not enough; it must fade fast enough.

ExplanationH

Why LTI systems are so central

Most systems are not exactly linear or time-invariant: amplifiers clip, components age, motors saturate. So why does so much of electrical engineering rest on LTI theory? Four reasons.

1. One measurement characterises the system. You have just seen this: hh predicts the response to anything. For a system you cannot open up, a room, a cable, a vibrating bridge, a radio channel, measuring hh is usually the first thing an engineer does.

2. Many real systems are LTI, or nearly so. Any circuit built from resistors, capacitors, inductors and linear controlled sources, starting from rest, is exactly LTI, because Kirchhoff's laws and the component equations are linear with constant coefficients. Nonlinear systems, such as transistors, are close to linear for small deviations around an operating point, which is how amplifier design works. And many systems change slowly enough to be treated as time-invariant over the interval that matters.

3. Complex exponentials pass straight through. Feed x[n]=ejωnx[n] = e^{j\omega n} into the convolution sum: y[n]=∑kh[k] ejω(n−k)=(∑kh[k] e−jωk)ejωn=H(ejω) ejωn.y[n] = \sum_k h[k]\, e^{j\omega(n - k)} = \Bigl(\sum_k h[k]\, e^{-j\omega k}\Bigr) e^{j\omega n} = H(e^{j\omega})\, e^{j\omega n}. The output is the same complex exponential, multiplied by a single complex number HH that depends on the frequency but not on time. In the language of linear algebra, complex exponentials are eigenfunctions of every LTI system, and HH is the eigenvalue (see Linear algebra). This is the deep reason behind the phasor method of AC and phasors and the frequency response of Frequency response and filters, and it is where Fourier analysis begins.

4. LTI systems compose cleanly. Cascades, parallel paths and feedback loops of LTI systems are LTI, with impulse responses given by convolution and addition, and transfer functions given by multiplication. That is what makes block-diagram design possible.

Where convolution and impulse responses appear in practice

  • Acoustics and audio. Concert-hall designers and audio engineers measure room impulse responses, usually by playing a slow frequency sweep and processing the recording, and convolution reverbs apply them to dry recordings. Hearing-aid and headphone makers use measured responses of ears and heads.
  • Communications. A wireless signal reaches your phone along several paths (direct, off buildings, off the ground), so the channel acts like an impulse response with several echoes, much like the room in the code. Wi-Fi and 5G receivers estimate this response many times a second from known pilot symbols and undo it; Wireless systems returns to this.
  • Measurement and instrumentation. Every oscilloscope probe, sensor and anti-aliasing filter has an impulse response that slightly blurs what it measures. Seismologists model a seismogram as the earthquake's source signal convolved with the response of the Earth along the path and the response of the instrument.
  • Images. A blurred photograph is, to a good approximation, the sharp image convolved with a two-dimensional blur kernel; sharpening and edge detection are convolutions too. The "convolutional" layers of neural networks, covered in Lumenary CS's deep architectures, slide learned kernels across images in the same way (strictly, without the flip, so they compute a correlation).
Common misconceptionsH

Common misconceptions

Misconception

"A linear system is one whose graph is a straight line." Linearity means superposition, and it requires zero in to give zero out. y=3x+1y = 3x + 1 is affine, not linear. Conversely, y(t)=x(t−2)y(t) = x(t - 2) and y=∫xy = \int x have no "graph" at all, and both are linear.

Misconception

"Convolving two signals means multiplying them point by point." Point-by-point multiplication is a different operation (it is what an amplitude modulator or a window does). Convolution mixes every sample of one signal with many samples of the other, and its output is longer than either input: N+M−1N + M - 1 samples. The two operations are linked, though: Fourier analysis shows that convolution in time is multiplication in frequency, and the other way round.

Misconception

"Non-causal systems are impossible, so they are useless." A system that must react in real time has to be causal. But when the whole signal is already recorded (a photograph, an audio file, a day of sensor data), you may freely use "future" samples. Image filters are centred on the pixel they modify, and offline audio tools often filter forwards and then backwards to cancel phase shift.

Misconception

"If the impulse response dies away, the system is stable." It must die away fast enough for its absolute sum to be finite. h[n]=1/(n+1)h[n] = 1/(n+1) tends to zero, but the bounded input x=1x = 1 makes the output grow like ln⁡n\ln n for ever.

Misconception

"Real systems are never linear, so LTI theory is only an idealisation." It is an idealisation, but a controlled one: engineers deliberately keep signals within the range where a system is linear (that is what headroom on an audio mixer is for), and they measure how far from linear a system is (as harmonic distortion) when it matters. Where linearity fails badly, in power electronics or saturating motors, the LTI tools still work around each operating point.

History & global context

A short history

The idea that a response to a sum is the sum of responses goes back to the study of heat and vibration. In the 1830s the French mathematician Jean-Marie Duhamel showed how to build the solution of a heat problem with time-varying boundary conditions by adding up the responses to many small steps, the principle behind the convolution integral. In the 1880s and 1890s Oliver Heaviside, a self-taught English engineer working on telegraph cables, used the unit step and impulsive "shocks" freely to solve circuit equations, to the discomfort of the mathematicians of his day. Paul Dirac made the impulse, with its symbol δ\delta, a standard tool of physics in his quantum mechanics of the late 1920s, and in the late 1940s Laurent Schwartz's theory of distributions finally made it rigorous. The discrete-time theory grew with digital computers from the 1950s on, and it is now the form in which most signal processing is actually done.

Exercises

Exercises

1. Signals, systems and convolution

Warm-upHuman

Which of these systems are linear and time-invariant (LTI)? Select all that apply.

Choose all that apply

2. Signals, systems and convolution

Warm-upHuman

Let x=[2,1,−1]x = [2, 1, -1] and h=[1,3]h = [1, 3], both starting at n=0n = 0. Compute y=x∗hy = x * h by hand. What is y[1]y[1]?

3. Signals, systems and convolution

Warm-upHuman

A dry recording of 2.0 s, sampled at 48 kHz, is convolved with a room impulse response of 0.5 s at the same sample rate. How many samples does the full convolution have?

4. Signals, systems and convolution

CoreHuman

A voltage x(t)=3+4cos⁡(2π⋅50 t)x(t) = 3 + 4\cos(2\pi \cdot 50\,t) volts is applied across a 1 Ω1\ \Omega resistor. What is the average power dissipated, in watts?

5. Signals, systems and convolution

CoreHuman

Which of these discrete-time impulse responses (each zero for n<0n < 0) belong to BIBO-stable systems? Select all that apply.

Choose all that apply

6. Signals, systems and convolution

CoreComputational

Write convolve(x, h), returning the full convolution y[n]=∑kx[k] h[n−k]y[n] = \sum_k x[k]\,h[n-k] as a NumPy array of length len(x) + len(h) - 1. Write the loops yourself; the tests check your answer against np.convolve but block you from calling it.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

7. Signals, systems and convolution

StretchComputational

A recording has picked up an echo: y[n]=x[n]+a x[n−d]y[n] = x[n] + a\,x[n-d], with ∣a∣<1|a| < 1, delay d≥1d \ge 1 samples, and x[n]=0x[n] = 0 for n<0n < 0. In other words y=x∗hy = x * h with h=[1,0,…,0,a]h = [1, 0, \dots, 0, a] (aa at index dd). Write remove_echo(y, a, d) that recovers xx exactly from yy (same length as yy).

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Further reading and resources

  • MIT OpenCourseWare RES.6-007: Signals and Systems · Alan V. Oppenheim · 2011A complete free video course on continuous and discrete signals and systems.
  • numpy.convolve · NumPyThe library function this lesson checks its hand-written convolution against, with its "full", "same" and "valid" modes.
  • Think DSP · Allen B. DowneyA free book that teaches signal processing through Python programs.
  • Julius O. Smith III's online books · CCRMA, Stanford UniversityFree books on filters, spectral analysis and audio signal processing, written for engineers.