A question no program can answer
Here is a five-line program. Given a positive integer , it repeatedly halves if is even and replaces by if is odd, and it stops when it reaches .
while n != 1:
if n % 2 == 0: n = n // 2
else: n = 3 * n + 1
Does it stop for every starting value? This is the Collatz problem, and nobody knows. Computers have checked starting values into the quintillions, and every one has reached — but that is evidence, not proof.
A software company would love a tool that reads any program and its input and reports, correctly and always, whether the program eventually stops or runs forever. Such a tool would settle the Collatz problem in an afternoon, and Goldbach's conjecture too (write a program that searches for a counterexample and ask whether it stops). In 1936 Alan Turing proved that no such tool can exist — not because it would be slow, or because we have not been clever enough, but because its existence is contradictory.
Turing was answering a question of David Hilbert's: is there a mechanical procedure that decides, for every statement of first-order logic, whether it is valid? To answer "no", he first had to say precisely what a mechanical procedure is. His answer — the Turing machine — is the founding definition of computer science, and this lesson is about what it can and cannot do. It builds on algorithms and their correctness and on countability and Cantor's diagonal argument, which will reappear in a new guise.
Programs are data
Two simple observations drive the whole subject.
Programs can be written down. A program is a finite string of symbols, so programs can be listed: (say, in order of length and then alphabetically). In particular there are only countably many programs, while there are uncountably many functions . Before doing anything else, we already know that almost all functions cannot be computed by any program.
Programs can read programs. Compilers, interpreters and virus scanners take programs as input. In particular there is a single universal program that, given the text of any program and an input , simulates on . A Python interpreter is, in effect, such a program.
Put these together and something strange becomes possible: a program can be run on its own text. Picture an infinite table whose rows are the programs and whose columns are the inputs ; entry records whether halts on input . Cantor's diagonal argument walks down the diagonal of a table like this and builds a row that differs from every row. Here the diagonal row is a program — provided some program could read the table — and that will be the contradiction behind the halting problem.
Turing machines and what they compute
Definition— Turing machine
A (deterministic, one-tape) Turing machine consists of a finite set of states with a start state , a finite tape alphabet containing a blank symbol , and a transition function
The machine works on a tape of cells indexed by , all blank except finitely many, with a head scanning one cell. If it is in state scanning symbol and , it writes , moves the head one cell in direction , and enters state — or stops, if .
A configuration is a snapshot (state, tape contents, head position); one application of is a step. Started on an input written on the tape, the machine either halts after finitely many steps or runs forever. Many conventions differ in inessential ways (several tapes, a one-way infinite tape, accepting and rejecting states, larger alphabets); all of them compute the same functions.
Definition— computable function
Fix a way of writing natural numbers on the tape (in binary, say). A machine computes the partial function if, started on input , it halts with on the tape whenever is defined, and runs forever whenever is undefined. A partial function is (partial) computable if some machine computes it; a computable function is a total one. Write for the partial function computed by the -th machine in a fixed listing, if halts on and if it does not.
Definition— decidable and recursively enumerable sets
A set is decidable (or computable, recursive) if its characteristic function is computable: some machine halts on every input and answers correctly whether . It is recursively enumerable (r.e., semi-decidable, computably enumerable) if some machine halts exactly on the members of , i.e. for some . It is co-r.e. if its complement is r.e.
A semi-decider for says "yes" on members and stays silent forever on non-members — and silence never turns into an answer, because at no finite time can you tell "silent so far" from "silent forever". The same definitions apply to sets of strings, pairs or programs, via any reasonable coding as natural numbers.
Definition— the halting problem
The halting set is , where is a computable pairing of with . Its diagonal is . The halting problem is the question whether is decidable.
Two facts about the listing are used constantly. The universal machine theorem: there is a machine with for all (both sides defined and equal, or both undefined). Turing built one in 1936; a modern interpreter is one. Kleene's s-m-n theorem: there is a computable function such that — from a program and a value you can compute a program with that value "hard-wired" in. Both are straightforward but long to prove in full detail for Turing machines; any of the texts in the reading list does it.
Note
The Church–Turing thesis says that every function which is effectively calculable — computable by a finite, deterministic, mechanical procedure, with unlimited time and paper — is computable by a Turing machine. It is a thesis, not a theorem: "effectively calculable" is an informal notion, so the claim cannot be proved. The evidence for it is strong. Turing's 1936 analysis of what a human computer can do with pencil and paper leads to his machines; and every other model proposed — Church's λ-calculus, Gödel–Herbrand–Kleene general recursive functions, Post systems, register machines, cellular automata, every programming language — computes exactly the same class of functions, a fact that is proved, model by model.
Machines, decidable sets and undecidable ones
Example— A binary incrementer
One state, carry, with the head on the least significant bit:
| state | read | write | move | next |
|---|---|---|---|---|
| carry | 1 | 0 | L | carry |
| carry | 0 or blank | 1 | L | halt |
On (eleven) it turns the two trailing s into s, turns the into a and halts: (twelve), in steps. In general it takes steps, where is the number of trailing s. Although a single increment can take many steps ( takes ), counting from to takes steps in total — fewer than two per increment on average.
Example— A unary adder
Write as ( ones, a separator, ones). The machine walks right in state A over the first block; at the separator it writes and switches to state B; B walks right over the second block; at the blank after it, B steps back left into state C, and C erases the last and halts. The tape now holds , after steps.
Example— The two-state busy beaver
Started on a blank tape, the machine with states A, B and table
runs for steps and leaves four s on the tape. Tibor Radó (1962) defined the busy-beaver function : the largest number of steps taken by an -state, two-symbol machine that halts when started on a blank tape. There are only finitely many such machines, so is a perfectly well-defined number — but we shall see that is not computable. The machine above shows ; the experiment below proves by settling all two-state machines.
Example— Decidable sets
The primes, the even numbers, the set of such that is a perfect number, the set of (codes of) propositional tautologies, and the set of pairs such that the finite graph has a -colouring are all decidable: in each case a finite search answers the question. The set of such that machine has fewer than states is decidable — you only have to read the table, not run it. Decidability says nothing about speed; that is the subject of computational complexity.
Example— Recursively enumerable sets
is r.e.: run on and halt if the output is . The halting set is r.e.: run the universal machine. The set of theorems of Peano arithmetic is r.e.: enumerate all finite strings and halt when one is a proof of the given sentence. The set of polynomials with an integer root is r.e.: search through all integer tuples. We will see that is not decidable; the other three are not decidable either.
Example— A set that is not even r.e.
, the set of machines that do not halt on their own index, is not r.e. (Corollary below). Neither is (Exercise).
The halting problem, reductions and Rice's theorem
Theorem— Theorem 1 (most functions are not computable)
There are only countably many computable partial functions , but uncountably many functions . In particular, some total function is not computable.
Proof
A Turing machine is determined by its finite table, which can be written as a finite string over a fixed finite alphabet (name the states in binary). The set of finite strings over a finite alphabet is countable, so there are countably many machines, and each computes exactly one partial function.
On the other hand, given any sequence of functions , the function differs from at the input , so is not in the sequence: is uncountable. A countable set cannot contain an uncountable one, so some is computed by no machine.
∎
This proof is non-constructive: it does not name a non-computable function. The next theorem does.
Theorem— Theorem 2 (Turing, 1936: the halting problem is undecidable)
The set is not decidable. Consequently the halting set is not decidable.
Proof
Suppose some machine decides : it computes the total function with if and otherwise. Build a new machine which, on input , computes and then
- if , enters a loop that never halts;
- if , halts.
is a genuine Turing machine (it runs the machine for as a subroutine and then does one of two trivial things), so for some index . Now ask whether halts on its own index :
The first equivalence is the construction of ; the second is the correctness of . This is a contradiction, so no machine decides .
If were decidable, then so would be : to decide whether , compute the pair and ask the decider for .
∎
Note
Notice what the proof did not say. For each single machine , the question "does halt on ?" has an answer, and one of the two constant algorithms "always say yes" and "always say no" gives it. Undecidability means that no single algorithm answers correctly for all . Many individual instances are easy; some (like Collatz) may be hard; the theorem is about uniformity.
Theorem— Theorem 3 (the halting set is recursively enumerable)
and are r.e.
Proof
The universal machine halts on exactly when halts on , so it is a semi-decider for . For , first compute and then run the universal machine.
∎
Theorem— Theorem 4 (decidable = r.e. and co-r.e.)
A set is decidable if and only if both and its complement are recursively enumerable.
Proof
() If a machine decides , modify it to loop forever instead of answering "no": that semi-decides . Modify it to loop instead of answering "yes": that semi-decides .
() Let semi-decide and semi-decide . On input , simulate one step of on , then one step of on , then the next step of , and so on (dovetailing). Every lies in exactly one of and , so exactly one of the two simulations eventually halts, after finitely many rounds. Answer "yes" if halted and "no" if did. This procedure always halts and is always correct.
∎
This is often called Post's theorem. Note the importance of interleaving: running to completion first would hang forever on inputs outside .
Theorem— Corollary 5
is not recursively enumerable. So the class of r.e. sets is not closed under complement.
Proof
is r.e. (Theorem 3). If were r.e. too, would be decidable by Theorem 4, contradicting Theorem 2.
∎
The r.e. sets have a useful alternative description, which is how they appear in logic and number theory.
Theorem— Theorem 6 (r.e. = projection of decidable)
is r.e. if and only if there is a decidable relation with .
Proof
() If , let say " halts on input within steps". is decidable — simulate for steps — and iff for some .
() Given decidable , the machine that on input tests in turn and halts at the first success semi-decides .
∎
So "r.e." means "membership has a finite, checkable certificate ": a halting computation, a proof, an integer solution. Decidable sets are those where non-membership also has one.
Reductions
To show a new problem undecidable we rarely repeat the diagonal argument. Instead we reduce a known undecidable problem to it.
Definition— many-one reduction
is many-one reducible to , written , if there is a total computable with for all .
Theorem— Lemma 7 (reductions transfer decidability)
Suppose . If is decidable, so is ; if is r.e., so is . Equivalently: if is undecidable, so is ; if is not r.e., neither is .
Proof
Let be the reduction. To decide (or semi-decide) whether , compute — this always halts — and run the decider (or semi-decider) for on . Correctness is exactly .
∎
The direction matters: to prove hard, reduce a known hard problem to .
Example— Halting on the blank tape
Let . Given , compute (by the s-m-n theorem) an index for the machine that ignores its input, writes on the tape and then runs . Then halts on the blank tape iff , so and is undecidable.
Theorem— Corollary 8 (the busy-beaver function is not computable)
is not computable.
Proof
If it were, we could decide : given , count the states of (passing first to an equivalent two-symbol machine, which can be done effectively), compute , and simulate on the blank tape for steps. If it has not halted by then, it never will, by the definition of . This contradicts the undecidability of .
∎
In fact eventually exceeds every computable function (Radó, 1962). The known values are , , , and — settled in 2024 by the online Busy Beaver Challenge collaboration, with a proof checked in the Coq proof assistant — . Already for six states, the question involves machines whose behaviour resembles open problems of Collatz type.
Rice's theorem
Halting is not special. Every question about what a program computes — as opposed to what its text looks like — is undecidable, unless it is trivial.
Theorem— Theorem 9 (Rice, 1953)
Let be a set of partial computable functions which is non-trivial: some partial computable function lies in and some does not. Then the index set is undecidable.
Proof
Let denote the nowhere-defined function. Decidability is preserved by complements, and is the complement of , so we may assume (otherwise replace by its complement in the partial computable functions, which is also non-trivial). Choose , computed by the machine .
For each , let be an index of the machine that, on input , first runs on and, if that halts, then runs on and outputs the result. The function is computable (s-m-n theorem), and
Since and , we get . So , and is undecidable by Lemma 7.
∎
Example— What Rice's theorem does and does not say
Undecidable by Rice: " is total", "", " is the zero function", " is finite", " computes the same function as ". Not covered by Rice, because they are about the program text or running time rather than the function computed: " has an even number of states" (decidable), " halts on input within steps" (decidable). Rice's theorem is the reason a perfect bug-finder cannot exist: "this program never divides by zero" is a semantic property.
Computability in Lean and Mathlib
Mathlib's computability library (built by Mario Carneiro) does not start from Turing machines. Its basic notion is the partial recursive function Nat.Partrec : (ℕ →. ℕ) → Prop, generated from zero, successor and pairing by composition, primitive recursion and unbounded search — a model proved equivalent to Turing machines in the classical literature. Programs are an inductive type Nat.Partrec.Code with a universal evaluator Code.eval : Code → ℕ →. ℕ, and Code is countable (indeed Denumerable). The arrow →. is a partial function, valued in Part ℕ; (c.eval n).Dom is the proposition "program c halts on n". Mathlib separately develops Turing machines and proves that they can simulate partial recursive functions.
To make the definitions concrete, this lesson's module also contains a small two-symbol Turing machine simulator of its own:
/-- A Turing machine with states `Q`: in state `q` reading symbol `b`, it writes a symbol,
moves, and goes to a new state — or halts (`none`). The table is finite when `Q` is. -/
def Machine (Q : Type) := Q → Bool → Bool × Dir × Option Q/-- One step of the machine `M`. A halted configuration stays where it is. -/
def step {Q : Type} (M : Machine Q) : Config Q → Config Q
| ⟨none, t⟩ => ⟨none, t⟩
| ⟨some q, t⟩ =>
let (w, d, q') := M q t.head
⟨q', Tape.move d { t with head := w }⟩Concrete runs are checked by evaluation (decide): the incrementer on , and the busy beaver halting at step 6 with four s.
/-- The incrementer turns `1011` (eleven) into `1100` (twelve) in three steps, checked by
evaluation. The tape then reads `1 1 0 0` from left to right, with the head on its first cell. -/
theorem incr_eleven :
run incr 3 ⟨some (), ⟨[true, false, true], true, []⟩⟩ =
⟨none, ⟨[], true, [true, false, false]⟩⟩ := by
decide/-- The two-state busy beaver (states `false` = A, `true` = B):
A0 → 1RB, A1 → 1LB, B0 → 1LA, B1 → 1R(halt). -/
def bb2 : Machine Bool
| false, false => (true, .right, some true)
| false, true => (true, .left, some true)
| true, false => (true, .left, some false)
| true, true => (true, .right, none)/-- Started in state A on a blank tape, `bb2` is still running after 5 steps, halts at step 6,
and leaves four `1`s: the values `S(2) = 6` and `Σ(2) = 4` of Radó's busy-beaver functions are
attained by this machine. (That no two-state machine does better needs a proof that every other
two-state machine either halts sooner or never halts; that is not formalised here.) -/
theorem bb2_run :
(run bb2 5 ⟨some false, ⟨[], false, []⟩⟩).state ≠ none ∧
(run bb2 6 ⟨some false, ⟨[], false, []⟩⟩).state = none ∧
(run bb2 6 ⟨some false, ⟨[], false, []⟩⟩).tape.ones = 4 := by
decideThe diagonal argument, abstractly. The heart of Theorem 2 needs nothing about Turing machines: only a "halts" relation and the ability to build the diagonal program. Proved here:
/-- **The diagonal argument behind the halting problem**, for any notion of program.
Let `Halts p q` mean "program `p` halts on input `q`", and let `d` be any proposed decider.
If programs are flexible enough to build the *diagonal program* `e`, which on input `q` halts
exactly when `d` says "`q` does not halt on `q`", then `d` is wrong somewhere: feeding `e` to
itself, `e` halts on `e` iff `d e e = false`, while correctness would demand `d e e = true`. -/
theorem no_halting_decider {P : Type*} (Halts : P → P → Prop) (d : P → P → Bool)
(diag : ∃ e : P, ∀ q, Halts e q ↔ d q q = false) :
¬ ∀ p q, Halts p q ↔ d p q = true := by
intro hd
obtain ⟨e, he⟩ := diag
have h1 : Halts e e ↔ d e e = false := he e
have h2 : Halts e e ↔ d e e = true := hd e e
cases hde : d e e
· -- `d e e = false`: then `e` halts on `e`, so correctness forces `d e e = true`.
exact Bool.false_ne_true (hde ▸ h2.1 (h1.2 hde))
· -- `d e e = true`: then `e` halts on `e`, so the diagonal property forces `d e e = false`.
exact Bool.false_ne_true (hde ▸ (h1.1 (h2.2 hde)).symm)Theorem 1, formalised. Cantor's diagonal argument for , and then the counting argument, using Mathlib's theorem that every computable function has a code:
/-- **Cantor's diagonal argument.** No sequence `e 0, e 1, e 2, …` of functions `ℕ → ℕ`
contains every function: `f n = e n n + 1` differs from `e n` at the input `n`. -/
theorem cantor_diag (e : ℕ → ℕ → ℕ) : ¬ Function.Surjective e := by
intro he
obtain ⟨k, hk⟩ := he (fun n => e n n + 1)
have : e k k = e k k + 1 := congrFun hk k
omega/-- **Most functions are not computable.** Every computable `f : ℕ → ℕ` is evaluated by some
program (`Nat.Partrec.Code.exists_code`), different functions need different programs, and there
are only countably many programs. So if every function were computable, `ℕ → ℕ` would be
countable and could be listed as a sequence — contradicting `cantor_diag`. -/
theorem exists_not_computable : ∃ f : ℕ → ℕ, ¬ Computable f := by
by_contra hall
push Not at hall
-- Choose a program for every function.
have hcode : ∀ f : ℕ → ℕ, ∃ c : Code, c.eval = (f : ℕ →. ℕ) := fun f =>
Nat.Partrec.Code.exists_code.1 (Partrec.nat_iff.1 (hall f))
choose code hcode using hcode
-- Different functions get different programs.
have hinj : Function.Injective code := by
intro f g hfg
funext n
have h : ((f : ℕ →. ℕ) n) = ((g : ℕ →. ℕ) n) := by
rw [← hcode f, ← hcode g, hfg]
exact Part.some_injective h
-- So `ℕ → ℕ` is countable, hence listed by a sequence, contradicting the diagonal argument.
have : Countable (ℕ → ℕ) := hinj.countable
obtain ⟨e, he⟩ := exists_surjective_nat (ℕ → ℕ)
exact cantor_diag e heThe halting problem, Post's theorem and Rice's theorem from Mathlib. Mathlib's ComputablePred p means that p is decidable and its decision procedure is computable; REPred p means p is the domain of a partial recursive function. Note that Mathlib's halting problem is stated for each fixed input n, a slightly sharper form than Theorem 2.
/-- **The halting problem is undecidable** (Turing 1936): for any fixed input `n`, no computable
predicate decides, for every program `c`, whether `c` halts on `n`. This applies Mathlib's
`ComputablePred.halting_problem`. -/
theorem halting_problem (n : ℕ) : ¬ ComputablePred fun c : Code => (c.eval n).Dom :=
ComputablePred.halting_problem n/-- The halting set *is* recursively enumerable: run the program and say "yes" if it stops.
This applies Mathlib's `ComputablePred.halting_problem_re`. -/
theorem halting_re (n : ℕ) : REPred fun c : Code => (c.eval n).Dom :=
ComputablePred.halting_problem_re n/-- **Post's theorem**: a predicate is decidable if and only if both it and its negation are
recursively enumerable. This applies Mathlib's `ComputablePred.computable_iff_re_compl_re'`. -/
theorem post {α : Type*} [Primcodable α] (p : α → Prop) :
ComputablePred p ↔ REPred p ∧ REPred fun a => ¬ p a :=
ComputablePred.computable_iff_re_compl_re'Corollary 5 and Lemma 7, proved here from those:
/-- The set of programs that do **not** halt on `n` is not even recursively enumerable:
if it were, the halting set would be decidable by Post's theorem. -/
theorem non_halting_not_re (n : ℕ) : ¬ REPred fun c : Code => ¬ (c.eval n).Dom := by
intro h
exact halting_problem n ((post _).2 ⟨halting_re n, h⟩)/-- **Reductions transfer undecidability.** If `p` many-one reduces to `q` (a computable `f`
with `p a ↔ q (f a)`) and `p` is undecidable, then so is `q`: a decider for `q`, composed with
`f`, would decide `p`. The composition step is Mathlib's
`ComputablePred.computable_of_manyOneReducible`. -/
theorem undecidable_of_reduces {α β : Type*} [Primcodable α] [Primcodable β]
{p : α → Prop} {q : β → Prop} (hpq : ManyOneReducible p q) (hp : ¬ ComputablePred p) :
¬ ComputablePred q := fun hq =>
hp (ComputablePred.computable_of_manyOneReducible hpq hq)/-- **Rice's theorem** (1953): a property of programs that depends only on the partial function
a program computes is decidable only if it is trivial — true of no program or of every program.
This applies Mathlib's `ComputablePred.rice₂`. -/
theorem rice (C : Set Code) (H : ∀ cf cg : Code, cf.eval = cg.eval → (cf ∈ C ↔ cg ∈ C)) :
(ComputablePred fun c => c ∈ C) ↔ C = ∅ ∨ C = Set.univ :=
ComputablePred.rice₂ C HAnd an application of Rice's theorem, proved here — the property "computes the constant function " is semantic and non-trivial:
/-- **No algorithm decides whether a program computes the constant function `0`.**
The property depends only on the function computed, and it is non-trivial: the program
`Code.const 0` has it, and a program that never halts does not. Rice's theorem finishes. -/
theorem zero_function_undecidable :
¬ ComputablePred fun c : Code => c.eval = fun _ => Part.some 0 := by
intro h
-- A program that diverges on every input.
obtain ⟨cnone, hnone⟩ := Nat.Partrec.Code.exists_code.1 Nat.Partrec.none
have hC := (rice {c : Code | c.eval = fun _ => Part.some 0}
(fun cf cg hfg => by
change cf.eval = _ ↔ cg.eval = _
rw [hfg])).1 h
rcases hC with hempty | huniv
· -- `Code.const 0` computes the zero function, so the set is not empty.
have hmem : Code.const 0 ∈ {c : Code | c.eval = fun _ => Part.some 0} := by
show (Code.const 0).eval = fun _ => Part.some 0
funext m
exact Nat.Partrec.Code.eval_const 0 m
rw [hempty] at hmem
exact hmem
· -- The diverging program does not, so the set is not everything.
have hmem : cnone ∈ {c : Code | c.eval = fun _ => Part.some 0} := by
rw [huniv]
exact Set.mem_univ _
have h0 : (Part.none : Part ℕ) = Part.some 0 := by
have := congrFun hmem 0
rw [hnone] at this
exact this
exact Part.some_ne_none 0 h0.symmWhat is not formalised here: the equivalence of our toy machines with Nat.Partrec (we use the toy machines only for examples), Theorem 6, the busy-beaver corollary, and the claim that no two-state machine beats bb2 (the Python experiment below gives a computer-assisted proof of that, outside Lean). The Church–Turing thesis cannot be formalised, because it is not a mathematical statement.
Running machines, and proving that some never stop
The first program simulates the incrementer and the unary adder and checks them on every input in a range, together with the step counts claimed in the examples.
"""Two small Turing machines, simulated and checked on many inputs.
A Turing machine is a finite table: in state q reading symbol s, write a
symbol, move one cell left or right, and go to a new state (or halt). We
simulate two machines from the lesson:
* a one-state binary incrementer, checked on every input n < 256, with the
number of steps it takes (the amortised cost of counting is below 2 steps);
* a three-state unary adder turning 1^m 0 1^n into 1^(m+n), checked for all
m, n <= 12.
The checks are exhaustive for the stated ranges only. They are evidence that
the machines are correct, not a proof for all inputs; the proofs are short
inductions given in the lesson's exercises.
"""
import textwrap
HALT = None
LEFT, RIGHT = -1, 1
#: Binary incrementer. Head starts on the least significant bit.
INCREMENT = {
("carry", 1): (0, LEFT, "carry"),
("carry", 0): (1, LEFT, HALT),
}
#: Unary adder: input 1^m 0 1^n with the head on the leftmost cell.
ADD = {
("A", 1): (1, RIGHT, "A"),
("A", 0): (1, RIGHT, "B"),
("B", 1): (1, RIGHT, "B"),
("B", 0): (0, LEFT, "C"),
("C", 1): (0, LEFT, HALT),
("C", 0): (0, LEFT, HALT),
}
def run(machine, tape, head, state, max_steps=10_000):
"""Run until the machine halts. Return (tape dict, head, steps).
``tape`` maps positions to symbols (missing cells are blank 0). Raises
RuntimeError if the machine has not halted after ``max_steps`` steps:
a step bound can never certify that a machine runs forever.
"""
tape = dict(tape)
for steps in range(1, max_steps + 1):
write, move, state = machine[(state, tape.get(head, 0))]
tape[head] = write
head += move
if state is HALT:
return tape, head, steps
raise RuntimeError("no halt within the step bound")
def binary_tape(n):
"""Binary digits of n, least significant bit at position 0, others left."""
bits = bin(n)[2:]
return {-i: int(b) for i, b in enumerate(reversed(bits))}
def read_binary(tape):
"""Read the number written on the tape (the most significant 1 first)."""
cells = [p for p, s in tape.items() if s == 1]
if not cells:
return 0
return sum(2 ** (-p) for p in cells if p <= 0)
def increment(n):
"""Run the incrementer on n; return (output, steps)."""
tape, _, steps = run(INCREMENT, binary_tape(n), 0, "carry")
return read_binary(tape), steps
def unary_tape(m, n):
"""The input 1^m 0 1^n written from position 0 to the right."""
cells = [1] * m + [0] + [1] * n
return {i: s for i, s in enumerate(cells)}
def add(m, n):
"""Run the unary adder on (m, n); return (number of 1s left, steps)."""
tape, _, steps = run(ADD, unary_tape(m, n), 0, "A")
return sum(tape.values()), steps
def trailing_ones(n):
"""Number of 1s at the end of the binary expansion of n."""
count = 0
while n & 1:
n >>= 1
count += 1
return count
def main():
print("1. Binary incrementer (1 state): carry 1s to the left, then halt.")
rows = [(n, *increment(n)) for n in range(256)]
bad = [n for n, out, _ in rows if out != n + 1]
steps_ok = all(s == trailing_ones(n) + 1 for n, _, s in rows)
print(f" inputs 0..255 checked: {len(rows)}, wrong outputs: {len(bad)}")
print(f" steps = (trailing 1s) + 1 for every input: {steps_ok}")
for n in (11, 127, 128):
out, s = increment(n)
unit = "step" if s == 1 else "steps"
print(f" {n:>3} = {n:08b} -> {out:>3} = {out:08b} in {s} {unit}")
total = sum(s for _, _, s in rows)
print(f" average steps over 0..255: {total}/256 = {total / 256:.4f}")
print()
print("2. Unary adder (3 states): 1^m 0 1^n -> 1^(m+n).")
pairs = [(m, n) for m in range(13) for n in range(13)]
wrong = [(m, n) for m, n in pairs if add(m, n)[0] != m + n]
print(f" pairs with m, n <= 12 checked: {len(pairs)}, wrong: {len(wrong)}")
print(f" steps for (3, 4): {add(3, 4)[1]}; for (12, 12): {add(12, 12)[1]}")
print(" steps = m + n + 3 on every pair: "
f"{all(add(m, n)[1] == m + n + 3 for m, n in pairs)}")
print()
print(textwrap.fill(
"Epistemic status: exact simulation, exhaustive for the 256 inputs and "
"169 pairs above, so it is evidence of correctness and of the step "
"formulas, not a proof for all inputs (that needs an induction on the "
"length of the input). Every run here halted; a run that had not halted "
"would have proved nothing, since no step bound separates slow machines "
"from machines that never halt.",
width=78,
))
if __name__ == "__main__":
main()Exhaustive checks over a finite range are evidence, not proof; the general statements need an induction on the input (Exercise). Notice the asymmetry built into the simulator: if a run had not halted within the step bound, the program would have learned nothing.
The second program confronts that asymmetry head-on. It takes all two-state, two-symbol machines and classifies every one. Halting is observed directly. Non-halting is never observed — instead the program looks for a certificate, a reason that is a valid proof of non-halting:
- unreachable halt: no halting instruction can be reached from the start state in the table's state graph;
- repeated configuration: the exact configuration (state, head position, tape) occurs twice, so the run is periodic;
- translated cycle: at two times the head is at a new record position (left or right), in the same state, and the tape contents on the side it came from agree, as far back as the head travelled between and ; beyond the record the tape is blank both times, so the machine repeats the same moves forever, shifted along the tape.
"""Every two-state Turing machine, classified: the busy beaver S(2) = 6.
Radó's busy-beaver function S(n) is the largest number of steps taken by an
n-state, two-symbol Turing machine that halts when started on a blank tape;
Sigma(n) is the largest number of 1s such a machine can leave. Both grow
faster than any computable function, so no algorithm computes them.
For n = 2 there are 12^4 = 20736 machines, and we can settle every one:
it either halts (we watch it) or it carries a certificate that it never
halts. The three certificates are sound arguments, explained in the lesson:
* no halting instruction can ever be reached from the start state;
* a whole configuration repeats, so the run is periodic;
* a translated cycle: the machine returns to the same state at a new record
position with the same tape contents behind it, so it repeats forever,
shifted along the tape.
Running a machine for longer never certifies that it runs forever; the
certificates are what turn this search into a (computer-assisted) proof.
"""
import itertools
import textwrap
HALT = -1
OPTIONS = [(w, m, n) for w in (0, 1) for m in (-1, 1) for n in (0, 1, HALT)]
#: Rado's champion: A0 -> 1RB, A1 -> 1LB, B0 -> 1LA, B1 -> 1R halt.
CHAMPION = ((1, 1, 1), (1, -1, 1), (1, -1, 0), (1, 1, HALT))
def all_machines():
"""All tables (A0, A1, B0, B1); entry = (write, move, next state)."""
return itertools.product(OPTIONS, repeat=4)
def halt_reachable(machine):
"""Can a halting instruction be reached from state A in the state graph?"""
seen, todo = {0}, [0]
while todo:
q = todo.pop()
for symbol in (0, 1):
nxt = machine[2 * q + symbol][2]
if nxt == HALT:
return True
if nxt not in seen:
seen.add(nxt)
todo.append(nxt)
return False
def _translated(history, t1, t2, d):
"""Sound test for a translated cycle between two records in direction d."""
s1, p1, tape1 = history[t1]
s2, p2, tape2 = history[t2]
if s1 != s2 or p1 == p2:
return False
back = max(-d * (history[t][1] - p1) for t in range(t1, t2 + 1))
return all(tape1.get(p1 - d * o, 0) == tape2.get(p2 - d * o, 0)
for o in range(back + 1))
def classify(machine, max_steps=30):
"""Return ("halts", steps, ones) or (certificate name, step, None).
Certificates: "unreachable", "cycle", "translated". If none applies
within max_steps the answer is ("unknown", max_steps, None).
"""
if not halt_reachable(machine):
return "unreachable", 0, None
tape, pos, state, lo, hi = {}, 0, 0, 0, 0
history, seen = [], set()
records = {1: [], -1: []}
for step in range(max_steps):
key = (state, pos, frozenset(p for p, s in tape.items() if s))
if key in seen:
return "cycle", step, None
seen.add(key)
history.append((state, pos, dict(tape)))
for d in (1, -1):
if step == 0 or (pos > hi if d == 1 else pos < lo):
records[d].append(step)
for t1 in records[d][:-1]:
if _translated(history, t1, step, d):
return "translated", step, None
lo, hi = min(lo, pos), max(hi, pos)
write, move, nxt = machine[2 * state + tape.get(pos, 0)]
tape[pos] = write
pos += move
if nxt == HALT:
return "halts", step + 1, sum(tape.values())
state = nxt
return "unknown", max_steps, None
def census(max_steps=30):
"""Classify every machine; return (counts, max steps, max ones)."""
counts, best_steps, best_ones = {}, 0, 0
for machine in all_machines():
kind, steps, ones = classify(machine, max_steps)
counts[kind] = counts.get(kind, 0) + 1
if kind == "halts":
best_steps = max(best_steps, steps)
best_ones = max(best_ones, ones)
return counts, best_steps, best_ones
def main():
counts, best_steps, best_ones = census()
total = sum(counts.values())
print(f"Two-state, two-symbol machines: {total} (12 choices for 4 entries)")
labels = [
("halts", "halt on the blank tape (watched)"),
("unreachable", "never halt: no halt instruction reachable"),
("cycle", "never halt: a configuration repeats"),
("translated", "never halt: translated cycle"),
("unknown", "unclassified within 30 steps"),
]
for kind, label in labels:
print(f" {counts.get(kind, 0):>6} {label}")
print()
print(f"Longest halting run S(2) = {best_steps} steps; "
f"most 1s Sigma(2) = {best_ones}.")
kind, steps, ones = classify(CHAMPION)
print(f"Rado's champion A0:1RB A1:1LB B0:1LA B1:1RH -> {kind}, "
f"{steps} steps, {ones} ones.")
print("Known values (from the literature, not computed here):")
print(" S(3) = 21, S(4) = 107, S(5) = 47,176,870; S(6) is unknown.")
print()
print(textwrap.fill(
"Epistemic status: an exhaustive, exact classification of all 20736 "
"machines. Each non-halting verdict rests on a certificate that is a "
"valid proof of non-halting, so the computation proves S(2) = 6 and "
"Sigma(2) = 4 - provided this code implements the machines and the "
"certificates correctly, which tests check but do not prove. Simply "
"running machines longer could never have shown that they run forever.",
width=78,
))
if __name__ == "__main__":
main()Every machine is settled, so — provided the code is correct — this is a proof that and . The 2024 proof that worked the same way on a vastly larger scale: all five-state machines were either run to halting or given certificates by a collection of "deciders", and the deciders were verified in Coq so that the code's correctness was no longer an assumption. The undecidability of the halting problem says that no finite collection of certificate types can work for all .
Pictures of the two key arguments
The diagonal. Tabulate halting behaviour, with for "halts" and for "runs forever":
| input 0 | input 1 | input 2 | input 3 | … | |
|---|---|---|---|---|---|
| ↓ | ↑ | ↓ | ↓ | ||
| ↑ | ↑ | ↓ | ↑ | ||
| ↓ | ↓ | ↓ | ↑ | ||
| ↑ | ↓ | ↑ | ↓ | ||
| ↑ | ↓ | ↑ | ↑ |
The machine of Theorem 2 does the opposite of the diagonal entry in every column, so its row differs from row in column . If a decider for existed, would be a machine — some row — and the entry in row , column would have to differ from itself.
A reduction is a translation of questions that preserves answers. Here and are the characteristic functions, with values in :
Things to try: in the proof of Rice's theorem, what are , and ? (Answer: , , .) Draw the triangle for the reduction .
Exercises
1. Computability
Warm-upHumanThe one-state binary incrementer of the lesson starts on the least significant bit of (twenty-three). How many steps does it take before it halts?
2. Computability
Warm-upHumanA Turing machine has states and the two symbols . Each of its table entries chooses a symbol to write, a direction (L or R) and a next state, which is one of the states or halt. How many different tables are there?
3. Computability
CoreHumanWhich of these sets are recursively enumerable? (Select all that apply.)
4. Computability
CoreHumanIn the proof that is undecidable we assumed a machine computing the total function . Suppose instead we only had a semi-decider for : it halts with output when and runs forever otherwise. Build as before (on input : run ; if it outputs , loop forever; otherwise halt). What happens?
5. Computability
CoreHumanYou want to prove that a new set is undecidable, using the undecidability of . Which do you need to show?
6. Computability
StretchHumanFor which of these properties of an index does Rice's theorem show that the set of with the property is undecidable? (Select all that apply.)
7. Computability
CoreFormalIn Mathlib, c.eval n is the (partial) result of running program c : Nat.Partrec.Code on input n, and (c.eval n).Dom says that it halts. Which Lean statement says that halting on input n is undecidable?
8. Computability
CoreComputationalWrite run(machine, max_steps) that simulates a two-symbol Turing machine started in state 0 on a blank tape. machine is a dict mapping (state, symbol) to (write, move, next_state), with move in (-1, 1) and next_state equal to None for halt. Return ("halt", steps, ones) if it halts within max_steps steps (counting the halting step) and ("running", max_steps, ones) otherwise, where ones is the number of 1s on the tape at that moment.
9. Computability
StretchHumanWe want to prove for all that the incrementer, started on the binary expansion of with the head on the last bit, halts with on the tape after steps, where is the number of trailing s. Which induction works?
Common misconceptions
Misconception
"The Church–Turing thesis is a theorem." It cannot be: it relates the informal notion of an effective procedure to a mathematical definition. What are theorems are the equivalences between formal models — Turing machines, λ-calculus, recursive functions, register machines — which are the main evidence for the thesis.
Misconception
"Undecidable means no instance can be solved." Undecidability says that no single algorithm answers every instance correctly. Particular instances are often easy (this program has no loops, so it halts), and for any one instance one of the two constant answers is correct. What fails is uniformity.
Misconception
"If a program has not halted after a very long time, it probably never will." The busy-beaver function shows how wrong this can be: a five-state machine runs for steps before halting. No computable bound on running time exists for halting machines, or the halting problem would be decidable.
Misconception
"Rice's theorem says every property of programs is undecidable." Only semantic properties — those that depend on the function computed — and only non-trivial ones. Syntactic questions ("does the program contain a loop?", "does it halt within steps on input ?") are often decidable.
Misconception
"Recursively enumerable means the members can be listed in increasing order." An infinite set can be enumerated in increasing order by an algorithm if and only if it is decidable. An r.e. set like can be listed, but in a scrambled order: elements keep appearing whose size bears no relation to how long you have waited.
Misconception
"Undecidability is a fact about physical computers with finite memory." A computer with a fixed finite memory has finitely many configurations, so its halting problem is decidable in principle (watch for a repeated configuration). The theorem concerns unbounded memory — the idealisation that makes "algorithm" a mathematical notion — and its practical message is that no general-purpose analysis tool can be both always-terminating and always-correct.
Many ways of seeing
Logic— Incompleteness from the halting problem
Hilbert's Entscheidungsproblem asked for an algorithm deciding validity of first-order sentences. By Gödel's completeness theorem the valid sentences are r.e. (enumerate proofs); Church and Turing showed in 1936 that they are not decidable, by coding the halting of a machine as the validity of a sentence. The halting problem also gives a quick form of Gödel's first incompleteness theorem. Let be any theory whose theorems are r.e. and which is sound for statements of the form " does not halt on " (it proves only true ones). If proved every true such statement, then would be r.e. — contradicting Corollary 5. So some true sentence " does not halt on " is unprovable in . Gödel's theorem is stronger (it assumes only consistency, and produces an explicit sentence), but the halting argument shows why incompleteness is unavoidable: proofs are finite certificates, and non-halting has none in general. See Gödel's incompleteness theorems and first-order logic.
Number theory— Hilbert's tenth problem
Hilbert's tenth problem (1900) asked for a procedure deciding whether a polynomial equation with integer coefficients has a solution in integers. The MRDP theorem (Matiyasevich 1970, completing work of Martin Davis, Hilary Putnam and Julia Robinson) says that every r.e. set is Diophantine: there is a polynomial with . Applied to , this produces a polynomial family for which solvability is undecidable, so Hilbert's procedure does not exist. Applied to the primes it yields a polynomial whose positive values, as the variables range over , are exactly the primes (Jones, Sato, Wada and Wiens, 1976). Theorem 6 — r.e. sets are projections of decidable relations — is the first step: MRDP shows the decidable relation can be taken to be a single polynomial equation. Whether solvability in rational numbers is decidable remains open. See elementary number theory.
Algebra— The word problem for groups
A finitely presented group comes with a natural question: does a given word in the generators represent the identity? Max Dehn posed it in 1911 and solved it for surface groups. The set of trivial words is always r.e. (enumerate consequences of the relators), and for free groups, finite groups, one-relator groups (Magnus, 1932) and hyperbolic groups the word problem is decidable. But Pyotr Novikov (1955) and William Boone (1958) independently constructed finitely presented groups with undecidable word problem, by encoding machines into relators. The Adian–Rabin theorem (1955–58) goes further: whether a finite presentation defines the trivial group, a finite group, an abelian group, … are all undecidable — Rice's theorem for groups. See groups and geometric group theory.
Topology— Semi-decidable sets are open
Semi-decidable properties behave like open sets: finite intersections and (computably indexed) unions of r.e. sets are r.e., complements need not be — just as complements of open sets need not be open. This is more than an analogy. If a semi-decider accepts an input, it has read only finitely much of it; so in the space of infinite sequences (or of partial functions, with the Scott topology), a semi-decidable property is determined by finite information and is genuinely open. The Rice–Shapiro theorem makes this precise: if is r.e., then a partial computable lies in if and only if some finite restriction of lies in . Totality fails this test (no finite function is total), so is not r.e. Undecidability also appears inside topology: since every finitely presented group is the fundamental group of a closed 4-manifold, the Adian–Rabin theorem led A. A. Markov (1958) to prove that the homeomorphism problem for closed 4-manifolds is undecidable. In dimension 3, by contrast, homeomorphism of closed manifolds is decidable, a consequence of Perelman's geometrisation theorem together with earlier algorithmic work. See point-set topology and the fundamental group.
Geometry— Tilings that cannot be decided
A Wang tile is a unit square with coloured edges; a set of tiles tiles the plane if copies (translated, not rotated) can cover it with matching colours on adjacent edges. Hao Wang (1961) asked whether one can decide if a finite tile set tiles the plane, and observed that this would follow if every tile set that tiles the plane could tile it periodically. Robert Berger (1966) proved the domino problem undecidable, by encoding the space-time diagram of a Turing machine into tiles — rows of tiles are successive configurations — and on the way he constructed an aperiodic tile set, which tiles the plane but never periodically. This is the origin of the aperiodic tilings of Robinson (1971) and Penrose (1974), of the mathematics of quasicrystals, and of the 2023 discovery by Smith, Myers, Kaplan and Goodman-Strauss of a single aperiodic tile, the "hat". See symmetry and transformations.
Analysis— Computable real numbers
A real number is computable if some algorithm, given , outputs a rational with ; , and every algebraic number are computable, and only countably many reals are. Analysis looks different in this world. Ernst Specker (1949) constructed a computable, increasing, bounded sequence of rationals whose limit is not a computable real: list without repetitions by an algorithm as and take the partial sums of (from the limit one could decide ). So the monotone convergence theorem has no computable version. Equality of computable reals is undecidable — only is semi-decidable, by computing both to more and more precision until they separate — and as a consequence every computable function is continuous, so the sign function is not computable. Pour-El and Richards (1981) found computable initial data for the three-dimensional wave equation whose solution at a later time is not computable. See real analysis.
Category theory— Lawvere's fixed-point theorem
Cantor's theorem, Russell's paradox, Gödel's incompleteness theorem and the halting problem all walk down a diagonal. F. William Lawvere (1969) showed that the core of many such arguments is a single theorem about cartesian closed categories: if there is a morphism that is surjective on points, then every endomorphism has a fixed point. (Proof: given , form , choose a point with , and check that is fixed by .) Read contrapositively: since negation on has no fixed point, there is no surjection — Cantor's theorem. Turing's argument has the same shape, with "swap halting and looping". Noson Yanofsky's 2003 survey in the Bulletin of Symbolic Logic works through many such instances from logic and computer science. See categories and the λ-calculus, where fixed points of every term exist and give recursion.
History and global context
- Algorithms before computability. The word algorithm comes from the name of Muḥammad ibn Mūsā al-Khwārizmī, who worked in Baghdad in the 9th century and whose arithmetic book introduced Hindu–Arabic numeral calculation to readers of Arabic and, in Latin translation, of Europe. Euclid's algorithm, the Chinese remainder procedure in Sunzi's Suanjing and Indian kuṭṭaka methods for linear Diophantine equations are much older. Leibniz dreamed of a calculus ratiocinator that would settle disputes by calculation.
- Hilbert's questions. In 1900 David Hilbert asked for a procedure to decide Diophantine equations (his tenth problem); in 1928, with Wilhelm Ackermann, he posed the Entscheidungsproblem for first-order logic. Kurt Gödel's incompleteness theorems (1931) introduced the coding of syntax by numbers and the primitive recursive functions.
- 1936. Alonzo Church (with his student Stephen Kleene) proposed λ-definability as the definition of effective calculability and proved the Entscheidungsproblem unsolvable. Independently, Alan Turing, a young fellow of King's College, Cambridge, defined his machines from an analysis of human calculation, built a universal machine, proved that no machine can decide whether a given machine is "circle-free" (the ancestor of the halting problem, which was named later) and solved the Entscheidungsproblem negatively; his paper appeared in 1936–37. Emil Post published a closely similar model the same year. Gödel, initially unconvinced by Church's thesis, accepted Turing's analysis as decisive.
- The theory grows. Kleene developed recursive function theory (the s-m-n and recursion theorems); Post's 1944 paper founded the study of r.e. sets and degrees; Henry Gordon Rice proved his theorem in 1953. Pyotr Novikov in Moscow (1955) and William Boone (1958) proved the word problem undecidable; A. A. Markov proved the undecidability of the homeomorphism problem for 4-manifolds (1958).
- Hilbert's tenth problem was resolved in 1970 by the 22-year-old Yuri Matiyasevich in Leningrad, completing two decades of work by Martin Davis, Hilary Putnam and Julia Robinson.
- Busy beavers. Tibor Radó introduced the busy-beaver function in 1962; Shen Lin and Radó settled three states in 1965, Allen Brady four states in 1983, and the Busy Beaver Challenge, an open online collaboration, five states in 2024.
Further reading
- Michael Sipser, Introduction to the Theory of Computation (1996; 3rd edition 2012) — Part Two is a clear first course on Turing machines, decidability and reductions.
- George S. Boolos, John P. Burgess and Richard C. Jeffrey, Computability and Logic (5th edition 2007) — computability alongside incompleteness.
- Nigel Cutland, Computability: An Introduction to Recursive Function Theory (1980) — a careful mathematical introduction via register machines.
- Hartley Rogers, Theory of Recursive Functions and Effective Computability (1967) — the classic graduate text.
- Robert I. Soare, Turing Computability: Theory and Applications (2016) — modern computability theory.
- Alan M. Turing, "On computable numbers, with an application to the Entscheidungsproblem", Proceedings of the London Mathematical Society (1936–37); Charles Petzold, The Annotated Turing (2008) reads it line by line.
- Mario Carneiro, "Formalizing computability theory via partial recursive functions", Interactive Theorem Proving (2019) — the design of Mathlib's library.
Next steps in the Lumenary: computational complexity asks what can be computed efficiently; the λ-calculus is Church's equivalent model; Gödel's incompleteness theorems carry the diagonal argument into logic.