Everyone makes mistakes
On 9 September 1947, the team running the Harvard Mark II computer traced a fault to a moth trapped in one of the machine's relays. They taped the moth into the logbook with a note joking that this was the first actual case of a bug being found. The joke worked because engineers had already been calling faults "bugs" for decades; this time there really was one. The page, moth and all, is now kept by the Smithsonian's National Museum of American History. Grace Hopper, a pioneer of programming languages who worked on the Harvard machines, loved to tell the story, and it helped make "bug" and "debugging" part of every programmer's vocabulary.
Here is the honest truth about programming: everyone writes bugs. Beginners do, and so do the engineers who build operating systems, banking software and spacecraft. What separates experienced programmers is not that they make fewer mistakes, though they do make fewer of some kinds, but that they find and fix them methodically, and that they write tests that catch mistakes early, before a user ever sees them.
Debugging is not a talent you either have or don't. It is a method, and this lesson teaches it: how to read what Python tells you when something goes wrong, how to hunt down the mistakes it can't tell you about, and how to write small tests that guard your functions.
Debugging is detective work
When a program misbehaves, it is tempting to start changing things until the problem goes away. Resist! Random changes waste time and often add new bugs. Instead, debug the way a scientist runs an experiment, or a doctor makes a diagnosis:
- Observe. What exactly happens? What did you expect instead? Can you make it happen again, every time?
- Hypothesise. Make a specific guess about the cause: "I think
totalis never reset between classes." - Experiment. Check the guess, for example by printing
totalat the start of each class. - Conclude. If the guess was right, fix the cause. If not, you have still learned something: cross that cause off and make a new guess.
Analogy, not a formal construction
A doctor doesn't treat a fever by trying every medicine in the cupboard. She asks questions, looks at the evidence, forms a hypothesis about the cause, and orders a test that will confirm or rule it out. Debugging is the same: each experiment should be designed to answer one question.
Definitions
Definition— bug
A bug is a mistake in a program that makes it behave differently from what was intended. Debugging is finding and removing bugs.
Definition— three kinds of error
A syntax error breaks the rules of the language's grammar, so Python refuses to run the program at all. A runtime error (in Python, an exception) happens while the program is running, when an instruction can't be carried out; the program stops and reports it. A logic error is a mistake in the algorithm: the program runs to the end without complaint but gives the wrong result.
Definition— traceback
A traceback is the report Python prints when an exception stops a program. It lists the chain of function calls that were in progress, ending with the line where the error happened, and finally the kind of exception and a message.
Definition— test and edge case
A test runs a piece of code on a chosen input and checks that the result is what it should be. An edge case is an input at the boundary of what the code must handle, such as an empty list, zero, a negative number or a single item, where bugs like to hide.
Three kinds of error
Syntax errors are the easiest to fix, because Python finds them before running anything and points to the place:
for n in range(10) # SyntaxError: expected ':'
print(n)Look at the line Python points to, and the line before it: an unclosed bracket or quotation mark on one line is often only noticed on the next.
Runtime errors happen in programs that are grammatically fine but ask for something impossible. The most common ones:
| Exception | Typical cause | Example |
|---|---|---|
NameError | a misspelled name, or a name used before it is assigned | pritn("hi") |
TypeError | an operation on the wrong type of value | "Age: " + 12 |
ValueError | the right type but an unsuitable value | int("ten") |
IndexError | a list index that doesn't exist | [1, 2, 3][3] |
KeyError | a dictionary key that isn't there | {"a": 1}["b"] |
ZeroDivisionError | dividing by zero | len(xs) / 0 |
AttributeError | a method the value doesn't have | [3, 1].upper() |
Logic errors are the hardest, because nothing tells you about them. This function runs perfectly on every list you give it, yet it is wrong:
def average(values):
return sum(values) / (len(values) + 1)Only by checking its answers against answers you know, say average([2, 4]) should be 3 but is 2, do you discover the bug. That is exactly what tests are for.
Reading a traceback
Here is a small program that prints each class's average mark, and the traceback it produces:
def average(marks):
return sum(marks) / len(marks)
def report(classes):
for name in classes:
print(name, average(classes[name]))
report({"7A": [60, 75, 81], "7B": []})7A 72.0
Traceback (most recent call last):
File "report.py", line 8, in <module>
report({"7A": [60, 75, 81], "7B": []})
File "report.py", line 6, in report
print(name, average(classes[name]))
File "report.py", line 2, in average
return sum(marks) / len(marks)
ZeroDivisionError: division by zeroA traceback looks frightening at first, but it is the most helpful thing Python ever prints. Read it from the bottom up:
- The last line says what went wrong: a
ZeroDivisionError, with the message "division by zero". - The entry just above it says where: line 2, inside the function
average, and shows the line itself. - The entries above that say how the program got there: line 8, at the top level of the program (
<module>), calledreport, and line 6 insidereportcalledaverage. "Most recent call last" means the deepest call is at the bottom.
The traceback tells you where the program stopped, which is not always where the mistake is. Here, dividing by len(marks) is fine for any class that has marks; the real question is what should happen for a class with no marks, like 7B. Perhaps report should print "no marks yet", or perhaps average should return None for an empty list. Deciding that is a design decision, not a typo fix, and the traceback led you straight to it.
You can see a real traceback below. The program catches the error so that the page keeps working, and prints the traceback Python would have shown. The file name will be something like <exec> instead of report.py, and recent versions of Python also underline the exact part of the line that failed.
import traceback
def average(marks):
return sum(marks) / len(marks)
def report(classes):
for name in classes:
print(name, average(classes[name]))
try:
report({"7A": [60, 75, 81], "7B": []})
except ZeroDivisionError:
print(traceback.format_exc())A method for hunting bugs
When the program runs but gives a wrong answer, there is no traceback to guide you. Here is a method that works on bugs of every size.
1. Reproduce it, as small as possible. Find an input that shows the bug every time, then shrink it. A bug that appears on a list of 10 000 numbers is much easier to study once you find that it also appears on [-3, -1].
2. Make your guesses visible with print. The quickest way to test a hypothesis is often to print the values involved. Python has a handy shortcut for this: in an f-string, f"{x=}" prints the expression and its value, such as x=5. Print inside loops to see how values change, and check each printed value against what you expected it to be at that point.
def largest(numbers):
best = 0
for x in numbers:
if x > best:
best = x
return best
print(largest([3, 9, 4])) # 9, correct
print(largest([-3, -1, -7])) # should be -1. What do we get?
# Debugging: print the values inside the loop to test a hypothesis
def largest_debug(numbers):
best = 0
for x in numbers:
print(f" {x=} {best=}")
if x > best:
best = x
return best
print(largest_debug([-3, -1, -7]))The trace shows best=0 on every line: no negative number is ever bigger than 0, so best never changes. The hypothesis "the starting value is wrong" is confirmed. The fix is to start from a value that is actually in the list, best = numbers[0], and then to decide what an empty list should do. (One of the exercises asks you to make this fix.)
3. Explain the code aloud. Go through the program line by line and explain to someone what each line does and why. Very often, halfway through a sentence, you hear yourself say something the code doesn't actually do. This works so well that it works even when the listener is a toy: programmers call it rubber-duck debugging, after a story in the 1999 book The Pragmatic Programmer about a programmer who explained his code to a rubber duck on his desk.
4. Narrow it down. If a long program goes wrong somewhere, check the values halfway through. If they are already wrong, the bug is in the first half; otherwise it is in the second. Halving the search area each time finds the bug quickly, an idea you'll meet again as binary search in Searching and sorting.
5. Fix the cause, then make sure it stays fixed. Change the code that is actually wrong, not just the symptom. Then turn the input that showed the bug into a test, so that the bug can never quietly return.
Professional tools add more power: a debugger, built into most code editors, lets you pause a program at any line and inspect every variable, and Python's built-in breakpoint() function starts one. But the method is the same: observe, guess, test, conclude. And if you've been stuck for a long time, take a break. A rested mind often sees the mistake at once.
assert, and writing small tests
The statement assert condition, message says "I believe this is true here". If the condition is True, nothing happens. If it is False, Python raises an AssertionError with your message. Assertions are a way to make your assumptions visible and checked:
def percentage(part, whole):
assert whole > 0, "whole must be positive"
return 100 * part / whole(Python can be started in a mode that skips assertions, so they are for catching your own mistakes while developing. To reject bad input from users, programs raise an exception such as ValueError instead.)
A test is a small piece of code that calls a function with an input whose correct answer you know, and asserts that the function gives that answer. The checker in this course's exercises works exactly like this.
Choosing test cases
A good set of tests for a function includes:
- typical cases: ordinary inputs, the ones the function was written for;
- edge cases: an empty list, a single item, zero, negative numbers, all items equal, the first or last position;
- tricky cases: inputs related to how the function works inside, such as an even-length list for a median, or a year like 1900 for a leap-year rule;
- cases from past bugs, so that a fixed bug never comes back (these are called regression tests).
Example— Testing a median function
The median of a list of numbers is the middle value when they are sorted, or the average of the two middle values if there is an even number of them. The median of [7, 1, 4] is 4; the median of [7, 1, 4, 10] is the average of 4 and 7, which is 5.5. The code below tests a first attempt, written quickly, and a corrected version, with a tiny "test runner" that reports which tests fail instead of stopping at the first one.
def median_first_try(values):
return values[len(values) // 2]
def median(values):
ordered = sorted(values) # sorted() leaves the caller's list alone
n = len(ordered)
mid = n // 2
if n % 2 == 1:
return ordered[mid]
return (ordered[mid - 1] + ordered[mid]) / 2
tests = [
([5], 5), # edge case: one value
([1, 4, 7], 4), # typical, already sorted
([7, 1, 4], 4), # unsorted
([7, 1, 4, 10], 5.5), # even length
([2, 2, 2, 2], 2), # all equal
([-3, -8, -1], -3), # negatives
]
for f in [median_first_try, median]:
failures = 0
for values, expected in tests:
try:
got = f(values)
assert got == expected, f"{f.__name__}({values}) gave {got}, expected {expected}"
except AssertionError as e:
failures += 1
print(" FAIL:", e)
print(f"{f.__name__}: {len(tests) - failures} of {len(tests)} tests passed")Notice that the first attempt passes the one-value test, the sorted test and the all-equal test. Had you only tried those, you would believe it worked. Choosing tests that probe the edges and the tricky parts is what makes testing powerful.
Professionals test everything
In industry, testing is not an extra; it is part of the job. A typical professional project has thousands of unit tests, each checking one small function, plus larger tests that check whole features working together. The tests live alongside the code, written with a framework such as Python's built-in unittest or the popular pytest, and they are run automatically on every change by a continuous integration service: if a change makes any test fail, it is not accepted until it is fixed. Many teams also require that a second person reviews every change before it goes in, because a fresh pair of eyes catches what the author can't see. Some programmers even write the tests before the code, a practice called test-driven development: first a failing test that describes what the code should do, then just enough code to make it pass.
Lumenary does this too: every runnable box in this course, and the reference solution of every Python exercise, is run automatically by a test program, and a lesson is marked Code tested only when all of its code passes.
Testing has a limit, though, and you should know it. The computer scientist Edsger Dijkstra pointed out that tests can show that bugs are present, but never that they are absent: no matter how many inputs you try, there may be one you didn't. For code where mistakes are very costly, programmers combine testing with reasoning about why the code is correct for every input. You'll see how to argue that a loop is correct, using an invariant, in Searching and sorting, and this course has a whole lesson on professional testing later on.
Common misconceptions
Misconception
"No error message means the program works." Logic errors produce no message at all. The only way to find them is to check results against answers you know to be right.
Misconception
"The bug is on the line in the traceback." The traceback shows where the program stopped. The mistake may be earlier, in the code that produced a bad value, such as an empty list that should never have reached average.
Misconception
"If I change things until it works, it's fixed." If you don't know why a change worked, you may have hidden the bug rather than fixed it, and it will come back. Understand the cause, then fix it.
Misconception
"All my tests pass, so my code is correct." Passing tests is good evidence, but only about the inputs you tried. Ask what inputs you haven't tried: empty, negative, huge, repeated, unusual.
Misconception
"Good programmers don't need tests." The best programmers write the most tests, precisely because they know how easy it is to make a mistake, and how much cheaper it is to catch one early.
Optional Christian reflection · iron sharpens iron
A reflection from the Christian tradition that inspires Lumenary Academy. It is not part of the mathematics and you are welcome to skip it.
The book of Proverbs says, "Iron sharpeneth iron; so a man sharpeneth the countenance of his friend" (Proverbs 27:17, King James Version). Code review rests on the same humble idea: we see our own mistakes poorly, and we improve fastest when others are allowed to point them out, kindly, and when we are willing to hear it. Admitting "I was wrong; here's the fix" is a strength in a programmer. This reflection is optional, and nothing in the lesson depends on it.
Exercises
Several of these exercises give you buggy code. Before you change anything, run it, read what happens, and form a hypothesis about the cause.
1. Finding and fixing mistakes
Warm-upComputationalWhich exception does Python raise for this line? Type its name exactly, as it appears at the end of the traceback.
colours = ["red", "green", "blue"]
print(colours[3])
2. Finding and fixing mistakes
Warm-upHumanWhich of these is a logic error?
3. Finding and fixing mistakes
CoreComputationalA program stops with this traceback:
Traceback (most recent call last):
File "shop.py", line 14, in <module>
print(receipt(basket))
File "shop.py", line 9, in receipt
total += price_of(item)
File "shop.py", line 3, in price_of
return prices[item]
KeyError: 'mango'
Which statement is correct?
4. Finding and fixing mistakes
CoreHumanYou have written smallest(numbers), which should return the smallest number in a non-empty list. Which set of tests is most likely to catch a bug?
5. Finding and fixing mistakes
CoreComputationalThis is the buggy largest from the lesson. Fix it so that it returns the largest number in any non-empty list, including lists of negative numbers. For an empty list it should return None.
6. Finding and fixing mistakes
CoreComputationalThis median has two bugs. Find and fix both. The median is the middle value of the sorted numbers, or the average of the two middle values when there is an even number of them. The function must not change the list it is given. You may assume the list is not empty.
Run it first, and test it on [7, 1, 4] (should be 4) and [7, 1, 4, 10] (should be 5.5).
7. Finding and fixing mistakes
StretchComputationalNow you write the tests. Complete check_absolute(f), which should use assert to check that the function f correctly computes the absolute value (the distance from zero: 5 for both 5 and -5).
The checker will call check_absolute(abs) with Python's real abs, which must pass. Then it will call check_absolute on three buggy functions: one gets negative numbers wrong, one gets zero wrong, and one gets numbers with a fractional part (like -2.5) wrong. Your checks must catch all three (that is, raise an AssertionError).
Further reading and resources
- The Python Tutorial: Errors and exceptions · Python Software Foundation
- Built-in exceptions · Python Software FoundationWhat each kind of error means, from AssertionError to ZeroDivisionError.
- The assert statement · Python Software Foundation
- unittest: unit testing framework · Python Software FoundationThe testing framework that comes with Python.
- pytestThe most widely used third-party testing tool for Python.