Skip to content
LumenaryAcademy
Start 50 min

Describing data

Kinds of data, the mean, median and mode (and when each one misleads), measures of spread, and the four charts every statistician draws first.

Code tested · 5AI-draftedAwaiting review by a statistician

Your guides: Chloe · The Statistician, Margaret · The Simulator, Gideon · The Sceptic

MotivationH

Too many numbers to see

Imagine your teacher hands you a sheet with the heights of 30 pupils in your class, measured in centimetres:

151, 146, 156, 139, 150, 157, 149, 153, 165, 150, 154, 142, 156, 150, 148, 151, 161, 143, 156, 152, 147, 150, 154, 157, 149, 162, 151, 153, 156, 150

What can you say about these heights? Staring at the list does not help much. Your eyes jump around, and it is hard to say even what a "typical" height is. Now imagine 30,000 heights, or the incomes of 30 million people.

This lesson gives you the tools statisticians use first, every time, to turn a pile of numbers into something a person can understand: a few well-chosen numbers that summarise the data, and a few well-chosen pictures that show its shape. These are the same 30 heights you met as a simulation in What is statistics?, so you can compare.

There are two questions to ask about almost any set of numbers:

  1. Where is the middle? What is a typical value?
  2. How spread out are they? Are the values all close together, or scattered far apart?

But before either question, there is an even earlier one: what kind of data do we have?

DefinitionsH

Kinds of data

Here is a small part of a class survey:

PupilCountry of birthFavourite fruitSiblingsHeight (cm)How much do you like maths?
InesPortugalmango2151a lot
TomaszPolandapple0146a little
ArohaNew Zealandbanana3156a lot
HamidAfghanistanmango1139somewhat

Definition— categorical and numerical data

A categorical variable puts each observation into a group or category: country of birth, favourite fruit, eye colour. A numerical variable is a number that you could sensibly do arithmetic with: number of siblings, height, time.

Numerical variables come in two kinds. Counts (sometimes called discrete data) can only be whole numbers: you can have 2 siblings or 3 siblings, but not 2.4. Measurements (sometimes called continuous data) can take any value in between: a height could be 151 cm, or 151.3 cm, or 151.27 cm, depending only on how precise your tape measure is.

Some categorical variables have a natural order: "a little", "somewhat", "a lot", or clothing sizes S, M, L. They are still categories, not numbers, even if someone writes them as 1, 2 and 3.

Why does the kind of data matter? Because it decides which summaries and which pictures make sense. You can find the average height, but there is no such thing as the "average country of birth"!

IntuitionH

The middle: mean, median and mode

There are three common ways to describe a typical value, and each answers a slightly different question.

Definition— mean

The mean is what you get if you add up all the values and then share the total out equally: add the values, then divide by how many there are. It is what most people call "the average".

Definition— median

The median is the middle value when the data are put in order from smallest to largest. If there is an even number of values, there are two middle values, and the median is halfway between them. Half the values are at or below the median, and half are at or above it.

Definition— mode

The mode is the value, or category, that appears most often. A data set can have two modes, or more, or none worth mentioning.

Example— Points in six netball matches

Nomvula scored 8, 13, 4, 10, 8 and 5 points in six netball matches.

  • Mean. The total is 8 + 13 + 4 + 10 + 8 + 5 = 48 points. Shared equally over 6 matches: 48 ÷ 6 = 8 points.
  • Median. In order: 4, 5, 8, 8, 10, 13. With six values the two middle ones are the third and fourth, 8 and 8, so the median is 8 points.
  • Mode. 8 appears twice and everything else once, so the mode is 8 points.

Here all three agree. That happens when the data are fairly balanced around the middle. Often they don't agree, and then the differences tell you something.

When the mean misleads: income

Nine people work at a small bakery. Their monthly pay, in an imaginary currency, is:

300, 310, 320, 320, 340, 350, 380, 400 and 2,500.

The first eight are the bakers; the 2,500 is the owner. The total is 5,220, so the mean pay is 5,220 ÷ 9 = 580. The median is the fifth value in order, 340. The mode is 320.

An advertisement that says "average pay at our bakery: 580!" would be telling the truth about the mean. But eight of the nine people earn less than 580, and none of the bakers earns anything close to it. One very large value has dragged the mean far above what is typical. The median, 340, describes a typical worker much better.

This is not a made-up problem. In nearly every country, a few people earn very much more than most, so the mean income is well above the median income. That is why statistics offices usually report the median income when they want to describe a typical household.

Lumenary Math proves this, and more, in Statistics: centre, spread and distributions.

When the median and mode mislead

The median is not always the right choice either. If you are planning how much flour a bakery needs for a month, you need the total, and the mean (total ÷ days) is what helps you; the median throws away information about the very busy days.

The mode can mislead too. Fifteen pupils said how many books they read over the holidays:

0, 0, 0, 0, 1, 2, 2, 3, 3, 4, 5, 5, 6, 8, 10.

The mode is 0, because four pupils read no books at all. But it would be wrong to say "the typical pupil read nothing": eleven of the fifteen read at least one, the median is 3 books, and the mean is 49 ÷ 15, about 3.3 books. The mode is most useful for categorical data, where it is the only one of the three that makes sense: the modal favourite fruit in a class is a perfectly good summary, but the "mean fruit" is nonsense.

SummaryGood forWatch out for
Meantotals and fair shares; balanced dataa few extreme values pull it a long way
Mediana "typical" value in lopsided data, like incomes or house pricesignores how big the extreme values are
Modecategories (favourite fruit, most common shoe size)can be at an edge, far from typical
DefinitionsH

How spread out? Range, IQR and standard deviation

In the first lesson, two bus routes had the same mean journey time, 23 minutes, but one was dependable and the other wildly variable. A summary of the middle is never enough: we also need to describe the spread.

Definition— range

The range is the largest value minus the smallest value.

The range is easy, but it depends on just two values, the most extreme ones, so a single unusual value can make it huge. The next measure ignores the extremes.

Definition— quartiles and interquartile range

Put the data in order and split them at the median into a lower half and an upper half (if there is an odd number of values, leave the median itself out of both halves). The lower quartile (Q1) is the median of the lower half, and the upper quartile (Q3) is the median of the upper half. The interquartile range (IQR) is Q3 − Q1: the width of the middle half of the data.

Example— Quartiles of the class heights

Put the 30 heights in order:

139, 142, 143, 146, 147, 148, 149, 149, 150, 150, 150, 150, 150, 151, 151, | 151, 152, 153, 153, 154, 154, 156, 156, 156, 156, 157, 157, 161, 162, 165

There are 30 values, so the median is halfway between the 15th and 16th, which are both 151: the median is 151 cm. The lower half is the first 15 values, and its middle (8th) value is Q1 = 149 cm. The upper half is the last 15, and its middle value is Q3 = 156 cm. So the IQR is 156 − 149 = 7 cm: the middle half of the class lies within a band just 7 cm wide, while the range is 165 − 139 = 26 cm.

The five numbers minimum, Q1, median, Q3, maximum (here 139, 149, 151, 156, 165) are called the five-number summary. They are what a box plot draws, as you will see below.

The third measure of spread is the one scientists use most.

Definition— standard deviation

The standard deviation measures how far values typically are from the mean. To work it out: find the mean; find how far each value is from the mean; square each of those distances; find the mean of the squares; take the square root.

That recipe sounds long, but it is only arithmetic. Here it is for Nomvula's netball points.

Points45881013
Distance from the mean (8)−4−30025
Distance squared16900425

The squared distances add up to 16 + 9 + 0 + 0 + 4 + 25 = 54, and their mean is 54 ÷ 6 = 9. The square root of 9 is 3, so the standard deviation is 3 points. In words: Nomvula's scores are typically about 3 points away from her mean of 8.

Why square the distances and then take a square root at the end? Because the plain distances, −4, −3, 0, 0, 2 and 5, always add up to zero (the mean is exactly the balancing point), so their average tells you nothing. Squaring makes every distance positive, and the square root at the end brings the answer back to the original units, points rather than "points squared".

The standard deviation of the 30 class heights works out at about 5.6 cm. For data with a single hump in the middle, like these heights, about two thirds of the values usually lie within one standard deviation of the mean. Here the mean is about 151.9 cm, and 23 of the 30 pupils, a little more than three quarters, are between 146.3 and 157.5 cm.

Note

Statisticians often use a slightly different version of the standard deviation, which divides by one less than the number of values (54 ÷ 5 instead of 54 ÷ 6). It gives a better estimate of the spread of a whole population when you only have a sample. You will see why in Estimation and confidence intervals. For large data sets the two versions are almost the same.

Explore

Four pictures of data

Numbers summarise; pictures show the whole shape at once. A good statistician always draws the data before calculating anything. Here are the four most useful pictures.

Bar charts are for categorical data. Each category gets a bar, and the height of the bar shows how many observations are in that category. The bars have gaps between them, because the categories are separate and their order often doesn't matter.

Histograms are for numerical data. Split the number line into equal intervals called bins (here 135–140 cm, 140–145 cm and so on), count how many values fall into each bin, and draw touching bars, because the number line has no gaps.

Bar chart: favourite fruitHistogram: heights (cm)05100510mangobananaapplelycheeorangepapaya135140145150155160165170count
Left: a bar chart of favourite fruit in a class of 30, a categorical variable, with gaps between the bars. Right: a histogram of the 30 heights, a numerical variable, in 5 cm bins with no gaps. The histogram shows one hump in the middle, at 150 to 155 cm.

Box plots draw the five-number summary. The box runs from Q1 to Q3, so it holds the middle half of the data; a line across the box marks the median; and two "whiskers" reach out to the smallest and largest values. Box plots are compact, so they are excellent for comparing several groups side by side, like the heights of three different classes. (Many box plots also show unusually distant values as separate dots, called outliers. These heights have none.)

min 139Q1 149Q3 156max 165median 151135140145150155160165170 cm
A box plot of the 30 class heights. The box holds the middle half of the class, from 149 to 156 cm; the line inside it is the median, 151 cm; the whiskers reach the shortest pupil (139 cm) and the tallest (165 cm).

Scatter plots are for two numerical variables measured on the same observations, like each pupil's height and arm span (the distance from fingertip to fingertip with arms stretched out sideways). Each pupil becomes one dot: across for height, up for arm span. The pattern of the dots shows whether the two variables go together.

140145150155160165140145150155160165height (cm)arm span (cm)
A scatter plot of height and arm span for 15 pupils (made-up data). Each dot is one pupil. The dots rise from bottom left to top right: taller pupils tend to have longer arm spans, and for most people the two are close.

Two pupils in this made-up class have exactly the same height and arm span (156 cm and 154 cm), so their dots sit on top of each other and you see 14 dots for 15 pupils. Real scatter plots often have overlapping dots like this, and statisticians have tricks to show them, such as making dots slightly see-through.

PictureKind of dataWhat it shows best
Bar chartone categorical variablewhich categories are common or rare
Histogramone numerical variablethe shape: where values pile up, how far they spread, lopsidedness
Box plotone numerical variable, often in several groupsthe middle, the middle half, the extremes; comparing groups
Scatter plottwo numerical variableswhether they go together, and how closely
ComputationC

Summaries and charts in Python

Python's built-in statistics module can find the mean, median and mode for you. Here are the bakery's pay and the holiday books. Press Run.

python
import statistics

# Monthly pay at a small bakery, in an imaginary currency: eight bakers and the owner.
pay = [300, 310, 320, 320, 340, 350, 380, 400, 2500]
print("Pay:   ", pay)
print("Mean:  ", statistics.mean(pay))
print("Median:", statistics.median(pay))
print("Mode:  ", statistics.mode(pay))
below = sum(1 for p in pay if p < statistics.mean(pay))
print(below, "of", len(pay), "people earn less than the mean")
print()

# Books read over the holidays by 15 pupils.
books = [0, 0, 0, 0, 1, 2, 2, 3, 3, 4, 5, 5, 6, 8, 10]
print("Books: ", books)
print("Mean:  ", round(statistics.mean(books), 2))
print("Median:", statistics.median(books))
print("Mode:  ", statistics.mode(books))
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Try changing the owner's 2500 to 25000, and watch which summaries move.

The next program works through the standard deviation step by step, for Nomvula's netball points, and then finds the five-number summary of the class heights. It uses NumPy, a Python library for calculating with whole lists of numbers at once: points - mean subtracts the mean from every value in one go.

python
import numpy as np

# Points scored by a netball player in six matches.
points = np.array([4, 5, 8, 8, 10, 13])
mean = points.mean()
distances = points - mean
print("Points:               ", points.tolist())
print("Mean:                 ", mean)
print("Distance from mean:   ", distances.tolist())
print("Squared distances:    ", (distances ** 2).tolist())
print("Average squared dist.:", (distances ** 2).mean())
print("Standard deviation:   ", np.sqrt((distances ** 2).mean()))
print("NumPy agrees:         ", points.std())
print()

# The 30 heights from the lesson "What is statistics?"
heights = np.array([139, 142, 143, 146, 147, 148, 149, 149, 150, 150, 150, 150, 150, 151, 151,
                    151, 152, 153, 153, 154, 154, 156, 156, 156, 156, 157, 157, 161, 162, 165])
lower, upper = heights[:15], heights[15:]        # the two halves of the sorted data
q1, median, q3 = np.median(lower), np.median(heights), np.median(upper)
print("Five-number summary:", heights.min(), q1, median, q3, heights.max())
print("Range:", heights.max() - heights.min(), " IQR:", q3 - q1)
print("Mean:", round(heights.mean(), 1), " Standard deviation:", round(heights.std(), 1))
print("NumPy's own quartiles:", np.percentile(heights, [25, 75]))
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Last, the four pictures, drawn with nothing but characters. In the box plot, | marks the smallest and largest values, [ and ] the quartiles, M the median, and each character is one centimetre.

python
import numpy as np

# 1. A bar chart: favourite fruit in a class of 30 (categories).
fruit = {"mango": 9, "banana": 7, "apple": 5, "lychee": 4, "orange": 3, "papaya": 2}
for name, count in fruit.items():
    print(f"{name:>7} | {'#' * count} {count}")
print()

# 2. A histogram: the 30 heights, grouped in 5 cm bins (numbers).
heights = np.array([139, 142, 143, 146, 147, 148, 149, 149, 150, 150, 150, 150, 150, 151, 151,
                    151, 152, 153, 153, 154, 154, 156, 156, 156, 156, 157, 157, 161, 162, 165])
counts, edges = np.histogram(heights, bins=range(135, 175, 5))
for count, left in zip(counts, edges):
    print(f"{int(left)}-{int(left) + 4} | {'#' * int(count)}")
print()

# 3. A box plot of the same heights, on a scale from 135 to 170 cm (one character per cm).
lo, q1, med, q3, hi = 139, 149, 151, 156, 165
line = [" "] * 36
for cm in range(lo, hi + 1):
    line[cm - 135] = "=" if q1 <= cm <= q3 else "-"
line[lo - 135], line[hi - 135] = "|", "|"
line[q1 - 135], line[q3 - 135] = "[", "]"
line[med - 135] = "M"
print("".join(line))
print("135       145       155       165")
print()

# 4. A scatter plot: height and arm span of 15 pupils (two numbers for each pupil).
rng = np.random.default_rng(8)
height = rng.integers(140, 166, size=15)
span = height + rng.integers(-4, 5, size=15)
for top in range(168, 136, -4):                       # arm span, from the top down
    cells = [" "] * 28
    for h, s in zip(height, span):
        if top - 4 < s <= top:
            cells[int(h) - 140] = "o"
    print(f"{top:4} | " + "".join(cells))
print("     +" + "-" * 29)
print("       140       150       160      height (cm)")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The ASCII scatter plot is rougher than the drawn one, because each row lumps together arm spans 4 cm apart, but the rising pattern is clear. Professional statisticians draw charts with plotting libraries such as Matplotlib in Python or ggplot2 in R, but the questions are the same: what kind of data is this, and which picture shows its shape honestly? You will meet more in the Lumenary CS elective on visualisation and storytelling.

Common misconceptionsH

Common misconceptions

Misconception

"The average is always the mean." In everyday speech "average" usually means the mean, but the median and the mode are averages too. When a news report says "the average house price", ask which one. For house prices and incomes, the median is usually more informative.

Misconception

"A histogram is just a bar chart." They look alike, but a bar chart shows categories (with gaps), while a histogram shows a number line cut into bins (with no gaps). In a histogram, the order of the bars matters and cannot be shuffled; in a bar chart of favourite fruit you could put the bars in any order.

Misconception

"If two data sets have the same mean, they are basically the same." Wanjiru's two bus routes had the same mean and completely different spreads. Always describe the spread, and always draw a picture.

Misconception

"An outlier is a mistake, so delete it." Sometimes an extreme value is a typing error (a height of 1,510 cm instead of 151 cm), and then it should be corrected. But often it is real: the owner really does earn 2,500. Deleting real values to make the data look tidy hides the truth. Check it, and if it is real, keep it and choose summaries, like the median, that are not thrown off by it.

Exercises

Exercises

Work out the summaries by hand first, then check yourself. The Python exercises ask you to finish short programs.

1. Describing data

Warm-upHuman

A class survey records these variables. Which of them are categorical? Choose every one that is.

Choose all that apply

2. Describing data

Warm-upHuman

Seven friends' shoe sizes (EU) are: 36, 37, 37, 38, 39, 39, 40. What is the mode? If there is more than one, give them all.

Write elements separated by commas. Order and repeats don’t matter — just like in a set!

3. Describing data

CoreHuman

Six pupils spent these numbers of minutes on homework last night: 12, 30, 18, 26, 40, 22. What is the median, in minutes?

4. Describing data

CoreHuman

In a small town, most houses sell for about 80,000 crowns, but each year two or three huge villas sell for several million. A family wants to know what a typical house in the town costs. Which summary of last year's prices should they look at?

Choose one

5. Describing data

CoreHuman

Seven pupils did these numbers of push-ups: 4, 6, 9, 10, 13, 15, 20. Using the lesson's method (find the median, leave it out, and take the median of each half), what is the interquartile range?

6. Describing data

CoreHuman

A sports teacher has measured how long each pupil in three different classes takes to run 400 metres, and wants one picture that compares the three classes' times. Which kind of chart suits this best?

Choose one

7. Describing data

CoreComputational

Here is the rainfall, in millimetres, for each month of one year in a made-up town with a monsoon season. Replace each ... so that:

  • mean is the mean monthly rainfall (use statistics.mean);
  • median is the median monthly rainfall (use statistics.median);
  • wet_months is how many months had more rain than the mean.

Then look at the output: why are the mean and median so different?

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

8. Describing data

StretchComputational

Follow the lesson's recipe for the standard deviation, step by step, for the data [1, 5, 5, 7, 7, 11]. Replace each ...:

  1. mean: the mean of the data;
  2. squares: a list of the squared distances of each value from the mean;
  3. average_square: the mean of squares;
  4. sd: the square root of average_square (use math.sqrt).
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Further reading and resources