Skip to content
LumenaryAcademy
Start 50 min

Chance and probability

Probability as what happens in the long run, coins and dice simulated by the hundred thousand, why no result is ever "due", and how to work out whether a game is fair.

Code tested · 5AI-draftedAwaiting review by a statistician

Your guides: Chloe · The Statistician, Margaret · The Simulator, Gideon · The Sceptic

MotivationH

What does a 70% chance of rain mean?

The weather forecast says there is a 70% chance of rain tomorrow. Tomorrow comes, and it stays dry all day. Was the forecast wrong?

Not necessarily! A 70% chance of rain does not promise rain. It means something like this: on days like tomorrow, it rains on about 70 out of every 100. On the other 30, it stays dry. To check whether a forecaster is any good, you cannot look at one day. You have to look at many days when they said "70%" and see whether it rained on about 70% of them.

That is the big idea of this lesson. A single event is uncertain, but in the long run, chance has patterns, and those patterns can be measured, predicted and used. Probability is the language for describing them, and it underpins all of statistics: every time a statistician asks "could this be chance?", probability supplies the answer.

IntuitionH

Probability as the long run

Toss a coin once: heads or tails, and you cannot say which. Toss it ten times: you might get 3 heads, or 7. But toss it ten thousand times and something remarkable happens: the proportion of heads (the number of heads divided by the number of tosses) gets very close to one half, and stays there.

010020030040050000.51number of tossesproportion of heads
The proportion of heads in a simulated run of 500 coin tosses. After 1 toss it is 1 (one head out of one); after 5 tosses it has dropped to 0.2. The early wobbles are wild, but as the tosses pile up, the proportion settles close to 0.5, the dashed line.

This settling down is called the law of large numbers. It is why we can say the probability of heads is one half: not because each toss is half heads (that makes no sense), but because in a long run of tosses, about half are heads.

Definition— probability

The probability of an event is the proportion of times it would happen if we could repeat the situation a very large number of times. It is a number from 0 (the event never happens) to 1 (it always happens). It can be written as a fraction, a decimal or a percentage: one half, 0.5 and 50% all mean the same.

01/61/21impossiblea sixeven chance: headscertaina 7 on one diea die shows 1 to 6
The probability scale runs from 0, impossible, to 1, certain.

Some events cannot be repeated, like "Will it snow on my next birthday?" or "Will this team win the final?" People still talk about their probability, meaning how strongly the evidence points one way. That second meaning, probability as a degree of belief, is just as useful in statistics, and you will meet it in Bayesian inference. In this lesson we stay with things we can repeat: coins, dice and games.

DefinitionsH

Counting equally likely outcomes

For a fair coin or a fair die, we don't need to toss it ten thousand times to know the long-run proportions, because every outcome is equally likely.

Definition— outcome, event, equally likely

An outcome is one possible result, like rolling a 4. An event is a set of outcomes we are interested in, like "rolling an even number" (2, 4 or 6). When all outcomes are equally likely, the probability of an event is the number of outcomes in the event divided by the total number of outcomes.

So for one fair die, the probability of an even number is 3 out of 6, which is 3/6 = 1/2. The probability of a six is 1/6, about 0.167 or 16.7%.

Example— Two dice

Roll a red die and a blue die and add the two numbers. What is the probability that the total is 7?

It is tempting to say: the totals go from 2 to 12, that's 11 possible totals, so the probability is 1/11. But the 11 totals are not equally likely. To find outcomes that are equally likely, list every pair of numbers: the red die has 6 possibilities and for each one the blue die has 6, so there are 6 × 6 = 36 equally likely pairs.

blue diered die123456123456234567345678456789567891067891011789101112
All 36 equally likely outcomes of rolling a red and a blue die, with the total in each square. The six squares with a total of 7 are shaded: they run diagonally across the grid.

Six of the 36 squares have a total of 7, so the probability is 6/36 = 1/6. A total of 2 needs a 1 on both dice, which is only one square: probability 1/36. In the long run, a 7 comes up six times as often as a 2. That is why in so many board games, 7 is the most common total.

IntuitionH

Independent events, and coins without memories

Definition— independent events

Two events are independent if knowing whether one happened does not change the probability of the other.

The two dice are independent: the red die has no idea what the blue die did. A coin toss and a die roll are independent. And one toss of a coin is independent of the next toss of the same coin.

For independent events there is a simple rule: the probability that both happen is the product of their probabilities. The probability of heads and a six is 1/2 × 1/6 = 1/12. You can check this by counting: a coin and a die have 2 × 6 = 12 equally likely pairs, and just one of them is (heads, six).

Not everything is independent. A bag holds 3 strawberry sweets and 2 lemon ones. Kwame takes one without looking and eats it; then Sara takes one. If Kwame's was strawberry, the bag now holds 2 strawberry and 2 lemon, and Sara's chance of strawberry is 2/4 = 1/2. If Kwame's was lemon, the bag holds 3 strawberry and 1 lemon, and her chance is 3/4. What happened first changes the probability of what happens second, so these two events are not independent. (Lumenary Math has the full rules in Conditional probability and independence.)

The gambler's fallacy

Here is a puzzle. You toss a fair coin five times, and get heads, heads, heads, heads, heads. What is the probability that the sixth toss is heads?

Many people feel that tails is now "due": after so many heads, the coin must surely "balance things out". But the coin has no memory. Each toss is independent of the others, so the probability of heads on the sixth toss is still exactly 1/2. The mistaken feeling has a name: the gambler's fallacy.

The fallacy has cost people a great deal of money. In a famous story from the casino at Monte Carlo in 1913, black came up on the roulette wheel over and over again, more than twenty times in a row. With every black, more players bet heavily on red, sure that red was overdue, and they lost again and again.

But wait: doesn't the law of large numbers say the proportion of heads must get back to one half? It does, but not by tails catching up. A streak of five extra heads is simply swamped: after a million more tosses, about half a million of them heads, those five extra heads make almost no difference to the proportion. The coin never corrects anything; the long run just makes the early streak unimportant.

Worked examplesH

Expected value: is the game fair?

At a school fair, the maths club runs a stall. It costs 2 tokens to roll a die once. If you roll a six, you win 9 tokens. Otherwise you win nothing. Should you play?

Nobody can say what will happen in one game. But we can work out what happens on average, in the long run.

Example— 600 games

Imagine the game is played 600 times. A six comes up about 1/6 of the time, so expect about 600 ÷ 6 = 100 sixes.

  • The players pay 600 × 2 = 1,200 tokens.
  • The stall pays out about 100 × 9 = 900 tokens.
  • So the players lose about 1,200 − 900 = 300 tokens in total, which is 300 ÷ 600 = half a token per game.

Definition— expected value

The expected value of a game is the average amount a player wins (or loses) per game in the long run. To find it, work out the average prize per game, then take away what it costs to play.

Here the average prize per game is 9 tokens × 1/6 = 1.5 tokens, and it costs 2 tokens to play, so the expected value is 1.5 − 2 = −0.5 tokens: a player loses half a token per game on average. A game is fair if its expected value is zero. For this game to be fair, the prize for a six would have to be 12 tokens, because 12 × 1/6 = 2.

"Expected value" is a slightly misleading name: you can never lose exactly half a token in one game. You lose 2 tokens or gain 7. The expected value describes the long run, not any single game.

That is exactly how casinos and lotteries work. Every game is designed so that the expected value for the player is negative. Some players win, sometimes a lot, but over millions of games the casino is as sure of its profit as you are that a coin will land heads about half the time. (Gambling for money is illegal for children in most countries, and it is designed so that players lose on average; the school fair, at least, is for a good cause.)

ComputationC

Simulating chance

A computer can toss a coin a hundred thousand times in a blink. Here is the long run from the first figure, continued. Press Run.

python
import numpy as np

rng = np.random.default_rng(1654)
flips = rng.integers(0, 2, size=100_000)       # 1 = heads, 0 = tails

print("   flips    heads   proportion of heads")
for n in [10, 100, 1_000, 10_000, 100_000]:
    heads = int(flips[:n].sum())
    print(f"{n:8,} {heads:8,}   {heads / n:.4f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

After 10 tosses this run had only 3 heads, a proportion of 0.3. After 100,000 it had 50,041, a proportion of 0.5004. Notice something subtle: the number of heads can drift further from exactly half (41 more than 50,000), while the proportion gets closer to one half. That is the law of large numbers at work, and it is also why a streak never needs to be "paid back".

Next, the gambler's fallacy, tested. The program tosses a coin 200,000 times, finds every moment when the last five tosses were all heads, and looks at what the very next toss was.

python
import numpy as np

rng = np.random.default_rng(1913)
flips = rng.integers(0, 2, size=200_000).tolist()     # 1 = heads, 0 = tails

# Find every place where five heads in a row have just happened, and look at the next flip.
next_flips = []
run = 0
for i in range(len(flips) - 1):
    run = run + 1 if flips[i] == 1 else 0
    if run >= 5:
        next_flips.append(flips[i + 1])

next_flips = np.array(next_flips)
print("Times we saw five (or more) heads in a row:", len(next_flips))
print("How often the NEXT flip was heads:", round(next_flips.mean(), 4))
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Five heads in a row happened 6,362 times, and the next toss was heads 50.44% of the time: one half, give or take a little chance. Tails was never "due".

Last, the school-fair game, played 600 times:

python
import numpy as np

rng = np.random.default_rng(42)
games = 600
rolls = rng.integers(1, 7, size=games)

price = 2          # tokens to play once
prize = 9          # tokens paid out for rolling a six

sixes = int((rolls == 6).sum())
taken = games * price
paid = sixes * prize
print("Games played:     ", games)
print("Sixes rolled:     ", sixes)
print("Tokens taken in:  ", taken)
print("Tokens paid out:  ", paid)
print("Stall's profit:   ", taken - paid)
print("Player's average result per game:", round((paid - taken) / games, 3), "tokens")
print("Expected value per game (exact): ", round(prize / 6 - price, 3), "tokens")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

In this run there were 95 sixes rather than exactly 100, so the players lost a little more than expected: 0.575 tokens per game against the long-run −0.5. Change prize = 9 to prize = 12 to see a fair game, and games = 600 to games = 600_000 to watch the average settle close to the expected value.

History & global context

Where probability came from

People have played games of chance for thousands of years, with knucklebones, dice and lots, but the mathematics of chance is surprisingly young. In sixteenth-century Italy, Gerolamo Cardano wrote a book on games of chance in which he counted equally likely outcomes. In 1654, the French mathematicians Blaise Pascal and Pierre de Fermat exchanged letters about how to share out the stakes of an unfinished game fairly, and their answer contained the idea of expected value. Lumenary Math tells this story in Probability.

The Swiss mathematician Jacob Bernoulli spent some twenty years on the question this lesson started with: why do proportions settle down in the long run? His proof of the first law of large numbers was published in 1713, after his death. Since then, probability has grown from a theory of games into the foundation of statistics, insurance, weather forecasting, genetics and modern computing.

Common misconceptionsH

Common misconceptions

Misconception

"After a run of tails, heads is due." This is the gambler's fallacy. A fair coin has no memory, so each toss has probability 1/2 of heads, whatever happened before. Lottery numbers that "haven't come up for ages" are no more likely than any others.

Misconception

"A probability of 1/6 means a six will come up once in every six rolls." In the long run, about one roll in six is a six. In any particular six rolls you might get no sixes, or three. In fact, the chance of getting no six at all in six rolls is about one in three.

Misconception

"If there are 11 possible totals, each has probability 1/11." Counting works only when the outcomes are equally likely. The totals of two dice are not; the 36 pairs are.

Misconception

"The forecast said 70% and it didn't rain, so the forecast was wrong." A 70% forecast expects dry weather on about 30% of such days. You can judge a forecaster only over many forecasts.

Exercises

Exercises

Count carefully, and when in doubt, simulate. The Python exercises let you build your own simulations.

1. Chance and probability

Warm-upHuman

The probability of rolling a six with a fair die is 1/6. What does that mean?

Choose one

2. Chance and probability

CoreHuman

You roll a red die and a blue die. Of the 36 equally likely outcomes, how many give a total of 8? (Then the probability of a total of 8 is that number out of 36.)

3. Chance and probability

CoreHuman

Anaïs tosses a fair coin six times and gets tails every time. What is the probability that her seventh toss is heads?

Choose one

4. Chance and probability

CoreHuman

Which pair of events is independent?

Choose one

5. Chance and probability

CoreHuman

A spinner has 4 equal parts, and one of them is gold. It costs 3 tokens to spin once, and if it stops on gold you win 10 tokens. If people play 400 times, about how many tokens do the players lose in total?

6. Chance and probability

CoreComputational

The code rolls a red die and a blue die 10,000 times each, with a fixed seed. Replace each ... so that:

  • totals holds the 10,000 totals (red plus blue);
  • proportion_seven is the proportion of totals equal to 7.

Compare your answer with the exact probability 1/6, about 0.1667.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

7. Chance and probability

StretchComputational

Write a function expected_value(price, prize, chance) that returns the expected value of a game for the player: the average prize per game (the prize times the chance of winning it) minus the price of playing.

For the school-fair game, expected_value(2, 9, 1/6) should give about −0.5. Then set fair_prize to the prize that would make that game exactly fair.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Further reading and resources