Finding your way in the dark
Imagine waking up in a strange room in the middle of the night, with the lights off. How would you find the door?
You would probably stretch out your hands to feel for walls and furniture. You might slide your feet so you don't trip. You would count your steps, so you know roughly how far you have gone, and listen for sounds from the corridor. You would notice if the floor tilts. And you would not trust any single clue too much: a soft bump might be a curtain, not a wall.
A robot is in that situation all the time. It has no idea what is around it except what its sensors tell it, and every sensor is a bit like a hand in the dark: useful, limited and sometimes wrong. In the first lesson you met the sense–think–act loop. This lesson is about the first step, sense: what robots can measure, how the most common sensors work, and why a careful robot never trusts just one reading.
Senses inside and senses outside
Close your eyes and touch your nose. You can do it because your body has senses that report on itself: where your arm is, how your joints are bent, whether you are tilting. Other senses, such as sight and hearing, report on the world outside you. Robots have both kinds.
Definition— inside and outside sensors
A robot's inside sensors (engineers say proprioceptive) measure the robot itself: how far its wheels have turned, the angles of its joints, how fast it is spinning, how full its battery is.
Its outside sensors (exteroceptive) measure the world around it: distances to walls, what things look like, sounds, touch.
Inside senses: encoders and IMUs
Wheel encoders: counting turns
A wheel encoder counts how far a wheel or motor has turned. A simple one is a disc with slots, fixed to the wheel, with a tiny light shining through the slots onto a light sensor. As the wheel turns, the light flickers on and off, and the robot counts each flicker as a tick.
Example— How far has the robot gone?
A robot's wheels are 6.5 cm across, and each encoder disc has 20 slots, so there are 20 ticks per turn.
- One turn of the wheel rolls it along by its circumference: cm.
- So each tick means about cm.
- If the left wheel counts 500 ticks, it has rolled about cm, just over 5 metres.
If the two wheels count different numbers of ticks, the robot has turned. You will use exactly this idea to track a robot's position in Wheeled robots and odometry.
Encoders have a weakness: they count turns of the wheel, not movement of the robot. If a wheel slips on a smooth floor or spins on a rug's edge, the encoder still counts, and the robot thinks it has moved when it hasn't.
IMUs: which way is down, and how fast am I turning?
An IMU (inertial measurement unit) is a tiny chip, often smaller than your fingernail, that usually holds two sensors:
- an accelerometer, which feels pushes and also the pull of gravity, so it can tell which way is down (this is how your phone knows to turn the screen when you tilt it);
- a gyroscope, which measures how fast the robot is turning.
Many IMUs add a magnetometer, a compass that senses the Earth's magnetic field. Drones, phones, game controllers and almost every mobile robot carry an IMU. Its weakness is drift: to know which way it is facing, a robot adds up thousands of tiny gyroscope readings, and tiny errors add up too, so after a few minutes it may believe it is facing a few degrees away from where it really is.
Outside senses: touch, echoes, light and sound
Bumpers and touch. The simplest outside sensor is a switch that clicks when the robot hits something. Robot vacuums have a springy front bumper with switches behind it. Robot grippers may have force sensors in their fingers, so they can hold an egg without crushing it, and some research robots wear a "skin" of touch sensors.
Ultrasonic rangefinders: echoes. Bats and dolphins find their way by echoes, and so do many robots. An ultrasonic sensor sends out a short click of sound, far too high for people to hear (often 40,000 vibrations per second, while human hearing stops at about 20,000), and times how long the echo takes to come back.
Example— From an echo to a distance
In air at 20 °C, sound travels about 343 metres per second. A robot's ultrasonic sensor hears the echo 3 milliseconds (0.003 seconds) after its click.
- In that time the sound travels metres.
- That is the journey there and back, so the wall is half as far: metres, about 51 cm.
Forgetting to halve is the most common mistake with echo sensors: the robot would think the wall is twice as far away as it really is.
Ultrasonic sensors are cheap and work in the dark, but soft things such as curtains and cushions soak up the sound, and a wall hit at a slant bounces the echo away, so the robot may hear nothing at all.
Infrared sensors. These shine invisible infrared light and measure what bounces back. Robot vacuums point some of them straight down: these cliff sensors notice when the floor suddenly disappears at the top of a staircase, and the robot backs away. Infrared sensors can be fooled by very dark surfaces (which reflect little light) and by bright sunlight.
Lidar: laser echoes. A lidar sends out short pulses of laser light and times their return, exactly like an ultrasonic sensor but with light. Many lidars spin, measuring thousands of distances every second all the way round, which gives the robot a map of the room's outline. The lasers in robot lidars are low-power and designed to be eye-safe, but it is a good habit never to stare into any laser.
Cameras. A camera gives a robot more information than any other sensor: colours, shapes, writing, faces, the edges of roads. But a picture is just a grid of millions of coloured dots, and turning it into "a cup on the left, a person in the doorway" is hard work for the robot's computer, which you will meet in Cameras and computer vision. Cameras also need light and can be dazzled by glare.
Microphones. Robots use microphones to hear spoken commands and, with two or more microphones, to work out which direction a sound came from, by noticing which microphone heard it first. In some factories, microphones listen to machines for the changing sound of a part that is wearing out.
Satellite positioning (GPS). Outdoors, a robot such as a farm robot or a delivery drone can find its position from satellite signals, usually to within a few metres, or a few centimetres with extra correction signals. It does not work well indoors or under thick trees.
| Sensor | Measures | Good at | Trouble with |
|---|---|---|---|
| Wheel encoder | how far a wheel turned | precise, fast, cheap | wheel slip |
| IMU | tilt, turning speed | fast, works anywhere | drift over time |
| Bumper | "I hit something" | simple, reliable | only works after the crash |
| Ultrasonic | distance by echo | cheap, works in the dark | soft and slanted surfaces |
| Infrared | distance or "floor below?" | small, cheap | dark surfaces, sunlight |
| Lidar | many distances all round | accurate maps | glass and mirrors |
| Camera | pictures | richest information | darkness, glare, hard to interpret |
Noise: why one reading isn't enough
Point a distance sensor at a wall that is exactly 50 cm away and take five readings. You will not get 50, 50, 50, 50, 50. You might get 49.5, 51.0, 49.5, 49.4 and 48.1. The wall has not moved. The readings wobble because of tiny electrical disturbances, vibrations, air currents and the way the echo bounces. Every real sensor wobbles like this.
Definition— noise and bias
Noise is the random wobble in a sensor's readings: sometimes a bit too high, sometimes a bit too low, in no fixed pattern.
Bias is an error that pushes every reading the same way, for example a sensor that always reads 3 cm too far.
Analogy, not a formal construction
Think of an archer. If the arrows are scattered all around the bullseye, that is noise. If they are tightly grouped, but all a hand's width to the left, that is bias. A good archer, and a good sensor, needs both kinds of error to be small.
The good news about noise is that it partly cancels out. If some readings are too high and others too low, then the average (add them up, divide by how many) is usually closer to the truth than a single reading. In the next section you will check this in Python.
The bad news is that averaging does nothing about bias: if every reading is 3 cm too far, so is the average. The cure for bias is calibration: measure something whose true distance you know, see how far off the sensor is, and correct for it.
Simulating a noisy sensor
We can't attach a real sensor to this web page, but we can simulate one: write a program that behaves like a sensor, including its noise. Our pretend sensor faces a wall exactly 50 cm away. Each reading is 50 plus a small random number: most readings land within about 2 cm of the truth, and very few are more than 5 cm off.
The random numbers come from Python's random module. We give it a seed, a starting number, so that it produces the same "random" numbers every time you press Run. That makes the experiment repeatable, which is exactly what scientists and engineers want when they test a program. (If you have not used Python before, Lumenary CS's first programs will get you started.)
import random
rng = random.Random(7) # a fixed seed: the same "random" noise every run
TRUE_DISTANCE = 50.0 # the wall really is 50 cm away
def read_sensor():
# One reading: the true distance plus a little random noise.
return TRUE_DISTANCE + rng.gauss(0, 2.0)
print("Five single readings:")
for i in range(5):
print(f" {read_sensor():.1f} cm")
print("Five averages, each of 10 readings:")
for i in range(5):
total = 0.0
for j in range(10):
total = total + read_sensor()
print(f" {total / 10:.1f} cm")
print("How far off is a typical answer?")
for n in [1, 4, 16, 64]:
error_sum = 0.0
for trial in range(2000):
average = sum(read_sensor() for k in range(n)) / n
error_sum = error_sum + abs(average - TRUE_DISTANCE)
print(f" average of {n:>2} readings: off by {error_sum / 2000:.2f} cm")Here is what the output shows.
- The single readings range from 48.1 to 51.0 cm, spread over nearly 3 cm.
- The averages of 10 readings range from 49.2 to 50.6 cm: still not perfect, but much closer together and closer to 50.
- The last part repeats the experiment 2,000 times for each number of readings and reports how far off a typical answer is. One reading is typically off by about 1.61 cm; the average of 4 readings by 0.78 cm; of 16 readings by 0.40 cm; of 64 readings by 0.20 cm.
Averaging has a cost in time, too. If a sensor gives 20 readings a second, averaging 64 of them takes over 3 seconds, and a moving robot would have moved on by then. Engineers are always balancing accuracy against speed.
Try changing the program: make the noise bigger by changing 2.0 to 5.0; change the seed from 7 to another number and see that the details change but the pattern doesn't; or add a bias by changing TRUE_DISTANCE + rng.gauss(0, 2.0) to TRUE_DISTANCE + 3 + rng.gauss(0, 2.0) and watch the averages settle near 53, not 50.
How real robots combine their senses
No single sensor is good at everything, so real robots carry several and combine them, which engineers call sensor fusion. A robot vacuum uses its wheel encoders and gyroscope to keep track of how it has moved over the last second, its lidar or camera to recognise the room's walls and correct the drift, its bumper as a last resort, and its cliff sensors as a safety rule that overrides everything else.
Self-driving cars, which you will meet in Autonomous vehicles, carry cameras, radar (which measures distance and speed with radio waves and works well in rain and fog) and often lidar, and combine all of them. Most of the clever mathematics of robotics, such as the Kalman filter, is about combining uncertain readings from different sensors into the best possible guess. And if you want to know how a sensor's signal becomes a number inside a computer, Lumenary EE's Sensors and data conversion explains it.
Common misconceptions
Misconception
"Sensors tell the robot the truth." A reading is a clue, not a fact. Every sensor has noise, many have bias, and all of them can be fooled: a mirror can confuse a lidar, a curtain can swallow an echo. Robots are programmed to treat readings as evidence and to cross-check them.
Misconception
"If I average enough readings, the answer will be exact." Averaging shrinks noise but never removes bias, and it costs time. A thousand readings from a sensor that is 3 cm off will give an average that is 3 cm off.
Misconception
"More expensive sensors don't need any of this." Better sensors have less noise, but never none. Even the instruments on spacecraft are calibrated and their readings averaged and cross-checked.
Misconception
"A camera is all a robot needs, since people mostly use their eyes." People also use touch, balance, hearing and a sense of where their own body is, without noticing. Robots that only have a camera struggle in the dark, in glare and when something is too close to see. Many robots get a lot done with very simple sensors.
Exercises
Some of these exercises ask you to calculate, and two ask you to write a little Python. Open the hints one at a time if you get stuck.
1. How robots sense the world
Warm-upHumanSelect all the sensors below that are inside (proprioceptive) sensors: the ones that measure the robot itself rather than the world around it.
2. How robots sense the world
CoreHumanA robot's ultrasonic sensor hears the echo of its click 6 milliseconds (0.006 seconds) after sending it. Sound travels at 343 metres per second. How far away is the wall, in centimetres, to the nearest centimetre?
3. How robots sense the world
CoreHumanA robot's wheel rolls 20 cm along the floor every time it turns once, and its encoder gives 40 ticks per turn. On the way across a classroom, the encoder counts 300 ticks. Assuming the wheel never slips, how many centimetres has the wheel rolled?
4. How robots sense the world
CoreHumanAyşe tests her robot's distance sensor against a wall she has measured with a tape: it is exactly 100 cm away. Five readings are 103.1, 102.8, 103.3, 102.9 and 103.0 cm. What is the main problem, and what should she do?
5. How robots sense the world
Warm-upHumanWhich sensor stops a robot vacuum cleaner from driving off the top of a staircase?
6. How robots sense the world
Warm-upComputationalWrite a function average(readings) that returns the average of the numbers in the list readings: add them all up and divide by how many there are.
For the five readings from the lesson, [49.5, 51.0, 49.5, 49.4, 48.1], it should return 49.5 (or a number extremely close to it).
7. How robots sense the world
StretchComputationalUltrasonic sensors sometimes miss the echo completely, and then many of them report their largest possible distance, say 400 cm. One wild reading like that wrecks an average: the average of [51.2, 49.8, 400.0, 50.4, 49.1] is about 120 cm!
A better choice here is the middle reading: sort the readings and take the one in the middle. Write middle_reading(readings), which does that. You may assume the list always has an odd number of readings.
Further reading and resources
- What is lidar? · NOAA National Ocean ServiceA short, clear explanation of measuring distance with pulses of laser light.
- ROBOTS: your guide to the world of robotics · IEEE SpectrumLook up any robot and see which sensors it carries.
- micro:bit · Micro:bit Educational FoundationA small, inexpensive board with an accelerometer, a compass, a light sensor, a microphone and a thermometer built in, for experimenting with real sensors safely.