Same bits, different meanings
Here are four bytes, written in hexadecimal: 48 69 21 00. What are they?
- Read as ASCII text, they are the letters
H,i,!followed by a zero byte: "Hi!". - Read as an unsigned 32-bit integer on a typical laptop, they are 2,189,640.
- Read as the same integer on a machine that orders bytes the other way, they are 1,214,849,280.
- Read as a 32-bit floating-point number in that second order, they are 238,724.0.
Memory, files and network packets contain only bits, grouped into bytes. Nothing in the bytes says what they mean. Meaning comes from a representation: an agreement between whoever writes the bits and whoever reads them. Most of the bugs this lesson will help you avoid come from the two sides disagreeing: text saved in one encoding and read in another, a number written in one byte order and read in the other, a value too big for the bits set aside for it.
This lesson is the first step down through the layers of a computer. Next, From logic gates to a processor shows how circuits compute with these representations. You met the float surprise 0.1 + 0.2 in Values and variables; here you will see exactly where it comes from.
Bits, bytes, binary and hexadecimal
Definition— bit, byte, word
A bit is a single binary digit, 0 or 1. A byte is 8 bits, the smallest unit most computers address in memory, with possible values. A word is the natural unit of a processor's registers and arithmetic: 64 bits on today's laptops, phones and servers.
Binary is positional notation in base 2, exactly like decimal in base 10 but with place values :
Long binary strings are hard for people to read, so we write them in hexadecimal, base 16, with digits 0–9 and A–F (for 10–15). Because , each hex digit stands for exactly four bits, a nibble, and a byte is always two hex digits: 1011 0110 is B6. Converting between binary and hex is a lookup, not arithmetic, which is why memory addresses, colours (#1E90FF), hashes and machine code are written in hex. Python writes literals as 0b10110110, 0xB6 and (for octal, base 8, which you will meet in file permissions) 0o266, and converts with bin, hex, int("B6", 16) and format specifiers such as f"{x:08b}".
| Bytes | Bits | Distinct values | A typical use |
|---|---|---|---|
| 1 | 8 | 256 | a pixel channel, a UTF-8 code unit |
| 2 | 16 | 65,536 | an audio sample, a network port number |
| 4 | 32 | about 4.3 billion | an IPv4 address, a float32 |
| 8 | 64 | about | a memory address, a float64, a timestamp |
Integers: unsigned, two's complement, and overflow
Unsigned integers
With bits, an unsigned integer represents to . Hardware arithmetic on -bit numbers keeps only the low bits of the result, so it is arithmetic modulo : in 8 bits, , and the result is 44. This is overflow, and in most languages it happens silently. (Lumenary Math's clock arithmetic lesson is exactly the right mental model.)
Two's complement
To represent negative numbers, almost every computer designed since the 1970s uses two's complement.
Definition— two's complement
An -bit pattern represents the integer
The top bit has a negative weight. The range is to : for 8 bits, to .
So 1111 1111 is , 1000 0000 is , and 0111 1111 is . A pattern whose top bit is 1 is negative, so the top bit works as a sign bit, but the encoding is not "a sign plus a magnitude", and that is the point:
Theorem— One adder for both
For any -bit pattern, its two's complement value and its unsigned value are congruent modulo . Consequently the same -bit adder circuit, which computes sums modulo , gives correct results for both signed and unsigned numbers whenever the true result fits in the range.
Proof
The two readings differ only in the weight of the top bit: for unsigned, for two's complement. So unsigned value minus signed value equals , which is a multiple of . Addition modulo respects congruence, so adding the patterns with an unsigned adder produces a pattern congruent to the true signed sum; if that sum lies in , it is the unique value in the range with that residue, so the pattern reads back correctly.
∎
Two practical consequences. To negate a number, invert every bit and add 1 (because is all ones, that is , so ). To widen a number from 8 to 32 bits, copy its top bit into the new positions (sign extension): 1111 1011 () becomes 1111 … 1111 1011, still . And one trap: the range is lopsided, so does not fit in 8 bits and comes back as ; in C, abs(INT_MIN) is undefined behaviour.
def to_unsigned(x, bits):
"""The bit pattern of x in `bits` bits, read as an unsigned number (reduce mod 2**bits)."""
return x & ((1 << bits) - 1)
def to_signed(pattern, bits):
"""Read a `bits`-bit pattern as two's complement: the top bit has weight -2**(bits-1)."""
return pattern - (1 << bits) if pattern >> (bits - 1) else pattern
for x in [5, -5, 127, -128, -1]:
p = to_unsigned(x, 8)
print(f"{x:5} -> bits {p:08b} = 0x{p:02X} (as unsigned: {p:3}, as signed: {to_signed(p, 8):4})")
# Overflow: an 8-bit or 32-bit adder silently wraps around.
print("127 + 1 in int8:", to_signed(to_unsigned(127 + 1, 8), 8))
print("200 + 100 in uint8:", to_unsigned(200 + 100, 8))
print("2**31 - 1 + 1 in int32:", to_signed(to_unsigned(2**31 - 1 + 1, 32), 32))
# Negation is "invert every bit, then add 1".
x = 0b00010110 # 22
print(f"~x + 1 = {to_unsigned(~x + 1, 8):08b} = {to_signed(to_unsigned(~x + 1, 8), 8)}")
# Bitwise operations on flags: one bit per permission.
READ, WRITE, EXECUTE = 0b100, 0b010, 0b001
mode = READ | WRITE # set two flags
print("can write:", bool(mode & WRITE), " can execute:", bool(mode & EXECUTE))
mode &= ~WRITE # clear a flag
mode ^= EXECUTE # toggle a flag
print(f"mode now {mode:03b}")
# Two classic tricks.
n = 0b1011000
print(f"n & (n - 1) clears the lowest 1-bit: {n:b} -> {n & (n - 1):b}")
print("is a power of two:", [k for k in range(1, 70) if k & (k - 1) == 0])
print("bits set in 2026:", bin(2026), bin(2026).count("1"))Python's own int never overflows: it grows to as many bits as needed, so the code above simulates fixed-width hardware with masks. Most other languages (C, C++, Java, Go, Rust in release builds) and NumPy arrays use fixed widths, and overflow there is real. In 1996 the first Ariane 5 rocket was destroyed about forty seconds after launch when a 64-bit floating-point value was converted to a 16-bit signed integer that could not hold it. And on 19 January 2038, a signed 32-bit count of seconds since 1970 will overflow; systems that still store time that way will jump back to 1901.
Bitwise operations
Python's bitwise operators act on the binary representation:
| Operator | Name | Example (8 bits) | Typical use |
|---|---|---|---|
a & b | AND | 1100 & 1010 = 1000 | test or clear bits with a mask |
a | b | OR | 1100 | 1010 = 1110 | set bits |
a ^ b | XOR | 1100 ^ 1010 = 0110 | toggle bits; checksums; "differs" |
~a | NOT | ~0000 1100 = 1111 0011 | invert a mask |
a << k | left shift | 0000 0011 << 2 = 0000 1100 | multiply by ; build bit fields |
a >> k | right shift | 0000 1100 >> 2 = 0000 0011 | divide by ; extract bit fields |
Flags packed into one integer, as in the code's READ | WRITE, are how operating systems store file permissions (Operating systems), how network protocols pack header fields, and how a CPU decodes instruction words in the next lesson. Two Python details: since ints are unbounded, ~x is (think of an infinite row of sign bits), and >> on a negative number is an arithmetic shift that keeps the sign: -8 >> 1 == -4.
Floating point: IEEE 754
Integers cannot represent , or . For those, nearly every computer uses the IEEE 754 standard (first published 1985, current revision 2019), which is scientific notation in base 2.
Definition— IEEE 754 binary64 (a Python float)
A 64-bit pattern is split into a sign bit , an 11-bit exponent field and a 52-bit fraction field . For (a normal number) it represents
where means the binary number "1." followed by the 52 fraction bits. The leading 1 is implicit, so 53 significant bits are stored in 52. Exponent field 0 encodes zero and the subnormal numbers (no implicit 1, for gradual underflow near zero); field 2047 encodes infinities () and NaN, "not a number" ().
Example— Encoding 0.1
In binary, : the block 0011 repeats for ever, just as does in decimal, because 10 has the factor 5, which 2 lacks. Normalised, , so the exponent field is . The fraction must be cut to 52 bits, and IEEE 754 rounds to nearest (ties to even): the bits after the cut begin 1001…, more than half a unit, so the last kept bit rounds up. The stored value is slightly more than 0.1, by about .
import math, struct
from decimal import Decimal
def fields(x):
"""Split a Python float (IEEE 754 binary64) into its sign, exponent and fraction bits."""
(bits,) = struct.unpack(">Q", struct.pack(">d", x))
return bits >> 63, (bits >> 52) & 0x7FF, bits & ((1 << 52) - 1)
for x in [1.0, -2.5, 0.1]:
s, e, f = fields(x)
print(f"{x:>5}: sign {s} exponent {e:011b} ({e} - 1023 = {e - 1023:3}) fraction {f:052b}")
print("value stored for 0.1: ", Decimal(0.1))
print("value stored for 0.2: ", Decimal(0.2))
print("0.1 + 0.2 rounds to: ", Decimal(0.1 + 0.2))
print("value stored for 0.3: ", Decimal(0.3))
print("as hexadecimal floats:", (0.1 + 0.2).hex(), "vs", (0.3).hex())
print("machine epsilon:", 2.0 ** -52 == math.ulp(1.0), math.ulp(1.0))
print("next float after 1.0:", math.nextafter(1.0, 2.0))
print("2**53 + 1 == 2**53 as floats:", float(2**53 + 1) == float(2**53))
print("isclose(0.1 + 0.2, 0.3):", math.isclose(0.1 + 0.2, 0.3))
values = [0.1] * 10
print("sum of ten 0.1s:", sum(values), " math.fsum:", math.fsum(values))
print("(a + b) + c vs a + (b + c):", (1e16 + 1.0) + 1.0, 1e16 + (1.0 + 1.0))
inf, nan = float("inf"), float("nan")
print("specials:", 1e308 * 10, -1e308 * 10, inf - inf, nan == nan, 0.0 == -0.0, math.copysign(1, -0.0))
print("smallest subnormal:", 5e-324, " half of it:", 5e-324 / 2)
print("money in decimal:", Decimal("0.10") + Decimal("0.20"))Why 0.1 + 0.2 != 0.3
Three roundings happen. The literal 0.1 is rounded to the nearest float, slightly above 0.1. The literal 0.2 is rounded too, also slightly above. Their exact sum is then rounded to the nearest float, and that float is 0x1.3333333333334p-2. Meanwhile the literal 0.3 is rounded on its own to the nearest float to 0.3, which is slightly below 0.3: 0x1.3333333333333p-2. The two differ in the last bit, one unit in the last place (ULP). Neither side is "wrong"; each is the correctly rounded result of what was asked. The surprise comes from expecting decimal fractions to be exact in a binary format.
Rules for working with floats
- Every operation rounds, but correctly. IEEE 754 requires and to return the exactly rounded result, so a single operation has relative error at most . Errors come from accumulating roundings and from cancellation.
- Compare with a tolerance. Use
math.isclose(a, b)(relative tolerance by default) or an explicit tolerance that fits your problem, not==. - Arithmetic is not associative. , because at the spacing between floats is 2. Summation order changes results;
math.fsumreturns the correctly rounded sum. - Subtracting nearly equal numbers loses digits (catastrophic cancellation). Rearranging formulas to avoid it is a core skill of numerical analysis.
- Integers are exact up to . Beyond that, floats skip odd integers. IDs and money do not belong in floats; use integers (cents, paise, satoshis) or
decimal.Decimal. - Know the special values. Overflow gives
inf; invalid operations such as givenan, andnan != nan, so test withmath.isnan. There are two zeros, and , equal under==but with different signs.
Machine learning has pushed the other way, towards fewer bits. Training and inference use 32-bit, 16-bit (float16, bfloat16) and even 8-bit floating-point formats, trading precision for speed and memory; Accelerators and parallelism explains why that works.
Characters: ASCII, Unicode and UTF-8
ASCII
In 1963 the American Standard Code for Information Interchange assigned the numbers 0–127 to the English alphabet, digits, punctuation and control characters: A is 65 (0x41), a is 97 (0x61), the digit 0 is 48 (0x30), newline is 10. Seven bits suffice, and the layout has neat bit patterns (upper and lower case differ only in bit 5). But 128 symbols cannot write Hindi, Arabic, Chinese or even French. For decades every region used its own incompatible 8-bit extension or multi-byte scheme, and text moved between systems arrived as garbage.
Unicode code points
Unicode replaced that tangle with one numbering for every character of every writing system, plus symbols and emoji. Each character gets a code point, an integer written U+ followed by hex digits: U+0041 is A, U+00E9 is é, U+0928 is the Devanagari letter न, U+4E2D is 中, U+1F30D is the globe emoji 🌍. Code points range from U+0000 to U+10FFFF, room for 1,114,112, of which well over a hundred thousand are assigned. In Python, ord("é") is 233 and chr(233) is "é", and a str is a sequence of code points. A code point is an abstract number; to store or send text we need an encoding that turns code points into bytes.
UTF-8
UTF-8, designed by Ken Thompson and Rob Pike in 1992 and standardised in RFC 3629, is the encoding of the web, of Linux and macOS file names, and of most source code. It uses one to four bytes per code point:
| Code points | Bytes | Bit pattern (x = bits of the code point) |
|---|---|---|
| U+0000 – U+007F | 1 | 0xxxxxxx |
| U+0080 – U+07FF | 2 | 110xxxxx 10xxxxxx |
| U+0800 – U+FFFF | 3 | 1110xxxx 10xxxxxx 10xxxxxx |
| U+10000 – U+10FFFF | 4 | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx |
Example— Encoding € (U+20AC) by hand
, which needs 14 significant bits, so it falls in the three-byte row with 16 slots. Split the 16 bits as 4 + 6 + 6: 0010, 000010, 101100. Fill the templates: 1110 0010, 10 000010, 10 101100, which is E2 82 AC.
The design has properties worth knowing by heart. ASCII is UTF-8: every ASCII file is already valid UTF-8, which made adoption painless. It is self-synchronising: continuation bytes always start 10 and lead bytes never do, so a program dropped into the middle of a stream can find the next character boundary, and a corrupted byte damages only one character. No byte of a multi-byte character is below 128, so searching for / or a newline byte never matches inside another character. Byte order does not matter, since the unit is a single byte. And sorting UTF-8 strings bytewise sorts them by code point.
def utf8_encode(cp):
"""Encode one code point as UTF-8 bytes, following the table in RFC 3629."""
if cp < 0x80: # 0xxxxxxx
return bytes([cp])
if cp < 0x800: # 110xxxxx 10xxxxxx
return bytes([0xC0 | cp >> 6, 0x80 | cp & 0x3F])
if cp < 0x10000: # 1110xxxx 10xxxxxx 10xxxxxx
return bytes([0xE0 | cp >> 12, 0x80 | cp >> 6 & 0x3F, 0x80 | cp & 0x3F])
return bytes([0xF0 | cp >> 18, 0x80 | cp >> 12 & 0x3F, # 11110xxx and three continuation bytes
0x80 | cp >> 6 & 0x3F, 0x80 | cp & 0x3F])
for ch in ["A", "é", "€", "中", "न", "🌍"]:
cp = ord(ch)
mine = utf8_encode(cp)
assert mine == ch.encode("utf-8")
print(f"{ch} U+{cp:04X} {len(mine)} byte(s): {' '.join(f'{b:08b}' for b in mine)}")
text = "Grüße from Śrī Laṅkā \U0001F30D"
print(text, "-", len(text), "code points,", len(text.encode("utf-8")), "bytes in UTF-8,",
len(text.encode("utf-16-le")), "bytes in UTF-16")
print("the globe in UTF-16 is a surrogate pair:", "\U0001F30D".encode("utf-16-be").hex(" ", 2))
# Mojibake: bytes written as UTF-8 but read back as Latin-1.
print("café".encode("utf-8").decode("latin-1"))
try:
b"caf\xe9 noir".decode("utf-8") # Latin-1 bytes are not valid UTF-8
except UnicodeDecodeError as err:
print("UnicodeDecodeError:", err.reason)The last lines show the two classic failures. Mojibake: the UTF-8 bytes C3 A9 of é, read as Latin-1, come out as the two characters é; if you have ever seen "café" on a web page, this is what happened. And bytes in a legacy encoding are usually invalid UTF-8, which Python reports loudly rather than guessing. The rule that avoids both: decode bytes to str at the edges of your program, with an explicit encoding (almost always UTF-8), work with str inside, and encode on the way out.
UTF-16, the internal string format of Java, JavaScript and Windows, uses 2 bytes for code points up to U+FFFF and a surrogate pair of two 16-bit units for the rest, so in JavaScript "🌍".length is 2. UTF-32 uses 4 bytes for everything: simple, but wasteful. Remember too that a code point is not always what a reader calls a character: accents can be separate combining code points and emoji can be sequences, which String algorithms discussed under normalisation and grapheme clusters.
Endianness: which byte comes first?
A 32-bit integer occupies four bytes of memory. In which order?
Definition— big-endian, little-endian
A big-endian layout stores the most significant byte at the lowest address: 0x0A0B0C0D is stored as 0A 0B 0C 0D. A little-endian layout stores the least significant byte first: 0D 0C 0B 0A.
Both are reasonable. Little-endian lets a narrower read at the same address see the low part of the number; big-endian reads naturally in a hex dump. x86-64 processors are little-endian, and ARM and RISC-V systems almost always run little-endian, so nearly every machine you will program is. But many file formats and network protocols are big-endian: the Internet protocols define "network byte order" as big-endian (you will parse such headers in Networks), as do PNG and JPEG, while ZIP and BMP files are little-endian. Whenever bytes cross a boundary, between machines or into a file, the format must say which order it uses, and your code must say which it reads. The names come from Gulliver's Travels, whose Lilliputians war over which end of a boiled egg to break; Danny Cohen borrowed them in a 1980 note about exactly this argument.
import struct, sys
data = b"Hi!\x00"
print("as text: ", data.rstrip(b"\x00").decode("ascii"), "followed by a zero byte")
print("as little-endian: ", int.from_bytes(data, "little"), " struct '<I':", struct.unpack("<I", data)[0])
print("as big-endian: ", int.from_bytes(data, "big"), " struct '>I':", struct.unpack(">I", data)[0])
print("as float32 (big): ", struct.unpack(">f", data)[0])
print("this machine is", sys.byteorder, "endian")
n = 0x0A0B0C0D
print("0x0A0B0C0D little-endian bytes:", n.to_bytes(4, "little").hex(" "))
print("0x0A0B0C0D big-endian bytes: ", n.to_bytes(4, "big").hex(" "))
# The start of a PNG file: an 8-byte signature, then the IHDR chunk, all big-endian ("network order").
png = b"\x89PNG\r\n\x1a\n" + struct.pack(">I4sIIBBBBB", 13, b"IHDR", 1920, 1080, 8, 6, 0, 0, 0)
length, kind, width, height, depth = struct.unpack(">I4sIIB", png[8:8 + 17])
print(f"PNG header: chunk {kind.decode()} of {length} bytes, {width} x {height}, {depth}-bit")
wrong_w, wrong_h = struct.unpack("<II", png[16:24])
print(f"read with the wrong byte order: {wrong_w} x {wrong_h}")This is the motivation's puzzle solved: the same four bytes are text, two different integers and a float, depending only on how you agree to read them. In struct format strings, < means little-endian, > big-endian, and ! network order (big-endian); I is an unsigned 32-bit integer, d a 64-bit float, B an unsigned byte. Always write the byte-order character explicitly; the default is the machine's native order, which makes code that works on your laptop and fails on someone else's.
Common misconceptions
Misconception
"Hexadecimal numbers are a different kind of number." 0xB6, 0b10110110 and 182 are the same number written in different bases. Hex is a notation for humans; the machine stores bits either way.
Misconception
"Floating-point arithmetic is random or sloppy." It is exactly specified and deterministic: each operation returns the correctly rounded result. Surprises come from values that are not representable (like 0.1) and from rounding errors that accumulate. The same program on the same inputs gives the same answer, bit for bit, unless the order of operations changes.
Misconception
"One character is one byte." Only in ASCII. In UTF-8 a code point takes one to four bytes, and what a reader sees as one character may be several code points. len(s.encode("utf-8")) and len(s) answer different questions.
Misconception
"UTF-8 and Unicode are the same thing." Unicode is the numbering of characters (code points); UTF-8 is one way of turning those numbers into bytes. UTF-16 and UTF-32 are others.
Misconception
"Overflow raises an error." In Python's int there is no overflow; in C, Java, Go and NumPy arrays, integer overflow usually wraps silently (and for signed integers in C it is undefined behaviour, which compilers are allowed to assume never happens). Rust checks in debug builds and wraps in release builds unless you ask otherwise.
Exercises
1. Representing data: bits and bytes
Warm-upHumanThe byte 0xD3, read as an unsigned 8-bit integer, is which decimal number?
2. Representing data: bits and bytes
Warm-upHumanThe same byte 0xD3, read as an 8-bit two's complement integer, is which decimal number?
3. Representing data: bits and bytes
CoreComputationalProcessors report two flags after an addition: carry (the unsigned result did not fit) and overflow (the signed result did not fit). Write add_with_flags(a, b, bits) where a and b are bit patterns, integers from 0 to . Return a tuple (result, carry, overflow): result is the bits-bit pattern of the sum, carry is True if the true unsigned sum is at least , and overflow is True if a and b have the same sign bit but result has a different one.
4. Representing data: bits and bytes
CoreHumanWhich explanation of 0.1 + 0.2 != 0.3 in Python is correct?
5. Representing data: bits and bytes
CoreComputationalWrite float_from_fields(sign, exponent, fraction) that returns the value of a normal IEEE 754 binary64 number from its three fields: sign is 0 or 1, exponent is the 11-bit field (1 to 2046), and fraction is the 52-bit field as an integer. Use the formula from the lesson, , not the struct module. (Python's float arithmetic is exact for this computation when you scale with math.ldexp, or with 2.0 ** k.)
6. Representing data: bits and bytes
CoreHumanHow many bytes does the string naïve café take when encoded in UTF-8? It has 10 code points (count the space), with ï (U+00EF) and é (U+00E9) precomposed.
7. Representing data: bits and bytes
CoreComputationalUTF-8 is self-synchronising: continuation bytes always look like 10xxxxxx, and every other byte starts a new code point. Use that to write count_code_points(data), which takes a bytes object containing valid UTF-8 and returns how many code points it encodes, without decoding it (no .decode, no str). One pass over the bytes is enough.
8. Representing data: bits and bytes
Warm-upHumanA little-endian machine stores the 32-bit integer 0x12345678 at addresses 1000 to 1003. Which byte is at address 1000?
Further reading and resources
- What every computer scientist should know about floating-point arithmetic · David Goldberg · 1991The classic, thorough tour of IEEE 754 and its consequences.
- Floating-point arithmetic: issues and limitations · Python Software Foundation
- RFC 3629: UTF-8, a transformation format of ISO 10646 · 2003The standard definition of UTF-8.
- The Unicode Standard · The Unicode Consortium
- Unicode FAQ: UTF-8, UTF-16, UTF-32 and BOM
- Python Unicode HOWTO
- struct: interpret bytes as packed binary data
- On holy wars and a plea for peace (IEN 137) · Danny Cohen · 1980The note that named big-endian and little-endian.