The shapes of data
Erlang gives you few data structures, but each one is good at something specific, and choosing well is half of writing clear Erlang. Lists are for sequences you walk from the front. Tuples group a fixed number of values. Maps associate keys with values. And binaries hold raw bytes compactly, which matters enormously in a language designed for network protocols: Erlang's bit syntax lets you pattern match on a packet's header as easily as on a tuple.
All four are immutable, which shapes how you use them. "Changing" a map produces a new map that shares most of its structure with the old one; adding to the front of a list creates one new cell and reuses the rest. Large binaries (over 64 bytes) are stored outside process heaps and shared by reference, so passing a megabyte of data to another process does not copy it. Once you know what each operation costs, you can write code that is both clear and fast without fighting the language.
This lesson tours all four, together with the standard-library modules that work on them, the comprehension syntax that has grown several new forms in the last few releases, the json module that arrived in OTP 27, and ETS, the built-in in-memory tables you reach for when data must be shared between processes.
Lists and comprehensions
A list is a linked list: adding to the front ([X | List]) is instant, while length/1, lists:nth/2 and ++ walk or copy the list. The lists module has well over a hundred functions; here are many of the ones you will use most, together with list comprehensions:
#!/usr/bin/env escript
main(_) ->
Nums = lists:seq(1, 20),
io:format("~p~n", [[N * N || N <- Nums, N rem 3 =:= 0]]),
io:format("~p~n", [[{X, Y} || X <- [a, b], Y <- [1, 2]]]),
Results = [{ok, 3}, {error, late}, {ok, 9}],
io:format("~p~n", [[V || {ok, V} <- Results]]),
Names = ["Wanjiru", "Mateo", "Yuki"],
Shifts = [am, pm, am],
io:format("~p~n", [[{Name, Shift} || Name <- Names && Shift <- Shifts]]),
io:format("~p~n", [[-I, I || I <- [1, 2, 3]]]),
io:format("~p~n", [lists:zip(Names, Shifts)]),
io:format("~p~n", [lists:sum(lists:seq(1, 10))]),
io:format("~p~n", [lists:max([3, 9, 2])]),
io:format("~p~n", [lists:nth(2, Names)]),
io:format("~p~n", [lists:keyfind("Mateo", 1, lists:zip(Names, Shifts))]),
io:format("~p~n", [lists:partition(fun(N) -> N > 5 end, lists:seq(1, 10))]),
io:format("~p~n", [lists:usort([c, a, b, a])]),
io:format("~p~n", [lists:flatten([1, [2, [3, 4]], 5])]),
io:format("~p~n", [lists:join(", ", Names)]),
io:format("~p~n", [string:join(Names, ", ")]),
Triples = [{A, B, C} || A <- lists:seq(1, 20), B <- lists:seq(A, 20),
C <- lists:seq(B, 20), A * A + B * B =:= C * C],
io:format("~p~n", [Triples]).It prints:
[9,36,81,144,225,324]
[{a,1},{a,2},{b,1},{b,2}]
[3,9]
[{"Wanjiru",am},{"Mateo",pm},{"Yuki",am}]
[-1,1,-2,2,-3,3]
[{"Wanjiru",am},{"Mateo",pm},{"Yuki",am}]
55
9
"Mateo"
{"Mateo",pm}
{[6,7,8,9,10],[1,2,3,4,5]}
[a,b,c]
[1,2,3,4,5]
["Wanjiru",", ","Mateo",", ","Yuki"]
"Wanjiru, Mateo, Yuki"
[{3,4,5},{5,12,13},{6,8,10},{8,15,17},{9,12,15},{12,16,20}]A comprehension [Expression || Generator, Filter, ...] reads "the list of Expression for each element from the generators that passes the filters":
- Several generators nest like loops: for each
X, everyY. A later generator may use variables from an earlier one, asB <- lists:seq(A, 20)does in the Pythagorean triples. - A pattern in a generator filters:
{ok, V} <- Resultssilently skips anything that does not match. OTP 28 added the strict generator<:-, which instead raises an error on a non-matching element, for when a mismatch would be a bug. - Zip generators,
Name <- Names && Shift <- Shifts, new in OTP 28, walk several lists in step rather than nested, likelists:zip/2. - Multi-valued comprehensions, new in OTP 29, emit several elements per iteration:
[-I, I || I <- [1, 2, 3]].
A warning that applies to all of these: had we used lists:seq(1, 10) in the first line, the result [9,36,81] would have printed as "\t$Q", because every number in it is a printable character code and ~p guesses that such a list is a string. The values are right; only the display is misleading. When that bothers you, print with ~w.
Notice also lists:join/2, which returns a nested list rather than a flat string. That is not a mistake but an iolist, explained below with binaries.
Maps and records
A map associates keys with values, and any term can be a key. Build or add keys with =>; update existing keys with :=, which fails if the key is missing; match with :=, needing only the keys you mention.
#!/usr/bin/env escript
-record(book, {title, author, year, copies = 1}).
main(_) ->
Stock = #{"tea" => 250, "bread" => 120},
Stock1 = Stock#{"jam" => 480},
Stock2 = Stock1#{"tea" := 260},
io:format("~p~n", [Stock2]),
io:format("~p ~p~n", [maps:get("jam", Stock2), maps:get("milk", Stock2, none)]),
io:format("~p ~p~n", [maps:find("bread", Stock2), maps:find("milk", Stock2)]),
io:format("~p~n", [maps:update_with("bread", fun(P) -> P + 10 end, 0, Stock2)]),
io:format("~p~n", [maps:filter(fun(_Item, Price) -> Price > 200 end, Stock2)]),
io:format("~p~n", [#{Item => Price * 90 div 100 || Item := Price <- Stock2}]),
Total = maps:fold(fun(_Item, Price, Sum) -> Price + Sum end, 0, Stock2),
io:format("~p~n", [Total]),
Words = string:lexemes("the cat and the dog and the bird", " "),
Counts = lists:foldl(fun(W, Acc) -> maps:update_with(W, fun(C) -> C + 1 end, 1, Acc) end,
#{}, Words),
io:format("~p~n", [lists:sort(maps:to_list(Counts))]),
Book = #book{title = "Things Fall Apart", author = "Chinua Achebe", year = 1958},
io:format("~p~n", [Book]),
io:format("~ts (~p), ~p copy~n", [Book#book.title, Book#book.year, Book#book.copies]),
Updated = Book#book{copies = 3},
#book{copies = C} = Updated,
io:format("~p~n", [C]),
try Stock#{"milk" := 99}
catch error:{badkey, Key} -> io:format("no such key: ~p~n", [Key])
end.It prints:
#{"bread" => 120,"jam" => 480,"tea" => 260}
480 none
{ok,120} error
#{"bread" => 130,"jam" => 480,"tea" => 260}
#{"jam" => 480,"tea" => 260}
#{"bread" => 108,"jam" => 432,"tea" => 234}
860
[{"and",2},{"bird",1},{"cat",1},{"dog",1},{"the",3}]
{book,"Things Fall Apart","Chinua Achebe",1958,1}
Things Fall Apart (1958), 1 copy
3
no such key: "milk"The maps module covers the common needs. maps:get/3 returns a default for a missing key, while maps:get/2 raises; maps:find/2 returns {ok, Value} or error, so a missing key and a stored undefined stay distinguishable. maps:update_with/4 is the idiomatic way to count or accumulate into a map, as the word count shows. Map comprehensions (OTP 26), #{K => V2 || K := V <- Map}, transform a map into a new one. And updating with := a key that does not exist raises {badkey, Key}, which catches misspelt keys early.
Records predate maps (which arrived in OTP 17, in 2014) and are still everywhere in OTP and older code. -record(book, {title, author, year, copies = 1}) declares a record with a default; #book{title = ...} builds one; Book#book.year reads a field; Book#book{copies = 3} makes an updated copy; and #book{copies = C} = Updated matches. As the output shows, a record is only a tuple whose first element is the record name: the field names exist at compile time, so a record defined in one module must be shared through an include file (.hrl) for others to use it. Use maps for data that crosses module or process boundaries, and records where you need the speed and compile-time field checking inside one module.
Binaries and the bit syntax
Erlang was built to talk to telephone hardware, and it shows in one of its most admired features: the bit syntax, which lets you build and pattern match binary data field by field, with sizes in bits. A segment is written Value:Size/Type, where the type can include integer (the default), float, binary, utf8, signed or unsigned, and big (the default), little or native endianness.
Here is a small, made-up sensor protocol: a 4-bit version, a 4-bit message kind, a 16-bit sensor id, a signed 16-bit temperature in tenths of a degree, an 8-bit humidity, and a length-prefixed UTF-8 label.
#!/usr/bin/env escript
%% A made-up sensor packet:
%% version (4 bits), kind (4 bits), sensor id (16 bits, big-endian),
%% temperature in tenths of a degree (16 bits, signed), humidity percent (8 bits),
%% then a variable-length label: length (8 bits) and that many bytes of UTF-8.
main(_) ->
Packet = encode(#{sensor => 513, temp => -45, humidity => 87, label => "Nyeri farm"}),
io:format("~p~n", [Packet]),
io:format("~p bytes~n", [byte_size(Packet)]),
io:format("~p~n", [decode(Packet)]),
io:format("~p~n", [decode(<<2:4, 0:4, 7>>)]),
io:format("~p~n", [[Byte || <<Byte>> <= <<"ABC">>]]),
io:format("~p~n", [<< <<(B + 1)>> || <<B>> <= <<"HAL">> >>]),
io:format("~s~n", [binary:encode_hex(<<255, 16, 1>>)]),
IoList = ["Sensor ", integer_to_list(513), $:, <<" ok">>, [$\n]],
io:format("~p bytes in ~p~n", [iolist_size(IoList), IoList]),
io:format("~s", [IoList]).
encode(#{sensor := Id, temp := Temp, humidity := Hum, label := Label}) ->
LabelBin = unicode:characters_to_binary(Label),
<<1:4, 2:4, Id:16, Temp:16/signed, Hum:8, (byte_size(LabelBin)):8, LabelBin/binary>>.
decode(<<1:4, 2:4, Id:16, Temp:16/signed, Hum:8, Len:8, Label:Len/binary>>) ->
{ok, #{sensor => Id, temp => Temp / 10, humidity => Hum, label => Label}};
decode(<<Version:4, _/bitstring>>) ->
{error, {unsupported_version, Version}}.It prints:
<<18,2,1,255,211,87,10,78,121,101,114,105,32,102,97,114,109>>
17 bytes
{ok,#{label => <<"Nyeri farm">>,sensor => 513,temp => -4.5,humidity => 87}}
{error,{unsupported_version,2}}
"ABC"
<<"IBM">>
FF1001
15 bytes in ["Sensor ","513",58,<<" ok">>,"\n"]
Sensor 513: okencode/1 and decode/1 mirror each other exactly, and the decoder is a single pattern. Notice how much it checks for free: the version and kind must be 1 and 2; Temp:16/signed turns the bytes 255, 211 into -45; and Len:8, Label:Len/binary uses a size read earlier in the same pattern to cut out the label, so a truncated packet simply fails to match and falls through to the error clause. The first byte, 18, is 0001 0010 in binary: version 1 in the high four bits, kind 2 in the low four. Code like this, in production, parses network protocols, file formats and hardware messages, and it is typically both shorter and faster than hand-written byte twiddling.
Binary comprehensions walk a binary segment by segment, <<Byte>> <= Binary, and can build a list or, with << ... >> around the whole comprehension, a new binary. (Shifting every letter of "HAL" by one is an old joke.) binary:encode_hex/1 is one of many useful functions in the binary module.
Finally, iolists. When producing output, such as a web page, a log line or a network reply, you rarely need one flat binary. An iolist is any nesting of binaries, byte values and lists of them, and every output function in OTP accepts it directly. Building ["Sensor ", integer_to_list(513), $:, <<" ok">>, [$\n]] costs a few list cells; concatenating strings would copy them each time. iolist_size/1 and iolist_to_binary/1 measure or flatten one when you must. Writing output as iolists is one of the habits that makes Erlang servers fast.
JSON in the standard library
Since OTP 27 the standard library includes a json module, so encoding and decoding JSON needs no dependency. Maps become JSON objects, lists become arrays, binaries become strings, and the atoms true, false and null stand for the JSON literals.
#!/usr/bin/env escript
main(_) ->
Order = #{id => 42, paid => true,
items => [#{sku => <<"RICE-5KG">>, qty => 2}, #{sku => <<"OIL-1L">>, qty => 1}]},
Json = iolist_to_binary(json:encode(Order)),
io:format("~s~n", [Json]),
Text = <<"{\"name\":\"Zoë\",\"tags\":[\"new\",\"vip\"],\"score\":9.5,\"note\":null}"/utf8>>,
Decoded = json:decode(Text),
io:format("~tp~n", [Decoded]),
#{<<"name">> := Name, <<"tags">> := [FirstTag | _]} = Decoded,
io:format("~ts has tag ~ts~n", [Name, FirstTag]),
io:format("~p~n", [maps:get(<<"note">>, Decoded)]),
try json:decode(<<"{\"broken\": ">>)
catch error:Reason -> io:format("invalid JSON: ~p~n", [Reason])
end.It prints:
{"id":42,"items":[{"sku":"RICE-5KG","qty":2},{"sku":"OIL-1L","qty":1}],"paid":true}
#{<<"name">> => <<"Zoë"/utf8>>,<<"note">> => null,<<"score">> => 9.5,
<<"tags">> => [<<"new">>,<<"vip">>]}
Zoë has tag new
null
invalid JSON: unexpected_endjson:encode/1 returns an iolist, so we flatten it with iolist_to_binary/1 before printing; when writing to a socket you would send the iolist as is. Atom keys are encoded as strings. Decoding produces binary keys, never atoms (remember that atoms are never garbage-collected, so turning untrusted input into atoms would be a memory leak waiting to be exploited). JSON null becomes the atom null, not undefined; both json:decode/3 and json:encode/2 accept options and callbacks to customise such choices. Invalid input raises an error with a reason you can match, here unexpected_end.
What each operation costs
A rough cost model is worth memorising, because it decides which structure fits a job. Here n is the number of elements:
| Operation | List | Tuple | Map | Binary |
|---|---|---|---|---|
| read the first element | constant | constant | constant | |
| read element i or key k | walks i cells | constant (element/2) | about log n | constant (binary:at/2) |
| add one element | constant at the front; copies the list at the end | copies the tuple | about log n | appending is often optimised in place |
| size | walks the list (length/1) | constant | constant | constant (byte_size/1) |
Three consequences follow. Use lists for sequences you process from front to back, and build them by prepending, reversing once at the end if order matters. Keep tuples small and fixed, since every change copies them. Use maps for anything you look up by key. And appending to a binary in a loop (<<Acc/binary, Byte>>) is cheaper than it looks: the runtime leaves room at the end of a binary that is only appended to, so a loop that builds a binary this way does not copy it each time. The Efficiency Guide explains the details.
ETS: tables shared between processes
Everything so far lives inside one process. When many processes need fast access to the same large data (a cache, a session table, a routing table), Erlang offers ETS (Erlang Term Storage): in-memory tables of tuples, created by a process and readable by any process the table allows, with constant-time lookups for set tables.
#!/usr/bin/env escript
main(_) ->
Prices = ets:new(prices, [set, public]),
true = ets:insert(Prices, [{"tea", 250}, {"bread", 120}, {"jam", 480}]),
io:format("~p~n", [ets:lookup(Prices, "tea")]),
io:format("~p~n", [ets:lookup(Prices, "milk")]),
true = ets:insert(Prices, {"tea", 260}),
io:format("~p~n", [ets:lookup_element(Prices, "tea", 2)]),
Dear = ets:select(Prices, [{{'$1', '$2'}, [{'>', '$2', 200}], ['$1']}]),
io:format("~p~n", [lists:sort(Dear)]),
Hits = ets:new(hits, [set]),
[ets:update_counter(Hits, Page, 1, {Page, 0}) || Page <- [home, about, home, home]],
io:format("~p~n", [lists:sort(ets:tab2list(Hits))]),
io:format("~p rows, owned by ~p~n", [ets:info(Prices, size), ets:info(Prices, owner) =:= self()]).It prints:
[{"tea",250}]
[]
260
["jam","tea"]
[{about,1},{home,3}]
3 rows, owned by trueA table stores tuples, and by default the first element is the key. ets:lookup/2 returns a list of matching tuples (empty if none), so a missing key is not an error. Inserting a tuple with an existing key replaces it in a set table. ets:select/2 takes a match specification, a small query language written as terms: here "for rows {Key, Price} where Price > 200, return Key". ets:update_counter/4 increments a counter atomically, creating the row with a default if needed, which makes ETS a common choice for metrics.
ETS tables are not garbage-collected values: a table belongs to the process that created it and disappears when that process dies. Its contents are copied in and out of the table on every access, so it suits many small reads and writes better than huge values. A common design, which the OTP lessons will make concrete, is a long-lived process that owns a table, handles writes, and lets other processes read it directly.
Common mistakes
Misconception
"Strings are cheap because they are just text." A list string costs two machine words per character, 16 bytes on a 64-bit system, so a 1 MB file read as a list takes about 16 MB. Read files and network data as binaries, and convert to lists only for small strings you need to manipulate character by character.
Misconception
"#{K := V} and #{K => V} are interchangeable." In an expression, => inserts or replaces, while := only replaces an existing key and fails otherwise. In a pattern, only := is allowed.
Misconception
"Records are a runtime data type with named fields." Records are tuples; the names exist only in the compiler. That is why the shell prints {book,"Things Fall Apart",...}, and why changing a record definition requires recompiling every module that uses it. OTP 29's experimental native records are a different, genuinely run-time data type.
Misconception
"I must flatten output into one binary before sending it." Every OTP output function takes iolists. Flattening first only adds a copy.
Exercises
1. Lists, tuples, maps and binaries
Warm-upComputationalWhat does io:format("~w~n", [[X * 2 || X <- [1, 2, 3, 4], X rem 2 =:= 0]]) print?
2. Lists, tuples, maps and binaries
CoreComputationalWhat does io:format("~p~n", [[{A, B} || A <- [1, 2] && B <- [x, y]]]) print on Erlang/OTP 28 or later?
3. Lists, tuples, maps and binaries
CoreComputationalWhat is byte_size(<<1:4, 2:4, 513:16>>)?
4. Lists, tuples, maps and binaries
CoreComputationalAfter <<Temp:16/signed>> = <<255, 254>>, what is Temp?
5. Lists, tuples, maps and binaries
CoreComputationalWith M = #{a => 1}, what happens when you evaluate M#{k := 1}?
6. Lists, tuples, maps and binaries
Warm-upComputationalWhat is maps:get(b, #{a => 1}, 0)?
7. Lists, tuples, maps and binaries
StretchComputationalWhich of these expressions evaluate to true? List their letters.
- A:
is_list("abc") - B:
is_binary(<<"abc">>) - C:
"abc" =:= <<"abc">> - D:
length([1, [2, 3]]) =:= 3 - E:
iolist_size(["ab", <<"c">>, $d]) =:= 4
Further reading and resources
- List comprehensionsThe official guide to comprehensions, generators and filters.
- The maps moduleEvery function for working with maps.
- Bit syntaxSegments, sizes, types, signedness and endianness, with examples.
- RecordsThe reference manual's chapter on records.
- The json moduleThe JSON encoder and decoder added to the standard library in OTP 27.