Skip to content
LumenaryAcademy
Practice 60 min

Lists, tuples, maps and binaries

The lists module and comprehensions old and new, records and maps for structured data, the bit syntax for binary protocols, iolists, and JSON.

Code tested · 5AI-draftedAwaiting review by a computer scientist
MotivationH

The shapes of data

Erlang gives you few data structures, but each one is good at something specific, and choosing well is half of writing clear Erlang. Lists are for sequences you walk from the front. Tuples group a fixed number of values. Maps associate keys with values. And binaries hold raw bytes compactly, which matters enormously in a language designed for network protocols: Erlang's bit syntax lets you pattern match on a packet's header as easily as on a tuple.

All four are immutable, which shapes how you use them. "Changing" a map produces a new map that shares most of its structure with the old one; adding to the front of a list creates one new cell and reuses the rest. Large binaries (over 64 bytes) are stored outside process heaps and shared by reference, so passing a megabyte of data to another process does not copy it. Once you know what each operation costs, you can write code that is both clear and fast without fighting the language.

This lesson tours all four, together with the standard-library modules that work on them, the comprehension syntax that has grown several new forms in the last few releases, the json module that arrived in OTP 27, and ETS, the built-in in-memory tables you reach for when data must be shared between processes.

Worked examplesH

Lists and comprehensions

A list is a linked list: adding to the front ([X | List]) is instant, while length/1, lists:nth/2 and ++ walk or copy the list. The lists module has well over a hundred functions; here are many of the ones you will use most, together with list comprehensions:

erlang
#!/usr/bin/env escript
main(_) ->
    Nums = lists:seq(1, 20),
    io:format("~p~n", [[N * N || N <- Nums, N rem 3 =:= 0]]),
    io:format("~p~n", [[{X, Y} || X <- [a, b], Y <- [1, 2]]]),
    Results = [{ok, 3}, {error, late}, {ok, 9}],
    io:format("~p~n", [[V || {ok, V} <- Results]]),
    Names = ["Wanjiru", "Mateo", "Yuki"],
    Shifts = [am, pm, am],
    io:format("~p~n", [[{Name, Shift} || Name <- Names && Shift <- Shifts]]),
    io:format("~p~n", [[-I, I || I <- [1, 2, 3]]]),
    io:format("~p~n", [lists:zip(Names, Shifts)]),
    io:format("~p~n", [lists:sum(lists:seq(1, 10))]),
    io:format("~p~n", [lists:max([3, 9, 2])]),
    io:format("~p~n", [lists:nth(2, Names)]),
    io:format("~p~n", [lists:keyfind("Mateo", 1, lists:zip(Names, Shifts))]),
    io:format("~p~n", [lists:partition(fun(N) -> N > 5 end, lists:seq(1, 10))]),
    io:format("~p~n", [lists:usort([c, a, b, a])]),
    io:format("~p~n", [lists:flatten([1, [2, [3, 4]], 5])]),
    io:format("~p~n", [lists:join(", ", Names)]),
    io:format("~p~n", [string:join(Names, ", ")]),
    Triples = [{A, B, C} || A <- lists:seq(1, 20), B <- lists:seq(A, 20),
                            C <- lists:seq(B, 20), A * A + B * B =:= C * C],
    io:format("~p~n", [Triples]).

It prints:

text
[9,36,81,144,225,324]
[{a,1},{a,2},{b,1},{b,2}]
[3,9]
[{"Wanjiru",am},{"Mateo",pm},{"Yuki",am}]
[-1,1,-2,2,-3,3]
[{"Wanjiru",am},{"Mateo",pm},{"Yuki",am}]
55
9
"Mateo"
{"Mateo",pm}
{[6,7,8,9,10],[1,2,3,4,5]}
[a,b,c]
[1,2,3,4,5]
["Wanjiru",", ","Mateo",", ","Yuki"]
"Wanjiru, Mateo, Yuki"
[{3,4,5},{5,12,13},{6,8,10},{8,15,17},{9,12,15},{12,16,20}]

A comprehension [Expression || Generator, Filter, ...] reads "the list of Expression for each element from the generators that passes the filters":

  • Several generators nest like loops: for each X, every Y. A later generator may use variables from an earlier one, as B <- lists:seq(A, 20) does in the Pythagorean triples.
  • A pattern in a generator filters: {ok, V} <- Results silently skips anything that does not match. OTP 28 added the strict generator <:-, which instead raises an error on a non-matching element, for when a mismatch would be a bug.
  • Zip generators, Name <- Names && Shift <- Shifts, new in OTP 28, walk several lists in step rather than nested, like lists:zip/2.
  • Multi-valued comprehensions, new in OTP 29, emit several elements per iteration: [-I, I || I <- [1, 2, 3]].

A warning that applies to all of these: had we used lists:seq(1, 10) in the first line, the result [9,36,81] would have printed as "\t$Q", because every number in it is a printable character code and ~p guesses that such a list is a string. The values are right; only the display is misleading. When that bothers you, print with ~w.

Notice also lists:join/2, which returns a nested list rather than a flat string. That is not a mistake but an iolist, explained below with binaries.

Worked examplesH

Maps and records

A map associates keys with values, and any term can be a key. Build or add keys with =>; update existing keys with :=, which fails if the key is missing; match with :=, needing only the keys you mention.

erlang
#!/usr/bin/env escript
-record(book, {title, author, year, copies = 1}).

main(_) ->
    Stock = #{"tea" => 250, "bread" => 120},
    Stock1 = Stock#{"jam" => 480},
    Stock2 = Stock1#{"tea" := 260},
    io:format("~p~n", [Stock2]),
    io:format("~p ~p~n", [maps:get("jam", Stock2), maps:get("milk", Stock2, none)]),
    io:format("~p ~p~n", [maps:find("bread", Stock2), maps:find("milk", Stock2)]),
    io:format("~p~n", [maps:update_with("bread", fun(P) -> P + 10 end, 0, Stock2)]),
    io:format("~p~n", [maps:filter(fun(_Item, Price) -> Price > 200 end, Stock2)]),
    io:format("~p~n", [#{Item => Price * 90 div 100 || Item := Price <- Stock2}]),
    Total = maps:fold(fun(_Item, Price, Sum) -> Price + Sum end, 0, Stock2),
    io:format("~p~n", [Total]),
    Words = string:lexemes("the cat and the dog and the bird", " "),
    Counts = lists:foldl(fun(W, Acc) -> maps:update_with(W, fun(C) -> C + 1 end, 1, Acc) end,
                         #{}, Words),
    io:format("~p~n", [lists:sort(maps:to_list(Counts))]),
    Book = #book{title = "Things Fall Apart", author = "Chinua Achebe", year = 1958},
    io:format("~p~n", [Book]),
    io:format("~ts (~p), ~p copy~n", [Book#book.title, Book#book.year, Book#book.copies]),
    Updated = Book#book{copies = 3},
    #book{copies = C} = Updated,
    io:format("~p~n", [C]),
    try Stock#{"milk" := 99}
    catch error:{badkey, Key} -> io:format("no such key: ~p~n", [Key])
    end.

It prints:

text
#{"bread" => 120,"jam" => 480,"tea" => 260}
480 none
{ok,120} error
#{"bread" => 130,"jam" => 480,"tea" => 260}
#{"jam" => 480,"tea" => 260}
#{"bread" => 108,"jam" => 432,"tea" => 234}
860
[{"and",2},{"bird",1},{"cat",1},{"dog",1},{"the",3}]
{book,"Things Fall Apart","Chinua Achebe",1958,1}
Things Fall Apart (1958), 1 copy
3
no such key: "milk"

The maps module covers the common needs. maps:get/3 returns a default for a missing key, while maps:get/2 raises; maps:find/2 returns {ok, Value} or error, so a missing key and a stored undefined stay distinguishable. maps:update_with/4 is the idiomatic way to count or accumulate into a map, as the word count shows. Map comprehensions (OTP 26), #{K => V2 || K := V <- Map}, transform a map into a new one. And updating with := a key that does not exist raises {badkey, Key}, which catches misspelt keys early.

Records predate maps (which arrived in OTP 17, in 2014) and are still everywhere in OTP and older code. -record(book, {title, author, year, copies = 1}) declares a record with a default; #book{title = ...} builds one; Book#book.year reads a field; Book#book{copies = 3} makes an updated copy; and #book{copies = C} = Updated matches. As the output shows, a record is only a tuple whose first element is the record name: the field names exist at compile time, so a record defined in one module must be shared through an include file (.hrl) for others to use it. Use maps for data that crosses module or process boundaries, and records where you need the speed and compile-time field checking inside one module.

ComputationC

Binaries and the bit syntax

Erlang was built to talk to telephone hardware, and it shows in one of its most admired features: the bit syntax, which lets you build and pattern match binary data field by field, with sizes in bits. A segment is written Value:Size/Type, where the type can include integer (the default), float, binary, utf8, signed or unsigned, and big (the default), little or native endianness.

Here is a small, made-up sensor protocol: a 4-bit version, a 4-bit message kind, a 16-bit sensor id, a signed 16-bit temperature in tenths of a degree, an 8-bit humidity, and a length-prefixed UTF-8 label.

erlang
#!/usr/bin/env escript
%% A made-up sensor packet:
%%   version (4 bits), kind (4 bits), sensor id (16 bits, big-endian),
%%   temperature in tenths of a degree (16 bits, signed), humidity percent (8 bits),
%%   then a variable-length label: length (8 bits) and that many bytes of UTF-8.
main(_) ->
    Packet = encode(#{sensor => 513, temp => -45, humidity => 87, label => "Nyeri farm"}),
    io:format("~p~n", [Packet]),
    io:format("~p bytes~n", [byte_size(Packet)]),
    io:format("~p~n", [decode(Packet)]),
    io:format("~p~n", [decode(<<2:4, 0:4, 7>>)]),
    io:format("~p~n", [[Byte || <<Byte>> <= <<"ABC">>]]),
    io:format("~p~n", [<< <<(B + 1)>> || <<B>> <= <<"HAL">> >>]),
    io:format("~s~n", [binary:encode_hex(<<255, 16, 1>>)]),
    IoList = ["Sensor ", integer_to_list(513), $:, <<" ok">>, [$\n]],
    io:format("~p bytes in ~p~n", [iolist_size(IoList), IoList]),
    io:format("~s", [IoList]).

encode(#{sensor := Id, temp := Temp, humidity := Hum, label := Label}) ->
    LabelBin = unicode:characters_to_binary(Label),
    <<1:4, 2:4, Id:16, Temp:16/signed, Hum:8, (byte_size(LabelBin)):8, LabelBin/binary>>.

decode(<<1:4, 2:4, Id:16, Temp:16/signed, Hum:8, Len:8, Label:Len/binary>>) ->
    {ok, #{sensor => Id, temp => Temp / 10, humidity => Hum, label => Label}};
decode(<<Version:4, _/bitstring>>) ->
    {error, {unsupported_version, Version}}.

It prints:

text
<<18,2,1,255,211,87,10,78,121,101,114,105,32,102,97,114,109>>
17 bytes
{ok,#{label => <<"Nyeri farm">>,sensor => 513,temp => -4.5,humidity => 87}}
{error,{unsupported_version,2}}
"ABC"
<<"IBM">>
FF1001
15 bytes in ["Sensor ","513",58,<<" ok">>,"\n"]
Sensor 513: ok

encode/1 and decode/1 mirror each other exactly, and the decoder is a single pattern. Notice how much it checks for free: the version and kind must be 1 and 2; Temp:16/signed turns the bytes 255, 211 into -45; and Len:8, Label:Len/binary uses a size read earlier in the same pattern to cut out the label, so a truncated packet simply fails to match and falls through to the error clause. The first byte, 18, is 0001 0010 in binary: version 1 in the high four bits, kind 2 in the low four. Code like this, in production, parses network protocols, file formats and hardware messages, and it is typically both shorter and faster than hand-written byte twiddling.

Binary comprehensions walk a binary segment by segment, <<Byte>> <= Binary, and can build a list or, with << ... >> around the whole comprehension, a new binary. (Shifting every letter of "HAL" by one is an old joke.) binary:encode_hex/1 is one of many useful functions in the binary module.

Finally, iolists. When producing output, such as a web page, a log line or a network reply, you rarely need one flat binary. An iolist is any nesting of binaries, byte values and lists of them, and every output function in OTP accepts it directly. Building ["Sensor ", integer_to_list(513), $:, <<" ok">>, [$\n]] costs a few list cells; concatenating strings would copy them each time. iolist_size/1 and iolist_to_binary/1 measure or flatten one when you must. Writing output as iolists is one of the habits that makes Erlang servers fast.

Worked examplesH

JSON in the standard library

Since OTP 27 the standard library includes a json module, so encoding and decoding JSON needs no dependency. Maps become JSON objects, lists become arrays, binaries become strings, and the atoms true, false and null stand for the JSON literals.

erlang
#!/usr/bin/env escript
main(_) ->
    Order = #{id => 42, paid => true,
              items => [#{sku => <<"RICE-5KG">>, qty => 2}, #{sku => <<"OIL-1L">>, qty => 1}]},
    Json = iolist_to_binary(json:encode(Order)),
    io:format("~s~n", [Json]),
    Text = <<"{\"name\":\"Zoë\",\"tags\":[\"new\",\"vip\"],\"score\":9.5,\"note\":null}"/utf8>>,
    Decoded = json:decode(Text),
    io:format("~tp~n", [Decoded]),
    #{<<"name">> := Name, <<"tags">> := [FirstTag | _]} = Decoded,
    io:format("~ts has tag ~ts~n", [Name, FirstTag]),
    io:format("~p~n", [maps:get(<<"note">>, Decoded)]),
    try json:decode(<<"{\"broken\": ">>)
    catch error:Reason -> io:format("invalid JSON: ~p~n", [Reason])
    end.

It prints:

text
{"id":42,"items":[{"sku":"RICE-5KG","qty":2},{"sku":"OIL-1L","qty":1}],"paid":true}
#{<<"name">> => <<"Zoë"/utf8>>,<<"note">> => null,<<"score">> => 9.5,
  <<"tags">> => [<<"new">>,<<"vip">>]}
Zoë has tag new
null
invalid JSON: unexpected_end

json:encode/1 returns an iolist, so we flatten it with iolist_to_binary/1 before printing; when writing to a socket you would send the iolist as is. Atom keys are encoded as strings. Decoding produces binary keys, never atoms (remember that atoms are never garbage-collected, so turning untrusted input into atoms would be a memory leak waiting to be exploited). JSON null becomes the atom null, not undefined; both json:decode/3 and json:encode/2 accept options and callbacks to customise such choices. Invalid input raises an error with a reason you can match, here unexpected_end.

IntuitionH

What each operation costs

A rough cost model is worth memorising, because it decides which structure fits a job. Here n is the number of elements:

OperationListTupleMapBinary
read the first elementconstantconstantconstant
read element i or key kwalks i cellsconstant (element/2)about log nconstant (binary:at/2)
add one elementconstant at the front; copies the list at the endcopies the tupleabout log nappending is often optimised in place
sizewalks the list (length/1)constantconstantconstant (byte_size/1)

Three consequences follow. Use lists for sequences you process from front to back, and build them by prepending, reversing once at the end if order matters. Keep tuples small and fixed, since every change copies them. Use maps for anything you look up by key. And appending to a binary in a loop (<<Acc/binary, Byte>>) is cheaper than it looks: the runtime leaves room at the end of a binary that is only appended to, so a loop that builds a binary this way does not copy it each time. The Efficiency Guide explains the details.

Worked examplesH

ETS: tables shared between processes

Everything so far lives inside one process. When many processes need fast access to the same large data (a cache, a session table, a routing table), Erlang offers ETS (Erlang Term Storage): in-memory tables of tuples, created by a process and readable by any process the table allows, with constant-time lookups for set tables.

erlang
#!/usr/bin/env escript
main(_) ->
    Prices = ets:new(prices, [set, public]),
    true = ets:insert(Prices, [{"tea", 250}, {"bread", 120}, {"jam", 480}]),
    io:format("~p~n", [ets:lookup(Prices, "tea")]),
    io:format("~p~n", [ets:lookup(Prices, "milk")]),
    true = ets:insert(Prices, {"tea", 260}),
    io:format("~p~n", [ets:lookup_element(Prices, "tea", 2)]),
    Dear = ets:select(Prices, [{{'$1', '$2'}, [{'>', '$2', 200}], ['$1']}]),
    io:format("~p~n", [lists:sort(Dear)]),
    Hits = ets:new(hits, [set]),
    [ets:update_counter(Hits, Page, 1, {Page, 0}) || Page <- [home, about, home, home]],
    io:format("~p~n", [lists:sort(ets:tab2list(Hits))]),
    io:format("~p rows, owned by ~p~n", [ets:info(Prices, size), ets:info(Prices, owner) =:= self()]).

It prints:

text
[{"tea",250}]
[]
260
["jam","tea"]
[{about,1},{home,3}]
3 rows, owned by true

A table stores tuples, and by default the first element is the key. ets:lookup/2 returns a list of matching tuples (empty if none), so a missing key is not an error. Inserting a tuple with an existing key replaces it in a set table. ets:select/2 takes a match specification, a small query language written as terms: here "for rows {Key, Price} where Price > 200, return Key". ets:update_counter/4 increments a counter atomically, creating the row with a default if needed, which makes ETS a common choice for metrics.

ETS tables are not garbage-collected values: a table belongs to the process that created it and disappears when that process dies. Its contents are copied in and out of the table on every access, so it suits many small reads and writes better than huge values. A common design, which the OTP lessons will make concrete, is a long-lived process that owns a table, handles writes, and lets other processes read it directly.

Common misconceptionsH

Common mistakes

Misconception

"Strings are cheap because they are just text." A list string costs two machine words per character, 16 bytes on a 64-bit system, so a 1 MB file read as a list takes about 16 MB. Read files and network data as binaries, and convert to lists only for small strings you need to manipulate character by character.

Misconception

"#{K := V} and #{K => V} are interchangeable." In an expression, => inserts or replaces, while := only replaces an existing key and fails otherwise. In a pattern, only := is allowed.

Misconception

"Records are a runtime data type with named fields." Records are tuples; the names exist only in the compiler. That is why the shell prints {book,"Things Fall Apart",...}, and why changing a record definition requires recompiling every module that uses it. OTP 29's experimental native records are a different, genuinely run-time data type.

Misconception

"I must flatten output into one binary before sending it." Every OTP output function takes iolists. Flattening first only adds a copy.

Exercises

Exercises

1. Lists, tuples, maps and binaries

Warm-upComputational

What does io:format("~w~n", [[X * 2 || X <- [1, 2, 3, 4], X rem 2 =:= 0]]) print?

2. Lists, tuples, maps and binaries

CoreComputational

What does io:format("~p~n", [[{A, B} || A <- [1, 2] && B <- [x, y]]]) print on Erlang/OTP 28 or later?

3. Lists, tuples, maps and binaries

CoreComputational

What is byte_size(<<1:4, 2:4, 513:16>>)?

4. Lists, tuples, maps and binaries

CoreComputational

After <<Temp:16/signed>> = <<255, 254>>, what is Temp?

5. Lists, tuples, maps and binaries

CoreComputational

With M = #{a => 1}, what happens when you evaluate M#{k := 1}?

Choose one

6. Lists, tuples, maps and binaries

Warm-upComputational

What is maps:get(b, #{a => 1}, 0)?

7. Lists, tuples, maps and binaries

StretchComputational

Which of these expressions evaluate to true? List their letters.

  • A: is_list("abc")
  • B: is_binary(<<"abc">>)
  • C: "abc" =:= <<"abc">>
  • D: length([1, [2, 3]]) =:= 3
  • E: iolist_size(["ab", <<"c">>, $d]) =:= 4

Write elements separated by commas. Order and repeats don’t matter — just like in a set!

Further reading and resources

  • List comprehensionsThe official guide to comprehensions, generators and filters.
  • The maps moduleEvery function for working with maps.
  • Bit syntaxSegments, sizes, types, signedness and endianness, with examples.
  • RecordsThe reference manual's chapter on records.
  • The json moduleThe JSON encoder and decoder added to the standard library in OTP 27.