Skip to content
LumenaryAcademy
Foundations 60 min

Arrays, broadcasting and linear algebra

Julia's multidimensional arrays, the dot that applies any function element by element, the LinearAlgebra standard library, and the other collections you will reach for.

Code tested · 4AI-draftedAwaiting review by a computer scientist
MotivationH

Arrays are the language

Scientific computing is, to a first approximation, arithmetic on arrays: vectors of measurements, matrices of coefficients, grids of temperatures, images, tensors of weights. Languages handle this in very different ways. In Python, the core language's lists are slow for numbers, so serious work happens in NumPy, a separate library written mostly in C, with its own rules. In MATLAB, arrays are everything and loops used to be something to avoid.

In Julia, arrays are part of the language and written in Julia itself, and loops over them are fast. You can write a vectorised one-liner when it reads well and a plain loop when that is clearer, and both run at compiled speed. Julia inherits a great deal from Fortran and MATLAB here: indices start at 1, matrices are stored column by column, and A \ b solves a linear system. This lesson covers arrays, the dot syntax for broadcasting, the LinearAlgebra standard library, and briefly the other collections: tuples, dictionaries and sets.

DefinitionsH

Vectors, matrices and indexing

Definition— Array{T, N}

An Array{T, N} is an N-dimensional, mutable, dense array whose elements all have type T. Vector{T} is another name for Array{T, 1} and Matrix{T} for Array{T, 2}. All array types share the abstract supertype AbstractArray{T, N}, which is what generic code should accept.

julia
# Creating, indexing and slicing arrays
v = [4, 8, 15, 16, 23, 42]          # a Vector{Int64}: one dimension
M = [1 2 3; 4 5 6]                  # a Matrix{Int64}: spaces separate columns, ; separates rows

println(typeof(v), " with ", length(v), " elements")
println(typeof(M), " of size ", size(M))
println("first and last: ", v[1], " ", v[end])
println("v[2:4] = ", v[2:4], ", every other one: ", v[1:2:end])
println("M[2, 3] = ", M[2, 3], ", row 1: ", M[1, :], ", column 2: ", M[:, 2])
println("elements above 10: ", v[v .> 10])

# Memory is column-major: columns are stored one after another
println("vec(M) = ", vec(M))
println("M[4] = ", M[4], " (linear indexing follows memory order)")

# Vectors grow at the end
push!(v, 108)
append!(v, [1, 2])
println("after push! and append!: ", v)

# Assignment shares; copy copies
a = [1, 2, 3]
b = a
b[1] = 100
println("a after changing b: ", a)
c = copy(a)
c[1] = -1
println("a after changing a copy: ", a)

# A slice is a copy; a view refers to the original
s = a[2:3]
s[1] = 0
println("after changing a slice: ", a)
w = view(a, 2:3)                    # or @view a[2:3]
w[1] = 0
println("after changing a view:  ", a)

# Constructors for common arrays
display(zeros(2, 3))
display(fill("·", 2, 2))

It prints:

text
Vector{Int64} with 6 elements
Matrix{Int64} of size (2, 3)
first and last: 4 42
v[2:4] = [8, 15, 16], every other one: [4, 15, 23]
M[2, 3] = 6, row 1: [1, 2, 3], column 2: [2, 5]
elements above 10: [15, 16, 23, 42]
vec(M) = [1, 4, 2, 5, 3, 6]
M[4] = 5 (linear indexing follows memory order)
after push! and append!: [4, 8, 15, 16, 23, 42, 108, 1, 2]
a after changing b: [100, 2, 3]
a after changing a copy: [100, 2, 3]
after changing a slice: [100, 2, 3]
after changing a view:  [100, 0, 3]
2×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0
2×2 Matrix{String}:
 "·"  "·"
 "·"  "·"

The rules on display:

  • Literals. Commas make a vector, [4, 8, 15]. Spaces separate columns and semicolons (or new lines) separate rows, so [1 2 3; 4 5 6] is a 2 × 3 matrix and [1 2 3] is a 1 × 3 matrix, not a vector.
  • Indexing starts at 1, and end means the last index. Ranges select several elements (v[2:4]), a step skips (v[1:2:end]), and : alone means "all of this dimension", so M[1, :] is the first row and M[:, 2] the second column. A vector of Bools selects where it is true: v[v .> 10].
  • Column-major order. The elements of a matrix are stored in memory column by column, so vec(M) gives [1, 4, 2, 5, 3, 6], and a single index, M[4], counts in that order. This is Fortran's convention (C and NumPy's default are row-major), and lesson 6 shows why it matters for speed: loop over the first index in the innermost loop.
  • Growing. push! adds one element at the end and append! adds a collection. Only one-dimensional arrays can grow.

Sharing, copying and viewing

These lines in the middle of the program deserve a slow reading, because they cause more bugs than anything else in this lesson:

  • b = a does not copy. It gives the same array a second name, so b[1] = 100 changes a.
  • copy(a) makes a new array with the same elements.
  • a[2:3], a slice, copies the selected elements into a new array, so changing the slice leaves a alone.
  • view(a, 2:3) or @view a[2:3] creates a lightweight window onto the same memory, so changing the view changes a. Views avoid copying large arrays, and the macro @views in front of a block or function turns every slice inside it into a view.
Worked examplesH

Broadcasting with the dot

Arithmetic operators on arrays mean what they mean in linear algebra. * between two matrices is the matrix product, and * between two column vectors is an error, because the product of a 3 × 1 and a 3 × 1 matrix is undefined. To work element by element you put a dot in front of the operator, or after a function's name:

text
julia> x = [1, 2, 3]
3-element Vector{Int64}:
 1
 2
 3

julia> x * x
ERROR: MethodError: no method matching *(::Vector{Int64}, ::Vector{Int64})
The function `*` exists, but no method is defined for this combination of argument types.
(the rest of the message is omitted here)

julia> x .* x
3-element Vector{Int64}:
 1
 4
 9

julia> x' * x
14

julia> sin(x)
ERROR: MethodError: no method matching sin(::Vector{Int64})
The function `sin` exists, but no method is defined for this combination of argument types.
(the rest of the message is omitted here)

julia> sin.(x)
3-element Vector{Float64}:
 0.8414709848078965
 0.9092974268256817
 0.1411200080598672

x' is the adjoint (conjugate transpose) of x, a 1 × 3 row, so x' * x is the dot product, 14. sin(x) is an error because the sine of a vector is not defined, but sin.(x) applies sin to each element. Unlike NumPy, Julia does not need a separate "vectorised" version of every function: the dot works for every function, including the ones you write.

julia
# Broadcasting: apply any function element by element with a dot
celsius = [-5.0, 0.0, 12.5, 21.0, 37.0]
fahrenheit = celsius .* 9 ./ 5 .+ 32
println(fahrenheit)

# The same, written once with @. (which adds every dot for you)
@. fahrenheit = celsius * 9 / 5 + 32       # .= writes into the existing array
println(fahrenheit)

# Any function can be broadcast, including your own
classify(t) = t < 0 ? "freezing" : t < 25 ? "mild" : "hot"
println(classify.(celsius))

# Shapes combine: a column (3×1) with a row (1×4) gives a 3×4 matrix
col = [1, 2, 3]
row = [10 20 30 40]
display(col .* row)

# Scalars and Refs are treated as single values
println(round.(fahrenheit; digits = 0))
println(in.(["Accra", "Quito"], Ref(["Accra", "Lagos"])))

# Reductions over whole arrays or along a dimension
grid = [3 1 4; 1 5 9; 2 6 5]
println("sum = ", sum(grid), ", max = ", maximum(grid))
println("column sums: ", sum(grid; dims = 1))
println("row maxima: ", vec(maximum(grid; dims = 2)))
println("how many are odd: ", count(isodd, grid), ", where: ", findall(isodd, vec(grid)))

It prints:

text
[23.0, 32.0, 54.5, 69.8, 98.6]
[23.0, 32.0, 54.5, 69.8, 98.6]
["freezing", "mild", "mild", "mild", "hot"]
3×4 Matrix{Int64}:
 10  20  30   40
 20  40  60   80
 30  60  90  120
[23.0, 32.0, 54.0, 70.0, 99.0]
Bool[1, 0]
sum = 36, max = 9
column sums: [6 12 18]
row maxima: [4, 9, 6]
how many are odd: 6, where: [1, 2, 4, 5, 8, 9]

A few things to notice:

  • @. puts a dot on every call and operator in an expression, which is much easier to read than a dozen dots.
  • .= writes the result into an existing array instead of allocating a new one. Together with fusion (below) this lets you update large arrays with no temporary memory at all.
  • Shapes extend. Broadcasting a 3-element column against a 1 × 4 row virtually repeats each along the other's dimension and produces a 3 × 4 matrix; dimensions of length 1 stretch to match. That is how the multiplication table was built without a loop.
  • Protecting an argument. To use a collection as a single value in a broadcast, wrap it in Ref. in.(cities, Ref(list)) asks, for each city, whether it is in list; without Ref, Julia would try to pair the two collections element by element. The result Bool[1, 0] is a vector of true and false, displayed compactly.
  • Reductions such as sum, maximum and count work on whole arrays, or along one dimension with dims, which keeps that dimension with length 1. findall returns the indices where a predicate holds.
  • Rounding halves. 54.5 rounded to 54.0, by the round-half-to-even rule from lesson 2.

Note

Fusion. In celsius .* 9 ./ 5 .+ 32, Julia does not compute three intermediate arrays. All the dotted operations in one expression are fused into a single loop that visits each element once, and with .= the results go straight into the destination. Steven G. Johnson's blog post More Dots (linked below) explains how this is done purely at the level of syntax, which is why it works for any function.

ComputationC

Linear algebra

The LinearAlgebra standard library is loaded with using LinearAlgebra. Under the hood it calls BLAS and LAPACK, the decades-old, heavily tuned Fortran and C libraries that every scientific language relies on (OpenBLAS by default; through the libblastrampoline layer, a package such as MKL.jl can switch to Intel's MKL without recompiling anything).

julia
using LinearAlgebra

# Solving a linear system. A farming cooperative blends three fertilisers, each with a known
# fraction of nitrogen (N), phosphorus (P) and potassium (K). How many kg of each give
# exactly the nutrients a field needs?
#            F1   F2   F3
nutrients = [0.20 0.10 0.00;      # nitrogen fraction in each fertiliser
             0.10 0.20 0.10;      # phosphorus
             0.10 0.10 0.30]      # potassium
target = [15.0, 14.0, 17.0]       # kg of N, P, K wanted in the mix

kg = nutrients \ target           # solve nutrients * kg = target
println("kg of each fertiliser: ", round.(kg; digits = 3))
println("check: ", nutrients * kg ≈ target, ", total mass ", sum(kg), " kg")

# Least squares: the same backslash fits a line when there are more equations than unknowns
x = [1.0, 2.0, 3.0, 4.0, 5.0]
y = [2.1, 3.9, 6.2, 7.8, 10.1]
X = [ones(5) x]                   # columns: intercept, slope
intercept, slope = X \ y
println("best line: y ≈ ", round(intercept; digits = 3), " + ", round(slope; digits = 3), "x")

# Vectors, norms, products and the adjoint
u = [3.0, 4.0]
w = [1.0, 2.0]
println("norm(u) = ", norm(u), ", u ⋅ w = ", u ⋅ w, ", u' * w = ", u' * w)
println("outer product u * w' = ", u * w')

# Eigenvalues of a symmetric matrix, with a structured type
S = Symmetric([2.0 1.0; 1.0 2.0])
vals, vecs = eigen(S)
println("eigenvalues: ", vals)
println("det = ", det(S), ", trace = ", tr(S))
println("I is the identity of any size: ", [1 2; 3 4] + 10I)

It prints:

text
kg of each fertiliser: [62.5, 25.0, 27.5]
check: true, total mass 115.0 kg
best line: y ≈ 0.05 + 1.99x
norm(u) = 5.0, u ⋅ w = 11.0, u' * w = 11.0
outer product u * w' = [3.0 6.0; 4.0 8.0]
eigenvalues: [1.0, 3.0]
det = 2.9999999999999996, trace = 4.0
I is the identity of any size: [11 2; 3 14]
  • A \ b solves A x = b. The backslash, borrowed from MATLAB, picks a suitable method for the matrix: an LU factorisation for a general square matrix, Cholesky for a symmetric positive definite one, and so on. For the fertiliser blend it found 62.5, 25 and 27.5 kg. Never compute inv(A) * b: it is slower and less accurate than solving directly.
  • Least squares for free. When X has more rows than columns, as when fitting a line through five points, there is usually no exact solution, and X \ y returns the coefficients that minimise the sum of squared errors (using a QR factorisation). The data were made up to lie near y=2xy = 2x, and the fit gives y≈0.05+1.99xy \approx 0.05 + 1.99x.
  • Vectors. norm is the Euclidean length, ⋅ (typed \cdot, or the function dot) is the dot product, u' is the adjoint, and u * w' is the outer product, a matrix.
  • Structured matrices. Symmetric(A) wraps a matrix and promises it is symmetric, so eigen can use a faster, more accurate algorithm that returns real eigenvalues in order. Diagonal, Tridiagonal, UpperTriangular and others work the same way: they are types, and multiple dispatch picks the right algorithm for each. I is a special identity object that adapts to whatever size it is combined with.
  • Floating point is honest. The determinant came out as 2.9999999999999996 rather than 3, a rounding error of one unit in the last place. Compare such results with ≈.
Worked examplesH

Tuples, dictionaries and sets

Arrays are not the only collections. Here are the other three you will use most:

julia
# Tuples, named tuples, dictionaries and sets
point = (3, 4)                               # a Tuple: fixed length, immutable
station = (name = "Mombasa", lat = -4.04, lon = 39.67)   # a NamedTuple
println(point[2], " ", station.name, " ", typeof(point))

# Counting words with a Dict
text = "the river feeds the field and the field feeds the village"
counts = Dict{String, Int}()
for word in split(text)
    counts[word] = get(counts, word, 0) + 1
end
for word in sort(collect(keys(counts)))
    println(rpad(word, 8), counts[word])
end
println(haskey(counts, "river"), " ", haskey(counts, "sea"))

# Sets: membership and set algebra
grown_in_ghana = Set(["cocoa", "cassava", "yam", "maize"])
grown_in_peru = Set(["potato", "maize", "quinoa", "cassava"])
println(sort(collect(grown_in_ghana ∩ grown_in_peru)))
println(sort(collect(setdiff(grown_in_peru, grown_in_ghana))))
println("yam" in grown_in_ghana, " ", "yam" ∈ grown_in_peru)

It prints:

text
4 Mombasa Tuple{Int64, Int64}
and     1
feeds   2
field   2
river   1
the     4
village 1
true false
["cassava", "maize"]
["potato", "quinoa"]
true false
  • Tuples like (3, 4) are fixed-length and immutable, and each position can have its own type. They are how functions return several values, and they are very cheap. Named tuples add field names, station.name, and make good lightweight records.
  • Dict{K, V} maps keys to values with hashing. get(d, key, default) avoids an error for a missing key, haskey tests membership, and keys and values iterate. Iteration order is not sorted, so the program sorts the keys before printing.
  • Set holds unique values with fast membership tests; ∩ (intersect), ∪ (union) and setdiff do set algebra, and ∈ (in) tests membership.
ExplanationH

How experienced Julia programmers handle arrays

Write functions for AbstractArray, and loop with eachindex. A function that takes v::Vector{Float64} refuses views, ranges, sparse vectors, GPU arrays and arrays of other number types. A function that takes v::AbstractVector (or no annotation at all) accepts them all, and costs nothing, because Julia compiles a specialised version for each concrete type. Inside, iterate over eachindex(v) rather than 1:length(v):

julia
function scale!(v::AbstractVector, factor)
    for i in eachindex(v)
        v[i] *= factor
    end
    return v
end

eachindex returns the most efficient way to visit every index of that particular array, and it keeps working for arrays whose indices do not start at 1 (the OffsetArrays package provides them, for people who really do want 0-based indexing or indices from −10 to 10). For matrices, axes(A, 1) gives the row indices, and eachrow(A), eachcol(A) and CartesianIndices(A) iterate rows, columns and multidimensional indices.

Allocate once. In a loop that runs millions of times, creating a new array on every iteration costs memory and time for the garbage collector. Experienced programmers create the output once, with similar(v) (same shape and type, uninitialised) or zeros(n), and fill it with a mutating function or .=. You saw the ! convention in the previous lesson; most array functions come in both flavours, such as map and map!, sort and sort!, mul! for matrix products in LinearAlgebra.

Pick the right array type. The generic design means that specialised arrays plug into the same code. Among the ones you will meet:

PackageArray typeUse it for
SparseArrays (standard library)SparseMatrixCSChuge matrices that are mostly zeros, as in networks and finite elements
StaticArrays.jlSVector, SMatrixsmall fixed-size vectors (points, colours, 3 × 3 rotations), stored without heap allocation
CUDA.jl, AMDGPU.jl, Metal.jlCuArray and friendsarrays living on a GPU; broadcasting and many array functions run there unchanged
OffsetArrays.jlOffsetArrayindices that start somewhere other than 1

A broadcast expression written for a Vector usually works on a CuArray without changes, because the dot syntax is lowered to generic functions that each array type can implement in its own way. This is multiple dispatch at work again.

Common misconceptionsH

Misconceptions

Misconception

"b = a makes a copy, as it does for numbers." For numbers, the question never arises, because numbers are immutable. For arrays and other mutable objects, assignment always binds a second name to the same object. Use copy (or deepcopy for nested structures) when you need independence.

Misconception

"Slicing makes a view, like in NumPy." In NumPy, basic slices are views. In Julia, a[2:3] on the right-hand side of an expression makes a copy; you ask for a view with view or @view. (On the left-hand side, as in a[2:3] .= 0, you are writing into a itself.)

Misconception

"Vectorised code is always faster than loops." That is advice for interpreted languages, where a loop runs slowly and a vectorised call jumps into C. In Julia, a loop inside a function compiles to the same kind of machine code that a vectorised library would, so choose whichever is clearer. Vectorised code that allocates temporary arrays can even be slower than a loop; that is exactly what fusion and .= are for.

Exercises

Exercises

1. Arrays, broadcasting and linear algebra

Warm-upComputational

With A = [1 2; 3 4], what is A[2, 1]?

2. Arrays, broadcasting and linear algebra

CoreComputational

What does println(vec([1 2; 3 4])) print?

3. Arrays, broadcasting and linear algebra

CoreComputational

What does println(size([1, 2, 3] .+ [10 20])) print?

4. Arrays, broadcasting and linear algebra

Warm-upComputational

What does this print?

a = [1, 2, 3]
b = a
b[1] = 99
println(a[1])

5. Arrays, broadcasting and linear algebra

StretchComputational

What does this print?

a = [1, 2, 3]
s = a[1:2]
s[1] = 0
v = view(a, 1:2)
v[2] = 0
println(a)

6. Arrays, broadcasting and linear algebra

CoreComputational

What does println(sum([1 2; 3 4]; dims = 1)) print?

7. Arrays, broadcasting and linear algebra

CoreComputational

With x = [1, 2, 3], which expression computes the dot product 1·1 + 2·2 + 3·3 = 14?

Choose one

8. Arrays, broadcasting and linear algebra

CoreHuman

For a square, invertible matrix A and a vector b, what does A \ b compute, and how?

Choose one

Further reading and resources