Skip to content
LumenaryAcademy
Foundations 65 min

Storage and managed databases

Block, file and object storage; what "eleven nines" really promises; replication and erasure codes; managed databases and their consistency; and backups you can actually restore.

Code tested · 5AI-draftedAwaiting review by a cloud engineer

Your guides: Chloe · The Computer Scientist, Margaret · The Experimenter, Gideon · The Verifier

MotivationH

Forty terabytes and a catalogue

A city museum in Kraków is digitising its collection: two million high-resolution scans, about 40 TB, which researchers worldwide will view online. It also has a catalogue — a relational database of objects, artists, loans and conservation records — and a small team of developers. The museum's director asks three questions that every cloud storage decision comes down to. Where should each kind of data live? How likely are we to lose any of it? And if something goes badly wrong — a region outage, a bug, a ransomware attack, a tired administrator typing a command into the wrong window — how much would we lose, and how long would we be closed?

The providers' marketing answers the second question with a number like "99.999999999% durability", eleven nines. This lesson explains what such a number means, computes it from a model, and shows which assumption in that model matters most. It then turns to the databases built on top of storage, the consistency they really offer, and backups. Lumenary CS's storage at scale (storage engines, partitioning, object stores), replication and consensus and databases lessons are the background; here we see how a cloud packages those ideas as services.

DefinitionsH

Block, file and object storage

Clouds offer three fundamentally different kinds of storage, distinguished by the interface they present.

Definition— Block, file and object storage

Block storage presents a virtual disk: an array of fixed-size blocks that one machine's operating system formats with a file system and reads and writes at any offset. File storage presents a shared file system — directories, files, permissions, locks — that many machines mount at once over a network protocol such as NFS or SMB. Object storage presents a flat namespace of buckets and keys, each holding an object written whole and read whole or by byte range, over an HTTP API.

BlockFileObject
Interfacea disk (/dev/nvme1n1)a mounted directory (/mnt/shared)HTTP: PUT, GET, DELETE a key
Shared byone machine at a time (usually)many machinesany number of clients, anywhere
Latencywell under a millisecond to a few msa few mstens of ms per request
Typical usedatabases, a VM's system diskshared home directories, render farms, legacy applicationsmedia, backups, data lakes, static websites
ExamplesAmazon EBS, Azure Managed Disks, Google Persistent Disk and Hyperdisk, Alibaba Cloud ESSD; Ceph RBDAmazon EFS and FSx, Azure Files, Google Filestore, Alibaba Cloud NAS; CephFS, NFS serversAmazon S3, Azure Blob Storage, Google Cloud Storage, Alibaba Cloud OSS; Ceph RGW

The S3 API has become a de facto standard, accepted by many object stores — Alibaba Cloud OSS, Google Cloud Storage's interoperability mode, many regional providers and open-source servers such as Ceph's gateway — though Azure Blob Storage has its own API. Block storage is usually zonal: a disk lives in one zone and attaches only to machines there (some providers also offer regional disks, replicated to a second zone). Managed file systems and object stores are usually replicated across the zones of a region, and object stores across regions if you ask.

There is a fourth kind that is easy to overlook: instance storage, NVMe drives physically inside the host. It is the fastest storage there is, and it is ephemeral: when the VM is stopped or moved, or the host fails, the data is gone. It suits caches, scratch space and databases that replicate themselves across machines, never the only copy of anything.

Object stores also offer storage classes that trade price per gigabyte against the cost and delay of reading. As of 2026, archive tiers on the large clouds cost roughly a tenth or less of the standard tier per gigabyte-month, but charge for retrieval and for deleting an object before a minimum period (from 30 days to a year, depending on class and provider), and some take hours to bring an object back. Lifecycle rules move objects between classes by age automatically: the museum's scans might stay in the standard class while new, move to an infrequent-access class after a year, and keep a second copy in an archive class in another region.

ComputationC

What eleven nines means

Definition— Durability and availability

Durability is the probability that a stored object is not lost over a period, usually a year. Availability is the fraction of time the service answers requests successfully. A store can be unavailable for an hour without losing anything; it can be available while having quietly lost an object.

The two numbers differ enormously in practice. The large object stores state designed annual durabilities of eleven nines or more, while their availability targets and service-level agreements are typically three or four nines. Eleven nines means a probability of about 10−1110^{-11} of losing a given object in a year. For the museum's two million scans that is an expected 2×106×10−11=2×10−52 \times 10^{6} \times 10^{-11} = 2 \times 10^{-5} objects a year: on average, one lost scan every 50,000 years — if the model behind the number is right.

Where does such a number come from? From a model of failure and repair. An object is stored as nn fragments — whole copies, or erasure-coded pieces — on nn different disks, and survives as long as no more than a certain number are lost at once. Disks fail at random; when one does, the system rebuilds its fragments elsewhere from the survivors. Data is lost only if enough further disks fail before the repairs finish. The block below computes this with a small Markov chain, for an illustrative annual disk failure rate of 2%.

python
import math

AFR = 0.02                          # each disk fails with probability about 2% a year (an example)
lam = -math.log(1 - AFR)            # the same, as a failure rate per disk per year

def mttdl_years(n, tolerate, repair_hours):
    """Mean time to data loss for an object stored as n fragments (copies or erasure-coded
    pieces) on n different disks, which survives losing up to `tolerate` of them. Lost
    fragments are rebuilt in parallel, each taking about repair_hours on average.

    A Markov chain on f = fragments currently lost: f -> f+1 at rate (n - f) * lam,
    f -> f-1 at rate f * mu. If tau_f is the expected time to first reach f+1 from f,
    then tau_f = (1 + f * mu * tau_(f-1)) / ((n - f) * lam), and MTTDL is their sum."""
    mu = 365 * 24 / repair_hours
    tau, total = 0.0, 0.0
    for f in range(tolerate + 1):
        tau = (1 + f * mu * tau) / ((n - f) * lam)
        total += tau
    return total

print("scheme                stores  repair   P(lose a given object in a year)  nines")
for name, n, k in [("2 copies", 2, 1), ("3 copies", 3, 1), ("erasure code (6, 3)", 9, 6),
                   ("erasure code (10, 4)", 14, 10)]:
    for hours in [24, 1]:
        p = -math.expm1(-1 / mttdl_years(n, n - k, hours))   # 1 - exp(-1/MTTDL), computed accurately
        print(f"{name:20} {n / k:5.1f}x {hours:5d} h {p:22.1e} {-math.log10(p):20.1f}")

# The model assumes independent failures. If every fragment is in one zone, an event that
# destroys the zone (fire, flood, a faulty bulk deletion) caps durability, whatever the code.
for p_zone in [1e-4, 1e-6]:
    print(f"all fragments in one zone, lost with probability {p_zone:.0e} a year: at most {-math.log10(p_zone):.0f} nines")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Three copies with a day to repair give about 9.7 nines; repairing within an hour gives 12.5. A (10,4)(10, 4) erasure code — ten data fragments plus four parity fragments, surviving any four losses — stores only 1.4 bytes per byte yet reaches 14.7 nines with day-long repairs. Two lessons follow. Fast repair is worth as much as extra copies: with rr copies, the loss probability falls roughly like (λ/μ)r−1(\lambda/\mu)^{r-1}, so cutting repair time 24-fold gains r−1r - 1 factors of 24. And erasure codes buy durability cheaply, which is why every large object store uses them for most data.

ComputationC

Replication and erasure coding

Replication is the simple answer: keep rr full copies on different disks, racks or zones. Reads can go to any copy, a lost copy is rebuilt by copying a survivor, and the logic is easy to verify. It costs rr bytes of disk per byte stored.

An erasure code with parameters (k,m)(k, m) splits data into kk fragments and computes mm extra parity fragments such that any kk of the k+mk + m reconstruct the original. Lumenary CS's storage-at-scale lesson met the (10,4)(10, 4) Reed–Solomon code; the block below builds a small code of the same kind from scratch, using nothing but polynomials.

python
P = 257                                       # a prime: arithmetic modulo P is a field

def interpolate(points, x):
    """The value at x of the unique polynomial of degree < len(points) through `points`
    (Lagrange interpolation, all arithmetic modulo P)."""
    total = 0
    for i, (xi, yi) in enumerate(points):
        num, den = 1, 1
        for j, (xj, _) in enumerate(points):
            if j != i:
                num = num * (x - xj) % P
                den = den * (xi - xj) % P
        total = (total + yi * num * pow(den, P - 2, P)) % P   # division = multiply by inverse
    return total

def encode(data, k, m):
    """Split data into k fragments and add m parity fragments: fragment i holds, for each
    stripe of k symbols, the value at x = i + 1 of the polynomial through that stripe."""
    data = data + bytes(-len(data) % k)       # pad to whole stripes
    stripes = [data[s:s + k] for s in range(0, len(data), k)]
    return [[interpolate(list(zip(range(1, k + 1), st)), x) for st in stripes]
            for x in range(1, k + m + 1)]     # fragments 1..k are the data itself

def decode(fragments, k):
    """Rebuild the data from ANY k surviving fragments (None marks a lost one)."""
    alive = [(x + 1, f) for x, f in enumerate(fragments) if f is not None][:k]
    if len(alive) < k:
        raise ValueError("too many fragments lost")
    out = []
    for s in range(len(alive[0][1])):
        pts = [(x, f[s]) for x, f in alive]
        out += [interpolate(pts, x) for x in range(1, k + 1)]
    return bytes(out)

record = "Synthetic test record: archive scan 0042, digitised 2026-11-19".encode()
k, m = 6, 3
frags = encode(record, k, m)
print(f"{len(record)} bytes -> {k + m} fragments of {len(frags[0])} symbols "
      f"(overhead {(k + m) / k:.2f}x, any {m} may be lost)")
print("data fragment 1 starts:", frags[0][:6], "  parity fragment 7 starts:", frags[6][:6])
for lost in [(0, 3, 8), (1, 2, 4), (5, 6, 7)]:
    damaged = [None if i in lost else f for i, f in enumerate(frags)]
    print(f"lose fragments {[i + 1 for i in lost]}: rebuilt correctly?",
          decode(damaged, k)[:len(record)] == record)
try:
    decode([None, None, None, None] + frags[4:], k)
except ValueError as e:
    print("lose 4 fragments:", e)
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The code is systematic: fragments 1 to 6 hold the data itself (interleaved, one symbol of each stripe), so a healthy read needs no decoding. Any three fragments can be lost — data, parity or a mixture — and the record comes back; a fourth loss is fatal. Production codes work in the field with 282^8 elements rather than modulo 257, so that every symbol is exactly one byte (here a parity symbol may be 256), and use fast matrix methods rather than Lagrange's formula, but the principle is the same.

Erasure coding has costs that replication does not. Rebuilding one lost fragment means reading kk others, so a (10,4)(10, 4) code moves ten times as much data across the network for a repair as replication would. Small objects are inefficient to split, so stores often replicate them and erasure-code large ones. And reads during a failure must decode. Engineers have attacked the repair cost directly: Azure's storage team described locally repairable codes, which add parity over small groups so that most repairs read only a few fragments, in a 2012 paper.

Placement completes the design. Storing fragments in three zones so that the loss of a whole zone leaves at least kk of them is what turns a code's durability into protection against the correlated failures Gideon described. Azure's redundancy options make this choice explicit — locally, zone- and geo-redundant storage — and the other providers offer similar choices through storage classes and multi-region buckets.

ExplanationH

Managed databases

Most applications keep their important state not in raw storage but in a database, and in the cloud most teams rent it managed. A managed relational database — Amazon RDS and Aurora, Azure SQL Database and Azure Database for PostgreSQL and MySQL, Google Cloud SQL and AlloyDB, Alibaba Cloud ApsaraDB RDS and PolarDB, among others — runs PostgreSQL, MySQL, SQL Server or a compatible engine for you. What "managed" includes is worth listing precisely:

  • Provided: installation and minor-version patching, automated backups with point-in-time recovery, a standby replica in another zone with automatic failover, read replicas, monitoring, encryption at rest, and storage that grows.
  • Still yours: the schema and indexes, query performance, connection management, access control (who can connect, with what privileges), choosing backup retention and testing restores, major-version upgrades (scheduled by you, tested by you), and the bill.

Several of these services separate compute from storage. Amazon Aurora, for example, sends only its redo log to a shared storage layer that keeps six copies across three zones and accepts a write once four have it, as its SIGMOD 2017 paper explains; Azure SQL Hyperscale, PolarDB, AlloyDB and the open-source Neon follow related designs. Failover and new replicas become faster because the storage is already shared.

Beyond one primary per database, two other families exist. Distributed SQL databases partition and replicate tables across many machines and zones, with consensus for each partition and transactions across them: Google's Spanner and open-source or source-available systems such as CockroachDB, YugabyteDB, TiDB and OceanBase. NoSQL services trade the relational model for predictable scaling on simpler access patterns: key-value and document stores such as Amazon DynamoDB, Azure Cosmos DB, Google Firestore and Bigtable, and Alibaba Cloud Tablestore, with Apache Cassandra, ScyllaDB and MongoDB among the self-run options.

A managed database is a trade: less operational work, more dependence on one provider's product, its limits and its price. Engines that are open source (PostgreSQL, MySQL) keep the option of moving; proprietary APIs make moving a rewrite. Neither choice is wrong, but it should be a choice.

ComputationC

Consistency in practice

Lumenary CS's replication-and-consensus lesson defined linearizability and eventual consistency. In the cloud the question becomes concrete: if I write and then read, will I see my write?

Within one primary database, yes. The trouble starts with read replicas, which most managed relational databases update asynchronously: a write commits on the primary and reaches each replica some milliseconds — occasionally seconds — later. A read sent to a replica in that window returns the old value. The block uses an illustrative lag distribution: usually about 20 ms, but 5% of the time the replica is busy and lags about a second.

python
import math

# Replication lag of an asynchronous read replica, as a mixture (illustrative numbers):
# usually it applies a write within ~20 ms, but 5% of the time it is busy and lags ~1 s.
def p_stale(delay_s):
    """Probability that a read sent delay_s after a write still sees the old value."""
    return 0.95 * math.exp(-delay_s / 0.020) + 0.05 * math.exp(-delay_s / 1.0)

print("read issued after the write   P(stale read from replica)")
for d in [0.005, 0.05, 0.2, 1, 5]:
    print(f"{d * 1000:10.0f} ms {p_stale(d):27.2%}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The usual remedy is a session guarantee rather than strong consistency everywhere: read-your-writes (a client always sees its own writes) and monotonic reads (it never sees time go backwards). Applications get them by sending a user's reads to the primary for a short while after that user writes, or by carrying a log position from the write and reading only from a replica that has applied it. Services expose such choices explicitly. Azure Cosmos DB offers five consistency levels, from strong through bounded staleness, session and consistent prefix to eventual; DynamoDB's reads are eventually consistent by default with a strongly consistent option within a region; Spanner provides externally consistent transactions across regions using tightly synchronised clocks; and the major object stores, after years of eventual consistency for some operations, now give strong read-after-write consistency for objects (Amazon S3 since December 2020).

Across regions the speed of light returns. A synchronous write to another continent costs at least one intercontinental round trip, the networking lesson's 100 ms or more, so most multi-region designs replicate asynchronously and accept that the newest writes can be lost if a region fails — which is exactly the RPO of the next section.

ComputationC

Backups and disaster recovery

Definition— RPO and RTO

The recovery point objective (RPO) is the most data, measured in time, that you are prepared to lose in a disaster: an RPO of 5 minutes means the restored system may lack at most the last 5 minutes of changes. The recovery time objective (RTO) is the longest you are prepared to be out of service before the system works again.

timelast recoverable pointdisasterservice restoredRPORTOchanges losttime out of service
RPO and RTO on a timeline. Changes made after the last recoverable point are lost; the time from the disaster until service is restored is downtime. Each design trades cost against both.

Both are business decisions, and both cost money: a smaller RPO needs more frequent copies or continuous replication; a smaller RTO needs more of the system already running somewhere else. The block compares four designs for the museum's 2 TB catalogue database, with illustrative throughput figures.

python
# Recovery for a 2 TB catalogue database, four designs (illustrative figures).
DB_GB = 2000
RESTORE_GB_PER_H = 1800          # copying a snapshot back from object storage, about 0.5 GB/s
WAL_GB_PER_H = 5                 # change log the database writes per hour
REPLAY_GB_PER_H = 50             # how fast a restored copy can replay that log
PROVISION_H = 0.25               # starting a new database server

def restore_hours(snapshot_every_h):
    worst_log = snapshot_every_h * WAL_GB_PER_H              # log written since the last snapshot
    return PROVISION_H + DB_GB / RESTORE_GB_PER_H + worst_log / REPLAY_GB_PER_H

designs = [
    # name, worst-case RPO in hours, worst-case RTO in hours
    ("nightly snapshot only", 24, PROVISION_H + DB_GB / RESTORE_GB_PER_H),
    ("nightly snapshot + log archived every 5 min", 5 / 60, restore_hours(24)),
    ("4-hourly snapshot + log archived every 5 min", 5 / 60, restore_hours(4)),
    ("warm standby in another region (async, ~2 s lag)", 2 / 3600, 10 / 60),
]
print("design                                              worst RPO     worst RTO")
for name, rpo, rto in designs:
    rpo_text = f"{rpo:.0f} h" if rpo >= 1 else (f"{rpo * 60:.0f} min" if rpo * 60 >= 1 else f"{rpo * 3600:.0f} s")
    print(f"{name:50} {rpo_text:>10} {rto * 60:9.0f} min")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Archiving the database's change log every five minutes, as PostgreSQL's point-in-time recovery and every managed service do, cuts the worst-case RPO from a day to five minutes — but lengthens the worst-case RTO from 82 to 226 minutes, because a restore just before the next nightly snapshot must replay almost a day of log. Snapshotting every four hours brings it back to 106 minutes. Only a standby already running in another region gets both down to seconds and minutes, at the price of a second database running all the time. Industry guides name a ladder of such designs — backup and restore, a "pilot light" of minimal standby infrastructure, warm standby, and active–active across regions — each costing more and recovering faster.

Five habits separate backups that work from backups that exist:

  1. Replication is not a backup. A mistaken DELETE, a buggy migration or ransomware is copied to every replica within seconds. Backups are copies from the past.
  2. Keep copies apart: the old 3-2-1 rule — three copies, on two kinds of media, one off-site — translates to copies in another region and, ideally, another account with separate credentials.
  3. Make some copies immutable. Object-lock or write-once retention settings stop anyone, including an attacker with stolen administrator credentials, from deleting or altering a backup before its retention ends.
  4. Protect the keys. Encrypted backups are only as recoverable as the keys that decrypt them; a key deleted with the account takes every backup with it.
  5. Test restores, regularly and timed. A backup that has never been restored is a hope, not a plan, and the measured restore time is your real RTO.

Backups of personal data also carry obligations: data-protection laws such as the GDPR expect retention periods to be justified, and deletion requests to reach backups eventually, typically by letting backups expire on a defined schedule rather than editing them.

Common misconceptionsH

Common misconceptions

Misconception

"Eleven nines of durability means my data is safe." It means the provider's storage is very unlikely to lose an object through hardware failure. It says nothing about you or your software deleting or overwriting it, about an attacker with your credentials, or about losing access to your account. Those are covered by versioning, immutable backups and access control — your side of the shared-responsibility model.

Misconception

"Durability and availability are the same thing." A store can be unreachable without losing data, and durability figures (eleven nines) are far higher than availability figures (three or four nines). Design for the store being unavailable sometimes.

Misconception

"A multi-zone database is a backup." A standby in another zone protects against losing a zone, not against a bad write, which it faithfully replicates. You need both: replicas for availability, backups for recovery.

Misconception

"Data on the VM's disk is stored durably." Network block volumes are replicated within their zone; local instance storage disappears with the VM or the host. Know which kind each path on your machine is.

Exercises

Exercises

1. Storage and managed databases

Warm-upHuman

A film-restoration studio in Mumbai needs storage for three things: (1) the data directory of the PostgreSQL server that tracks its projects; (2) a directory of shared scripts and reference frames that 40 rendering machines mount at the same time; (3) 300 TB of finished, rarely changed video masters that partners download over HTTPS. Which storage kinds fit best, in that order?

Choose one

2. Storage and managed databases

CoreHuman

A national library stores 500 million objects in an object store with a designed annual durability of eleven nines (a probability of 10−1110^{-11} of losing a given object in a year, independently). On average, how many years pass between losses of a single object?

3. Storage and managed databases

Warm-upHuman

A storage cluster protects data with a (12,4)(12, 4) erasure code: every object is split into 12 data fragments, 4 parity fragments are added, and any 12 of the 16 suffice to rebuild it. How many TB of raw disk does it need to store 600 TB of data (ignoring metadata)?

4. Storage and managed databases

CoreHuman

A storage service promises 99.9% availability per month. In a 30-day month, how many seconds of unavailability can occur before the promise is broken?

5. Storage and managed databases

CoreHuman

A clinic-booking service in Bogotá takes a database snapshot every 6 hours (at 00:00, 06:00, 12:00 and 18:00) and keeps no change log. At 17:40 a faulty migration corrupts the database. The team restores the 12:00 snapshot and the service is working again at 19:10. What recovery point and recovery time did they actually achieve?

Choose one

6. Storage and managed databases

CoreHuman

A company's managed database has a synchronous standby in a second zone and an asynchronous replica in another region. An engineer accidentally runs DELETE FROM orders; without a WHERE clause on the primary. Which of these will let the company get its orders back?

Choose one

7. Storage and managed databases

StretchHuman

A language-learning app in Hanoi sends all writes to its database's primary and all reads to two asynchronous read replicas, to spread the load. Users report that after they change their daily goal, the settings page sometimes still shows the old goal, and shows the new one if they reload a few seconds later. Which change fixes this with the least extra load on the primary?

Choose one

8. Storage and managed databases

CoreComputational

The simplest erasure code adds one parity block: the byte-by-byte XOR of kk equal-length data blocks. It survives the loss of any one block. Write xor_parity(blocks), which returns that parity as bytes, and rebuild(blocks, parity), which receives the data blocks with exactly one replaced by None and returns a new list with the lost block recomputed. Do not modify the list you are given.

The starter's functions do not compute anything yet.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Further reading and resources