Skip to content
LumenaryAcademy
Professional 70 min

Tools, where OCaml is used, and a complete project

How OCaml teams build, test, format and document code with dune and opam, where the language is used in industry, and a small compiler written from lexer to virtual machine.

Code tested · 2AI-draftedAwaiting review by a computer scientist
MotivationH

From files to projects

Every program in this course so far has been a single file run with ocaml. Real OCaml software is a dune project: several libraries and executables, dependencies installed by opam, tests run on every change, code formatted automatically and documented from its interface files. This lesson shows that everyday toolchain, describes honestly where OCaml is used and why, and then builds the course's final project: a small compiler, from source text to a stack-machine program and its result. Compilers are what OCaml was born for, and the project uses nearly every idea of the course.

ExplanationH

The professional toolchain

A dune project

dune init project calc creates a layout like this:

text
calc/
  dune-project          the project's name and dune language version
  calc.opam             package metadata, generated from dune-project
  bin/dune  bin/main.ml the executable
  lib/dune  lib/*.ml    the library with the real code
  test/dune test/*.ml   the tests

Each directory's dune file declares what it contains, in a small S-expression language:

text
; lib/dune
(library
 (name calc)
 (libraries str))

; bin/dune
(executable
 (name main)
 (libraries calc))

; test/dune
(test
 (name test_calc)
 (libraries calc alcotest))

Every .ml file in lib/ becomes a module of the library Calc (so lib/parser.ml is Calc.Parser), and libraries lists dependencies, both opam packages and libraries that ship with the compiler, such as str and unix. The commands you will type every day:

CommandWhat it does
dune buildBuild everything; in the default development profile many warnings are errors.
dune build --watchRebuild on every save.
dune exec bin/main.exeBuild and run the executable.
dune testBuild and run all tests.
dune fmtReformat the code with ocamlformat (and promote the changes).
dune utop libOpen utop with the library loaded.
opam install . --deps-only --with-testInstall the dependencies the project's .opam file declares.

Teams usually create a local opam switch in the project directory (opam switch create . --deps-only), so each project has its own compiler and library versions, and commit a lock file (opam lock) for reproducible builds.

Editors, formatting, documentation

  • ocaml-lsp-server, built on the long-standing merlin engine, gives VS Code, Neovim, Emacs and other editors types on hover, errors as you type, completion, go-to-definition and renaming. It is one of the reasons OCaml feels productive: you see every inferred type instantly.
  • ocamlformat formats code according to a .ocamlformat file in the project; most teams run it in continuous integration so style never comes up in review.
  • odoc turns the (** ... *) comments in .mli files into HTML documentation; the documentation on ocaml.org's package pages is generated this way.
  • ppx preprocessors extend the language at compile time: ppx_deriving generates printers, comparisons and serialisers from type definitions ([@@deriving show, eq]), and ppx_expect adds expect tests.

Testing

OCaml projects mix three styles of test:

  • Unit tests with Alcotest: named test cases that compare actual and expected values, with readable failure reports.
  • Expect tests with ppx_expect: a test prints output, the expected output is written in the source next to it, and when the output changes, dune shows a diff and can update the file for you (dune promote). Jane Street uses them extensively; they make tests of complex output cheap to write and to review. Dune's cram tests apply the same idea to whole command-line sessions.
  • Property-based tests with QCheck: generate many random inputs and check laws, shrinking failures to small counterexamples.

The next program shows the first and third ideas with nothing but the standard library, so you can see what the frameworks automate:

ocaml
(* The function under test: turn a title into a URL-friendly "slug". *)
let slugify s =
  String.lowercase_ascii s
  |> String.map (fun c -> if (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') then c else '-')
  |> String.split_on_char '-'
  |> List.filter (fun part -> part <> "")
  |> String.concat "-"

let has_substring sub s =
  let n = String.length sub in
  let rec at i = i + n <= String.length s && (String.sub s i n = sub || at (i + 1)) in
  at 0

(* Example-based tests: a table of inputs and expected outputs. *)
let examples =
  [ ("Hello, World!", "hello-world"); ("  OCaml 5.4 ", "ocaml-5-4"); ("---", ""); ("Dar es Salaam", "dar-es-salaam") ]

(* Property-based tests: laws that must hold for every input. *)
let properties =
  [
    ("slugify is idempotent", fun s -> slugify (slugify s) = slugify s);
    ("no double dashes", fun s -> not (has_substring "--" (slugify s)));
    ("no dash at either end", fun s ->
        let r = slugify s in
        r = "" || (r.[0] <> '-' && r.[String.length r - 1] <> '-'));
  ]

let () =
  let failed = List.filter (fun (input, expected) -> slugify input <> expected) examples in
  List.iter
    (fun (input, expected) ->
      Printf.printf "FAIL slugify %S: expected %S, got %S\n" input expected (slugify input))
    failed;
  Printf.printf "examples: %d of %d passed\n" (List.length examples - List.length failed) (List.length examples);
  let rng = Random.State.make [| 2026 |] in
  let random_string () =
    String.init (Random.State.int rng 25) (fun _ -> Char.chr (32 + Random.State.int rng 95))
  in
  let inputs = List.init 500 (fun _ -> random_string ()) in
  List.iter
    (fun (name, law) ->
      match List.find_opt (fun s -> not (law s)) inputs with
      | None -> Printf.printf "property %S: ok on %d random inputs\n" name (List.length inputs)
      | Some s -> Printf.printf "property %S: FAILED on %S\n" name s)
    properties
text
examples: 4 of 4 passed
property "slugify is idempotent": ok on 500 random inputs
property "no double dashes": ok on 500 random inputs
property "no dash at either end": ok on 500 random inputs

The example table catches regressions in specific cases you thought of; the properties check general laws on 500 inputs you did not think of, including empty strings, strings of punctuation and strings with many spaces. The random generator is seeded (Random.State.make [| 2026 |]), so the test is reproducible, which matters in continuous integration. QCheck adds generators for every common type, shrinking of failing inputs, and integration with Alcotest and dune.

History & global context

Where OCaml is used

OCaml is a niche language by headcount, and it is honest to say so: fewer jobs, fewer libraries and fewer tutorials than Python, Java or Go. Its users tend to be organisations for which correctness and maintainability of complex software matter a great deal, and they often say the language gives them an unusual degree of confidence when changing large systems. Some well-documented examples:

  • Finance. Jane Street, a trading firm, writes nearly all of its software in OCaml, from trading systems to tools, and is one of the language's main contributors (the Base and Core libraries, dune's origins, compiler optimisations). Other trading firms and Bloomberg have used it too; Bloomberg created BuckleScript, which became ReScript, a language for the web derived from OCaml.
  • Compilers, analysers and proof tools. The Rocq proof assistant (formerly Coq), the Frama-C and Astrée static analysers, Semgrep's analysis engine, Meta's Flow and Infer and its Hack type checker, the reference interpreter in the WebAssembly specification, the Haxe compiler, and the very first Rust compiler were all written in OCaml.
  • Systems and infrastructure. MirageOS builds "unikernels", single-purpose operating systems compiled from OCaml libraries, and its networking libraries are used in VPNKit, the networking component of Docker Desktop. The XenServer virtualisation toolstack is written in OCaml.
  • Blockchain. The reference implementation of the Tezos protocol, Octez, is written in OCaml, by teams with a strong emphasis on formal verification.
  • Web and data. Ahrefs runs a large web crawler and back end in OCaml; several companies use it for data pipelines.

The common thread is software whose core is a complex transformation of structured data (programs, financial instruments, network packets, proofs), which is exactly what variants, pattern matching and modules are good at. The project below is a small example of that kind of software.

Worked examplesH

Project: a compiler and a stack machine

The program reads arithmetic expressions with let bindings, such as let x = 5 in x * x - 1, and runs them through four stages, each its own module:

  1. Lexer: text to a list of tokens (LET, IDENT "x", EQUALS, INT 5, ...).
  2. Parser: tokens to a syntax tree, by recursive descent, one function per grammar rule.
  3. Compiler: syntax tree to instructions for a stack machine, resolving each variable to a position in the environment, and rejecting unknown variables before anything runs.
  4. Virtual machine: executes the instructions with a stack of values and a stack of bound variables.

Every stage returns a result, and the pipeline chains them with let*.

ocaml
(* A tiny compiler: source text -> tokens -> syntax tree -> stack-machine code -> result. *)

module Lexer = struct
  type token = INT of int | IDENT of string | LET | IN | PLUS | MINUS | STAR | SLASH | LPAREN | RPAREN | EQUALS

  let is_digit c = c >= '0' && c <= '9'
  let is_letter c = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c = '_'

  let tokenize (s : string) : (token list, string) result =
    let n = String.length s in
    let rec go i acc =
      if i >= n then Ok (List.rev acc)
      else
        let c = s.[i] in
        let span pred =
          let j = ref i in
          while !j < n && pred s.[!j] do incr j done;
          (String.sub s i (!j - i), !j)
        in
        match c with
        | ' ' | '\t' | '\n' -> go (i + 1) acc
        | '+' -> go (i + 1) (PLUS :: acc)
        | '-' -> go (i + 1) (MINUS :: acc)
        | '*' -> go (i + 1) (STAR :: acc)
        | '/' -> go (i + 1) (SLASH :: acc)
        | '(' -> go (i + 1) (LPAREN :: acc)
        | ')' -> go (i + 1) (RPAREN :: acc)
        | '=' -> go (i + 1) (EQUALS :: acc)
        | c when is_digit c ->
            let digits, j = span is_digit in
            go j (INT (int_of_string digits) :: acc)
        | c when is_letter c -> (
            let word, j = span (fun c -> is_letter c || is_digit c) in
            match word with
            | "let" -> go j (LET :: acc)
            | "in" -> go j (IN :: acc)
            | _ -> go j (IDENT word :: acc))
        | c -> Error (Printf.sprintf "unexpected character %C at position %d" c i)
    in
    go 0 []
end

module Ast = struct
  type op = Add | Sub | Mul | Div
  type expr = Int of int | Var of string | Neg of expr | Binop of op * expr * expr | Let of string * expr * expr
end

module Parser = struct
  open Lexer

  exception Error of string

  (* Recursive descent over a mutable token stream; errors become a result at the boundary. *)
  let parse (tokens : token list) : (Ast.expr, string) result =
    let rest = ref tokens in
    let peek () = match !rest with t :: _ -> Some t | [] -> None in
    let advance () = match !rest with _ :: tl -> rest := tl | [] -> () in
    let expect t what = if peek () = Some t then advance () else raise (Error ("expected " ^ what)) in
    let rec expr () =
      match peek () with
      | Some LET -> (
          advance ();
          match peek () with
          | Some (IDENT name) ->
              advance ();
              expect EQUALS "'='";
              let bound = expr () in
              expect IN "'in'";
              Ast.Let (name, bound, expr ())
          | _ -> raise (Error "expected a name after 'let'"))
      | _ -> sum ()
    and sum () =
      let rec loop left =
        match peek () with
        | Some PLUS -> advance (); loop (Ast.Binop (Add, left, product ()))
        | Some MINUS -> advance (); loop (Ast.Binop (Sub, left, product ()))
        | _ -> left
      in
      loop (product ())
    and product () =
      let rec loop left =
        match peek () with
        | Some STAR -> advance (); loop (Ast.Binop (Mul, left, unary ()))
        | Some SLASH -> advance (); loop (Ast.Binop (Div, left, unary ()))
        | _ -> left
      in
      loop (unary ())
    and unary () =
      match peek () with
      | Some MINUS -> advance (); Ast.Neg (unary ())
      | _ -> atom ()
    and atom () =
      match peek () with
      | Some (INT n) -> advance (); Ast.Int n
      | Some (IDENT x) -> advance (); Ast.Var x
      | Some LPAREN ->
          advance ();
          let e = expr () in
          expect RPAREN "')'";
          e
      | Some _ -> raise (Error "unexpected token")
      | None -> raise (Error "unexpected end of input")
    in
    match expr () with
    | e -> if !rest = [] then Ok e else Error "extra input after the expression"
    | exception Error msg -> Error msg
end

module Compiler = struct
  type instr = Push of int | Load of int | Bind | Unbind | Neg | Add | Sub | Mul | Div

  (* Variables become positions in the environment: 0 is the innermost binding. *)
  let compile (e : Ast.expr) : (instr list, string) result =
    let rec go scope = function
      | Ast.Int n -> Ok [ Push n ]
      | Ast.Var x -> (
          let rec index i = function
            | [] -> None
            | y :: rest -> if x = y then Some i else index (i + 1) rest
          in
          match index 0 scope with
          | Some i -> Ok [ Load i ]
          | None -> Error ("unknown variable " ^ x))
      | Ast.Neg e -> Result.map (fun c -> c @ [ Neg ]) (go scope e)
      | Ast.Binop (op, a, b) ->
          let ( let* ) = Result.bind in
          let* ca = go scope a in
          let* cb = go scope b in
          let i = match op with Ast.Add -> Add | Sub -> Sub | Mul -> Mul | Div -> Div in
          Ok (ca @ cb @ [ i ])
      | Ast.Let (x, bound, body) ->
          let ( let* ) = Result.bind in
          let* cbound = go scope bound in
          let* cbody = go (x :: scope) body in
          Ok (cbound @ [ Bind ] @ cbody @ [ Unbind ])
    in
    go [] e

  let to_string = function
    | Push n -> "push " ^ string_of_int n
    | Load i -> "load " ^ string_of_int i
    | Bind -> "bind"
    | Unbind -> "unbind"
    | Neg -> "neg"
    | Add -> "add"
    | Sub -> "sub"
    | Mul -> "mul"
    | Div -> "div"
end

module Vm = struct
  open Compiler

  (* The machine: a stack of values and a stack of bound variables. *)
  let run (code : instr list) : (int, string) result =
    let rec step stack env = function
      | [] -> ( match stack with [ v ] -> Ok v | _ -> Error "malformed program")
      | instr :: rest -> (
          match (instr, stack, env) with
          | Push n, s, _ -> step (n :: s) env rest
          | Load i, s, _ -> step (List.nth env i :: s) env rest
          | Bind, v :: s, _ -> step s (v :: env) rest
          | Unbind, s, _ :: e -> step s e rest
          | Neg, v :: s, _ -> step (-v :: s) env rest
          | Div, 0 :: _ :: _, _ -> Error "division by zero"
          | (Add | Sub | Mul | Div), b :: a :: s, _ ->
              let r = match instr with Add -> a + b | Sub -> a - b | Mul -> a * b | _ -> a / b in
              step (r :: s) env rest
          | _ -> Error "stack underflow")
    in
    step [] [] code
end

(* The pipeline: each stage returns a result, and let* stops at the first error. *)
let ( let* ) = Result.bind

let compile_source source =
  let* tokens = Lexer.tokenize source |> Result.map_error (fun e -> "lexing: " ^ e) in
  let* ast = Parser.parse tokens |> Result.map_error (fun e -> "parsing: " ^ e) in
  Compiler.compile ast |> Result.map_error (fun e -> "compiling: " ^ e)

let evaluate source =
  let* code = compile_source source in
  let* value = Vm.run code |> Result.map_error (fun e -> "running: " ^ e) in
  Ok (List.length code, value)

let () =
  (match compile_source "let x = 5 in x * x - 1" with
  | Ok code -> print_endline (String.concat "; " (List.map Compiler.to_string code))
  | Error e -> print_endline e);
  List.iter
    (fun source ->
      match evaluate source with
      | Ok (size, v) -> Printf.printf "%-42s = %d   (%d instructions)\n" source v size
      | Error e -> Printf.printf "%-42s ! %s\n" source e)
    [
      "1 + 2 * 3";
      "(1 + 2) * 3";
      "let x = 5 in x * x - 1";
      "let a = 10 in let b = a / 4 in a - b * 4";
      "let x = 2 in (let x = 10 in x) + x";
      "-(4 - 10) / 2";
      "7 / (2 - 2)";
      "y + 1";
      "let = 3";
      "2 +";
      "3 # 4";
    ]
text
push 5; bind; load 0; load 0; mul; push 1; sub; unbind
1 + 2 * 3                                  = 7   (5 instructions)
(1 + 2) * 3                                = 9   (5 instructions)
let x = 5 in x * x - 1                     = 24   (8 instructions)
let a = 10 in let b = a / 4 in a - b * 4   = 2   (13 instructions)
let x = 2 in (let x = 10 in x) + x         = 12   (9 instructions)
-(4 - 10) / 2                              = 3   (6 instructions)
7 / (2 - 2)                                ! running: division by zero
y + 1                                      ! compiling: unknown variable y
let = 3                                    ! parsing: expected a name after 'let'
2 +                                        ! parsing: unexpected end of input
3 # 4                                      ! lexing: unexpected character '#' at position 2

The first line shows the compiled code for let x = 5 in x * x - 1: push 5, bind it as the innermost variable, load it twice, multiply, push 1, subtract, and unbind. The table then shows results and errors from each stage, each labelled with the stage that caught it.

Some design decisions worth noticing:

Variants all the way down. Tokens, syntax trees and instructions are each a variant type, and each stage is a function from one to the next, with one case per constructor. Adding an operator such as % means adding a constructor to three types; the compiler's exhaustiveness checks then list every function that must handle it.

A local exception, converted at the boundary. Inside Parser.parse, a syntax error raises the exception Parser.Error from deep in the recursion, which keeps the grammar functions short. But parse itself catches it with match ... with exception and returns a result, so no exception ever escapes the module. That is exactly the division of labour recommended in lesson 6.

Errors found as early as possible. y + 1 fails in the compiler, before any code runs, because the compiler resolves every variable to an environment position (0 is the innermost let). Real compilers do the same: name resolution happens once, at compile time, so the machine never looks names up by string. 7 / (2 - 2) can only fail at run time, in the VM.

Shadowing works without special code. In let x = 2 in (let x = 10 in x) + x, the inner x is position 0 inside the parentheses, and after unbind the outer x is position 0 again; the answer is 12. The scope list in the compiler and the environment list in the VM grow and shrink in step.

One pattern match is the whole machine. Vm.run matches on the instruction and the shapes of both stacks at once, so "stack underflow" is simply the case where no pattern fits. (List.nth env i can only fail if the compiler produced a bad index, which it cannot; a production VM would use an array and check this invariant once.)

Taking it further

Natural next steps, roughly in order of difficulty: add comparison operators and if ... then ... else, which needs jump instructions in the VM; move each module into its own file in a dune library, with .mli files that hide the token and instruction types; add Alcotest tests for each stage and a QCheck property that the VM's answer equals a direct tree-walking evaluator's on random expressions; report error positions by keeping each token's offset; and compile to an array of instructions with a program counter, which is how real bytecode interpreters, including OCaml's own ocamlrun, work.

Common misconceptionsH

Common misconceptions

Misconception

"opam and dune do the same job." opam installs compilers and packages into switches; dune builds your project from the packages that are installed. A typical session uses both: opam install . --deps-only once, then dune build all day.

Misconception

"Real OCaml requires Jane Street's Core." Core is excellent and widely used, but plenty of production OCaml (the compiler itself, dune, MirageOS, Rocq, many libraries) is written against the standard library. Choose per project, and be consistent within one.

Misconception

"OCaml is an old language that no longer changes." It is old (its roots are in the 1970s and 1980s) and careful about compatibility, but it changes steadily: multicore parallelism and effect handlers in 5.0, new syntax for effects in 5.3, and a stream of standard-library additions and tooling improvements in every release.

Misconception

"A compiler is too big a project to write in an afternoon." The project above is a real, if tiny, compiler: lexing, parsing, name resolution, code generation and execution, in about two hundred lines. Production compilers add many more passes and much more care, but the shape (variants in, variants out, one pattern match per pass) is the same, which is why so many of them are written in ML-family languages.

Further reading

Where to go next

Exercises

Exercises

1. Tools, where OCaml is used, and a complete project

Warm-upHuman

In a dune project, the directory lib/ holds parser.ml and eval.ml, and the executable in bin/ should use them as Calc.Parser and Calc.Eval. What goes in lib/dune?

Choose one

2. Tools, where OCaml is used, and a complete project

Warm-upComputational

Using the compiler from the lesson, what value does it print for the program let a = 3 in a * (a + 1)?

3. Tools, where OCaml is used, and a complete project

CoreComputational

Using the compiler from the lesson, how many stack-machine instructions does 2 * (3 + 4) - 5 compile to?

4. Tools, where OCaml is used, and a complete project

CoreComputational

Using the compiler from the lesson, what instruction sequence does let y = 4 in -y compile to? Write it as the program prints it, with instructions separated by ; .

5. Tools, where OCaml is used, and a complete project

CoreComputational

Using the compiler from the lesson, what message does it print for let x = 1 in x + z? Give the message after the !.

6. Tools, where OCaml is used, and a complete project

CoreHuman

The parser raises Parser.Error deep inside its recursive functions but returns a result from parse. Why this design?

Choose one

7. Tools, where OCaml is used, and a complete project

CoreHuman

What characterises an expect test written with ppx_expect?

Choose one

Further reading and resources

  • Dune documentationProjects, libraries, executables, tests, and the many options of dune files.
  • The OCaml PlatformThe official overview of opam, dune, ocamlformat, ocaml-lsp-server, odoc and related tools.
  • OCaml packagesSearch the opam repository, with documentation for each package.
  • AlcotestA lightweight, colourful unit-testing framework.
  • ppx_expectExpect tests, where the expected output lives next to the test and is updated by dune.
  • QCheckProperty-based testing for OCaml.
  • OCaml DiscussThe community forum for questions, announcements and discussion.
  • Real World OCaml · Yaron Minsky and Anil MadhavapeddyThe natural next book, on OCaml in industrial practice.