Show, don't reward
The previous lesson ended with two problems: robot RL needs vast amounts of experience, and writing a reward that cannot be gamed is hard. For many tasks there is a more direct route. Folding a shirt, loading a dishwasher, threading a cable through a clip — a person who cannot write down a reward for any of these can simply do them, with the robot's own arms, while every camera image and joint command is recorded. The robot then learns to map what it sees to what the person did.
This is imitation learning, and its simplest form, behaviour cloning, is just supervised learning. It is old: in 1989 Dean Pomerleau's ALVINN at Carnegie Mellon learned to steer a van along a road from a few minutes of a human driver's steering. What changed after about 2023 is that cheap teleoperation rigs, transformers and diffusion models made it work for dexterous, two-handed manipulation. Projects such as ALOHA (Stanford, 2023) and Diffusion Policy (Columbia and Toyota Research Institute, 2023) showed robots learning fine tasks — putting a battery into a remote control, turning a mug the right way up, spreading sauce on a pizza base — from tens of demonstrations, and the generalist models of the Frontier level are, at heart, behaviour cloning at scale.
The method has two classic failure modes, and this lesson measures both in code. Small mistakes snowball, because the robot drifts into situations the demonstrator never showed it. And when demonstrators do the same task in different ways, a naive learner averages them into something nobody did. The fixes — DAgger, action chunking, diffusion policies — are the core of modern practice.
Behaviour cloning
Definition— Behaviour cloning
Given demonstrations — observations (images, joint angles, forces) paired with the actions an expert took — behaviour cloning fits a policy by supervised learning,
with a loss such as squared error (), absolute error, or the negative log-likelihood of a probabilistic policy.
Everything you know from Learning from data and Training in practice applies: training and held-out splits, overfitting, normalising inputs, choosing a model class. What is new is how the policy is used. A classifier labels a photo and is done. A policy's output moves the robot, which changes the next observation, which the policy then sees. The data you trained on came from the expert's behaviour; the data you meet at test time comes from your own.
Definition— Covariate shift
Let be the distribution of states visited when running policy . Behaviour cloning minimises the expected loss under the expert's state distribution, , but performance depends on the loss under the learner's own distribution . The mismatch between and is covariate shift.
Cloning a path-following robot, and fixing it with DAgger
A small robot drives at 1 m/s and must follow a curved line on the floor, , for 12 m, staying within 0.3 m of it. It observes its position and heading and chooses a turn rate. The expert is a hand-written steering law that aims back at the line. The wheels slip a little, so the heading jitters by about 0.02 rad every step, for the expert and the learner alike.
The program records ten expert demonstrations (2,400 state–action pairs), fits a linear policy by least squares and a one-hidden-layer MLP by Adam, and then runs each policy 50 times. For every policy it reports three numbers: how often it stays in the lane; how far from the line its visited states lie (95th percentile); and its on-policy action error, the RMS difference between its actions and the expert's in the states the learner itself visited. Then it runs DAgger, explained below, and finally plain behaviour cloning with four times as many expert demonstrations, as a control.
import numpy as np
# A robot drives at 1 m/s along the curved path y = 0.6 sin(0.8 x) for 12 m and must stay within
# 0.3 m of it. State (x, y, heading); action = turn rate (rad/s), limited to +-2.
V, DT, STEPS, LANE = 1.0, 0.05, 240, 0.3
def path_y(x):
return 0.6 * np.sin(0.8 * x)
def expert(S):
"""The demonstrator: aim back at the path, then turn towards that heading. Works on many states at once."""
S = np.atleast_2d(S)
x, y, th = S[:, 0], S[:, 1], S[:, 2]
aim = np.arctan(0.48 * np.cos(0.8 * x)) - np.arctan(3.0 * (y - path_y(x)))
return np.clip(4.0 * np.angle(np.exp(1j * (aim - th))), -2.0, 2.0)
def drive(policy, rng):
"""One run with wheel slip jiggling the heading. Returns the visited states and whether it stayed in the lane."""
s = np.array([0.0, rng.normal(0, 0.05), np.arctan(0.48) + rng.normal(0, 0.05)])
states = []
for _ in range(STEPS):
states.append(s)
th = s[2] + float(policy(s)[0]) * DT + rng.normal(0, 0.02)
s = np.array([s[0] + V * np.cos(th) * DT, s[1] + V * np.sin(th) * DT, th])
if abs(s[1] - path_y(s[0])) > LANE:
return np.array(states), False
return np.array(states), True
def features(S):
S = np.atleast_2d(S)
return np.stack([S[:, 0] / 6 - 1, S[:, 1] / 0.6, np.cos(S[:, 2]), np.sin(S[:, 2])], axis=1)
class MLP:
"""One hidden tanh layer, trained on squared error with Adam (see Lumenary CS, Training in practice)."""
def __init__(self, rng, n_in=4, hidden=32):
self.p = [rng.normal(0, n_in ** -0.5, (n_in, hidden)), np.zeros(hidden),
rng.normal(0, hidden ** -0.5, (hidden, 1)), np.zeros(1)]
self.m = [np.zeros_like(q) for q in self.p]; self.v = [np.zeros_like(q) for q in self.p]; self.t = 0
def __call__(self, S):
W1, b1, W2, b2 = self.p
return np.clip((np.tanh(features(S) @ W1 + b1) @ W2 + b2)[:, 0], -2.0, 2.0)
def fit(self, S, A, rng, steps=2000, lr=3e-3, batch=256):
X = features(S)
for _ in range(steps):
i = rng.integers(0, len(X), batch)
W1, b1, W2, b2 = self.p
h = np.tanh(X[i] @ W1 + b1)
err = (h @ W2 + b2)[:, 0] - A[i]
dh = err[:, None] * W2.T * (1 - h ** 2)
grads = [X[i].T @ dh / batch, dh.mean(0), h.T @ err[:, None] / batch, err.mean(keepdims=True)]
self.t += 1
for q, g, m, v in zip(self.p, grads, self.m, self.v):
m[...] = 0.9 * m + 0.1 * g; v[...] = 0.999 * v + 0.001 * g * g
q -= lr * m / (1 - 0.9 ** self.t) / (np.sqrt(v / (1 - 0.999 ** self.t)) + 1e-8)
def report(name, policy, n=50):
runs = [drive(policy, np.random.default_rng(1000 + k)) for k in range(n)]
S = np.concatenate([r[0] for r in runs])
offset = np.abs(S[:, 1] - path_y(S[:, 0]))
err = np.sqrt(np.mean((policy(S) - expert(S)) ** 2))
print(f"{name:22s} stays in lane {np.mean([r[1] for r in runs]):4.0%} 95% of offsets < {np.percentile(offset, 95):.3f} m"
f" action error on its own states {err:.3f}")
rng = np.random.default_rng(0)
demos = [drive(expert, rng)[0] for _ in range(10)] # 10 demonstrations, 2,400 states
S = np.concatenate(demos); A = expert(S)
w = np.linalg.lstsq(np.hstack([features(S), np.ones((len(S), 1))]), A, rcond=None)[0]
linear = lambda s: np.clip(np.hstack([features(s), np.ones((len(np.atleast_2d(s)), 1))]) @ w, -2, 2)
net = MLP(rng); net.fit(S, A, rng)
held_out = np.concatenate([drive(expert, np.random.default_rng(500 + k))[0] for k in range(10)])
for name, pol in [("linear", linear), ("MLP", net)]:
print(f"{name} BC: action error on held-out expert states {np.sqrt(np.mean((pol(held_out) - expert(held_out)) ** 2)):.3f} rad/s")
print()
report("expert", expert)
report("linear BC", linear)
report("MLP BC (10 demos)", net)
# DAgger: drive with the learner, ask the expert what it would have done in the states the LEARNER visited.
for it in range(1, 4):
visited = np.concatenate([drive(net, rng)[0] for _ in range(10)])
S = np.concatenate([S, visited]); A = expert(S) # aggregate, label with the expert
net.fit(S, A, rng, steps=1000)
report(f"DAgger round {it} ({len(S)})", net)
more = np.concatenate([drive(expert, rng)[0] for _ in range(40)]) # control: plain BC with MORE expert demos
big = MLP(np.random.default_rng(0)); big.fit(more, expert(more), rng)
report("MLP BC (40 demos)", big)Both cloned policies look respectable as supervised learners: on held-out expert data their action error is 0.19 rad/s. In closed loop they are poor. The linear policy stays in the lane in 24% of runs and the MLP in 36%, against the expert's 100%. The other two columns show why. The expert keeps 95% of its states within 0.039 m of the line; the cloned policies wander out to 0.24–0.26 m, into states that hardly appear in the demonstrations. There, their action error is 1.25–1.34 rad/s, six to seven times what it was on the expert's data. That is covariate shift, measured.
Why errors compound
Suppose the cloned policy makes a "mistake" (an action the expert would not take) with probability in states the expert visits. Over a task of steps, a first mistake can push the robot off the expert's distribution, where the policy may keep erring for the rest of the episode. Ross and Bagnell made this precise.
Theorem— Compounding errors in behaviour cloning (Ross and Bagnell, 2010)
Let costs per step lie in and let disagree with the expert with probability at most under the expert's state distribution. Then over a horizon of steps the expected cost satisfies , and there are problems on which this quadratic growth is attained. A policy trained on its own state distribution, as DAgger arranges, achieves , under conditions on how cheaply the expert can recover from a mistake (Ross, Gordon and Bagnell, 2011).
Proof
A sketch of the upper bound. Until first disagrees with the expert, the two visit the same states, so at each step the probability that the first mistake happens then is at most . After a first mistake at step , the worst case is that every remaining step costs 1: at most . Summing, the extra cost is at most . The bound is attained, up to a constant, by tasks where one mistake leads to states from which the learner never recovers.
∎
The quadratic is why long tasks are hard for behaviour cloning. Our robot takes 240 steps per run; even a disagreement rate of 0.1% per step allows a worst-case extra cost of steps' worth, against for a learner trained on its own distribution.
DAgger
DAgger (Dataset Aggregation; Ross, Gordon and Bagnell, 2011) attacks the cause directly. Repeat:
- Run the current learned policy and record the states it visits.
- Ask the expert what it would have done in each of those states.
- Add these labelled states to the dataset and retrain on everything.
The robot now learns how to recover from its own mistakes, because its mistakes are in the training data, labelled with the expert's corrections. In the program, one round lifts success from 36% to 98%, and after two rounds the policy stays in the lane every time, with its on-policy error down from 1.25 to 0.27 rad/s — using less data than the 40-demonstration control.
When demonstrators disagree
Five people asked to walk a robot around a pillar will not all pass on the same side. Both choices are correct; their average — straight ahead — is not. A policy trained with squared error predicts the conditional mean of the demonstrated actions, and when the demonstrations are multimodal that mean can be the worst action of all.
The program below makes this concrete. A pillar stands 2 m ahead; 20 demonstrations pass it on the left and 20 on the right, all swerving at the same point. Instead of a trained network it uses the simplest learner that shows the effect: a nearest-neighbour policy that weights demonstrated moments by how close their state is to the robot's (a Gaussian kernel, in the spirit of the VINN method of Pari and colleagues, 2021). The averaging policy outputs the weighted mean action — what squared-error regression converges to. The sampling policies instead pick one similar demonstration moment at random, with those weights, and copy what that demonstrator did next: either one action, re-chosen every step, or a chunk of the next ten actions, executed before choosing again.
import numpy as np
# A pillar of radius 0.45 m stands at (2, 0). The robot moves forward at 1 m/s; the policy chooses its
# sideways speed u (m/s). Twenty demonstrators passed on the left, twenty on the right.
DT, STEPS, PILLAR, H = 0.1, 40, 0.45, 0.2
def demonstrate(side, rng):
y, rows = rng.normal(0, 0.02), []
for k in range(STEPS):
x = k * DT
target = side * 0.75 if 0.4 < x < 2.6 else 0.0 # straight, swerve, pass, come back
u = float(np.clip(3.0 * (target - y), -1.5, 1.5))
rows.append((x, y, u))
y += u * DT + rng.normal(0, 0.01)
return np.array(rows)
rng = np.random.default_rng(0)
demos = [demonstrate(side, rng) for side in [1, -1] * 20]
D = np.concatenate(demos) # columns x, y, u
which = np.repeat(np.arange(len(demos)), STEPS) # which demo each row came from
when = np.tile(np.arange(STEPS), len(demos)) # and at which step
def weights(x, y):
"""How similar each demonstrated state is to (x, y): a Gaussian kernel of width H."""
w = np.exp(-((D[:, 0] - x) ** 2 + (D[:, 1] - y) ** 2) / (2 * H ** 2))
return w / w.sum()
def average_policy(x, y, rng):
return [float(weights(x, y) @ D[:, 2])] # the squared-error answer: a weighted mean
def sampling_policy(x, y, rng, horizon=10):
i = rng.choice(len(D), p=weights(x, y)) # pick ONE similar demonstration moment...
return list(demos[which[i]][when[i]:when[i] + horizon, 2]) # ...and copy what it did next
def drive(policy, rng, chunk):
y, plan, us = rng.normal(0, 0.02), [], []
for k in range(STEPS):
if not plan:
plan = policy(k * DT, y, rng)[:chunk] # replan only when the chunk runs out
u = plan.pop(0)
us.append(u)
y += u * DT + rng.normal(0, 0.01)
if ((k + 1) * DT - 2.0) ** 2 + y ** 2 < PILLAR ** 2:
return us, True
return us, False
test = np.random.default_rng(1)
for name, policy, chunk in [("average (squared error)", average_policy, 1),
("sample every step", sampling_policy, 1),
("sample a 10-step chunk", sampling_policy, 10)]:
runs = [drive(policy, test, chunk) for _ in range(200)]
hits = sum(hit for _, hit in runs)
flips = np.mean([np.sum(np.abs(np.diff(np.sign([u if abs(u) > 0.2 else 0 for u in us]))) == 2) for us, _ in runs])
print(f"{name:24s} hits the pillar {hits:3d}/200 left-right reversals per run {flips:.2f}")The averaging policy hits the pillar in 146 of 200 runs: at the swerve point the left and right demonstrations cancel, and by the time noise has nudged the robot to one side it is too late. Sampling one demonstrator's action every step is much better, 31 collisions, but it dithers: on average 1.79 times per run it commands a firm move one way and immediately a firm move the other, as it alternates between copying left-passers and right-passers. Sampling a ten-step chunk commits to one demonstrator's plan for a second at a time: no collisions, and almost no reversals.
That small experiment contains the two ideas behind the most successful imitation methods of 2023–2026: model the whole distribution of actions, not its mean, and predict sequences of actions, not single ones.
Action chunking with transformers (ACT)
ACT (Tony Zhao and colleagues, 2023) predicts a chunk of future joint targets at once, , from camera images and joint positions, with a transformer. To capture multimodality it is trained as a conditional variational autoencoder: during training an encoder compresses the demonstrated chunk into a latent "style" variable , and the loss is
At test time is set to the prior's mean, 0. Because a new chunk is predicted at every step, each moment is covered by several overlapping chunks; ACT executes their weighted average, with , where is the oldest prediction. This temporal ensembling smooths the motion without the delay of a low-pass filter. With chunks lasting one to two seconds at a 50 Hz control rate, ACT learned tasks such as opening a cup lid and slotting a battery from about ten minutes of demonstrations, with 80–90% success.
Diffusion policies
A diffusion policy (Cheng Chi and colleagues, 2023) represents the full distribution of action sequences given the observation , using the denoising diffusion models that generate images (Ho, Jain and Abbeel, 2020). Choose noise levels , let and . Training corrupts a demonstrated sequence with noise and teaches a network to predict that noise:
To act, start from pure noise and denoise step by step,
then execute the first few actions of and replan (receding horizon, as in model predictive control). Each denoising step only has to make the sequence slightly more plausible, so the sampler can settle into one mode instead of averaging them. Across 12 tasks from four benchmarks, the paper reported an average improvement of 46.9% over the best earlier methods.
The program below runs exactly this sampler on the pillar problem's decision point, where the demonstrated sideways speeds are about and m/s. In place of a trained network it uses the exact optimal denoiser for those 40 demonstrated actions, the posterior mean , which is what is trained to approximate.
import numpy as np
# At the decision point in front of the pillar, 40 demonstrators chose a sideways speed of about
# +1.5 m/s (left) or -1.5 m/s (right). Squared-error regression predicts their average.
rng = np.random.default_rng(0)
demo_actions = np.concatenate([1.5 + 0.05 * rng.standard_normal(20), -1.5 + 0.05 * rng.standard_normal(20)])
print(f"squared-error regression predicts {demo_actions.mean():+.3f} m/s: straight at the pillar\n")
# A diffusion policy learns to undo noise. Forward process: a_k = sqrt(abar_k) a_0 + sqrt(1 - abar_k) eps.
K = 50
beta = np.linspace(1e-3, 0.25, K)
alpha = 1 - beta
abar = np.cumprod(alpha)
def denoise(a_k, k):
"""Best estimate of the clean action given the noisy one, E[a_0 | a_k], for these demonstrations.
A real diffusion policy trains a network to approximate this (as a noise prediction)."""
logw = -(a_k[:, None] - np.sqrt(abar[k]) * demo_actions[None, :]) ** 2 / (2 * (1 - abar[k]))
w = np.exp(logw - logw.max(axis=1, keepdims=True))
return (w / w.sum(axis=1, keepdims=True)) @ demo_actions
print(f"noise left after {K} steps: sqrt(1 - abar_K) = {np.sqrt(1 - abar[-1]):.3f}")
print(f"denoiser at full noise, input 0.3: {denoise(np.array([0.3]), K - 1)[0]:+.3f} (almost the plain average)")
# Reverse process (DDPM): start from pure noise and denoise step by step.
a = rng.standard_normal(2000)
for k in range(K - 1, -1, -1):
a0_hat = denoise(a, k)
eps_hat = (a - np.sqrt(abar[k]) * a0_hat) / np.sqrt(1 - abar[k]) # implied noise prediction
a = (a - beta[k] / np.sqrt(1 - abar[k]) * eps_hat) / np.sqrt(alpha[k])
if k > 0:
a += np.sqrt(beta[k]) * rng.standard_normal(a.size)
counts, edges = np.histogram(a, bins=9, range=(-2.25, 2.25))
print("\n2000 actions sampled by the diffusion policy:")
for c, lo in zip(counts, edges[:-1]):
print(f" {lo:+.2f} to {lo + 0.5:+.2f} m/s | {'#' * (c // 40)} {c}")
print(f"passes left: {np.mean(a > 0):.1%} heads straight on (|a| < 0.5): {np.mean(np.abs(a) < 0.5):.1%}")The regression answer is m/s, straight at the pillar. Of 2,000 diffusion samples, 49.3% pass left, the rest right, and none goes straight on. Look also at the second line: at the highest noise level the denoiser returns almost exactly the plain average. A one-step regressor is, in effect, a diffusion model that stops after its first, blurriest step. The iterations are what let the sample commit.
Note
With the exact denoiser of a finite dataset, a diffusion model can only reproduce the demonstrated actions; a trained network instead smooths between them, which is what lets it generalise to new observations. Recent policies often use flow matching, a close cousin that learns a velocity field carrying noise to data in fewer, deterministic steps, which matters when a policy must act 50 times a second. Other ways to represent multimodal actions include discretising actions into tokens and predicting them with a transformer (Behavior Transformers, 2022; and the vision-language-action models of the Frontier level) and energy-based policies (Implicit BC, 2021).
Where demonstrations come from
A behaviour-cloning policy is only as good as its demonstrations, and how they are collected shapes everything: what the robot can see, how fast and smoothly it moves, how much data a day yields. The main methods, as of 2026:
- Kinesthetic teaching. Switch the arm into a compliant mode and guide it by hand. Natural for single-arm motions, but your hands appear in the camera images, and two-handed or fast tasks are awkward.
- Leader–follower ("puppet") rigs. The operator moves a pair of small, passive leader arms whose joints match the robot's; the follower arms copy the joint angles in real time. ALOHA (Stanford, 2023) built a bimanual rig of this kind on a budget of about USD 20,000; Mobile ALOHA (2024) put it on a wheeled base for household tasks, and ALOHA 2 (Google DeepMind, 2024) made it sturdier and published the design. GELLO (UC Berkeley, 2023) builds leader arms from 3D-printed parts and off-the-shelf servo motors for industrial arms such as Franka, UR5 and xArm.
- Virtual reality and hand tracking. A headset streams the robot's camera view to the operator and tracks their hands, which are retargeted to the robot's arms and hands — the usual way to drive humanoids with dexterous hands (for example Open-TeleVision, UC San Diego and MIT, 2024).
- Handheld grippers. The Universal Manipulation Interface (Stanford and Columbia, 2024) records demonstrations with a hand-held gripper and a wrist camera, with no robot present at all, so data can be collected in any kitchen or shop; the policy later runs on a robot carrying the same gripper and camera.
- Space mice, game pads, phones. Cheap and universal, but it is hard to control six or more degrees of freedom smoothly with them, and the resulting demonstrations are slower and jerkier.
Experienced teams treat demonstration collection as a craft. Operators are trained and given clear task definitions, because inconsistent strategies create exactly the multimodality you saw above, and long pauses teach the robot to hesitate. Latency between leader and follower must be low, or operators move unnaturally slowly. Most rigs give no force feedback, so the data contains little about how hard to push — one reason contact-rich tasks remain difficult (Dexterous manipulation). The robomimic study found that the same algorithms did markedly worse on demonstrations pooled from operators of mixed skill than on demonstrations from proficient operators. And a cloned policy inherits its teachers' habits: if they never let go of a cup near the table's edge, neither will it, which is sometimes a safety feature and sometimes a limitation.
The open-source LeRobot project (Hugging Face) packages ACT, diffusion policies and others in PyTorch, with recorded datasets and designs for low-cost arms that can be built for a few hundred US dollars; small arms like these, moving slowly and with low torque, are a safe way to try imitation learning on a desk. The last lesson of this level asks what happens when demonstrations are collected not in tens but in millions.
Common misconceptions
Misconception
"Low validation loss means the policy works." Validation loss is measured on the expert's states; the policy lives on its own. In the program, a held-out error of 0.19 rad/s went with a 36% success rate. Imitation policies must be evaluated by running them.
Misconception
"More demonstrations of the same kind will fix compounding errors." They help, but slowly, because they keep sampling the expert's narrow distribution. Data that shows recoveries — from DAgger, interventions or deliberately perturbed demonstrations — is worth far more per sample.
Misconception
"Averaging demonstrations gives a safe compromise." Between two good ways round an obstacle lies the obstacle. Squared-error regression is only safe when demonstrations are unimodal for every observation.
Misconception
"Imitation learning cannot exceed the demonstrator." Pure behaviour cloning aims at the demonstrator's level, but it can be faster (once the operator's hesitations are filtered out), and imitation is often the first stage: reinforcement learning or human corrections then improve the cloned policy beyond its teachers.
Exercises
1. Imitation learning
Warm-upHumanA behaviour-cloned driving policy has a very low error on held-out demonstrations, yet it leaves the lane in most test runs. What is the most likely explanation?
2. Imitation learning
Warm-upHumanA cloned policy disagrees with the expert with probability per step on the expert's states. For a task of steps with costs in , what is the worst-case extra cost allowed by the behaviour-cloning bound?
3. Imitation learning
CoreComputationalWrite fit_linear_policy(S, A) that fits a linear behaviour-cloning policy by least squares. S has shape (N, d) (states), A has shape (N, m) (actions). Return W of shape (d + 1, m), the last row being the bias. Also write act(W, s) returning the action for one state s of shape (d,).
4. Imitation learning
CoreComputationalACT predicts a chunk of actions at every step and executes, at time , a weighted average of all predictions for . Write ensemble(chunks, t, m). chunks is a dict mapping the time t0 at which a chunk was predicted to a NumPy array of shape (k, action_dim), whose row j is the action for time t0 + j. Average all predictions for time t, oldest first, with weights where is the oldest prediction that covers t. Return the averaged action.
5. Imitation learning
CoreHumanDemonstrators pass an obstacle on the left or on the right, half and half. Which policy representations can, in principle, reproduce both behaviours rather than a blend? Select all that apply.
6. Imitation learning
CoreHumanWhat does DAgger require that plain behaviour cloning does not?
7. Imitation learning
StretchHumanAn ACT policy runs at 50 Hz, predicts a chunk of actions at every control step, and uses temporal ensembling. Once it has been running for a few seconds, how many separate predictions are averaged to produce each executed action?
Further reading and resources
- ALVINN: An Autonomous Land Vehicle in a Neural Network · Dean A. Pomerleau · 1989Advances in Neural Information Processing Systems 1. A small network at Carnegie Mellon that learned to steer a van from a human driver's example — behaviour cloning, decades early.
- End to End Learning for Self-Driving Cars · Mariusz Bojarski et al. (NVIDIA) · 2016Behaviour cloning for lane keeping, with side cameras used to synthesise recovery examples.
- Efficient Reductions for Imitation Learning · Stéphane Ross and J. Andrew Bagnell · 2010The quadratic error bound for behaviour cloning.
- A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning · Stéphane Ross, Geoffrey J. Gordon and J. Andrew Bagnell · 2011DAgger.
- DART: Noise Injection for Robust Imitation Learning · Michael Laskey et al. · 2017
- HG-DAgger: Interactive Imitation Learning with Human Experts · Michael Kelly et al. · 2018
- What Matters in Learning from Offline Human Demonstrations for Robot Manipulation · Ajay Mandlekar et al. · 2021The robomimic study; a careful look at how demonstration quality and policy design matter.
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware · Tony Z. Zhao, Vikash Kumar, Sergey Levine and Chelsea Finn · 2023ALOHA and Action Chunking with Transformers (ACT).
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion · Cheng Chi et al. · 2023
- Denoising Diffusion Probabilistic Models · Jonathan Ho, Ajay Jain and Pieter Abbeel · 2020
- Implicit Behavioral Cloning · Pete Florence et al. · 2021
- Behavior Transformers: Cloning k modes with one stone · Nur Muhammad Mahi Shafiullah et al. · 2022
- The Surprising Effectiveness of Representation Learning for Visual Imitation · Jyothish Pari et al. · 2021A nearest-neighbour imitation policy (VINN), the idea behind this lesson's second program.
- Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation · Zipeng Fu, Tony Z. Zhao and Chelsea Finn · 2024
- ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation · ALOHA 2 Team · 2024
- GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators · Philipp Wu et al. · 2023
- Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots · Cheng Chi et al. · 2024
- Open-TeleVision: Teleoperation with Immersive Active Visual Feedback · Xuxin Cheng et al. · 2024
- Generative Adversarial Imitation Learning · Jonathan Ho and Stefano Ermon · 2016
- LeRobot · Hugging FaceOpen-source PyTorch implementations of ACT, diffusion policies and others, with datasets and low-cost arm designs.