Skip to content
LumenaryAcademy
Learning 75 min

Crossing the gap from simulation to reality

Policies trained in simulation meet a real robot that is heavier, slower, stickier and later than the model. System identification, domain randomisation and teacher–student training — and the robots that crossed the gap.

Code tested · 5AI-draftedAwaiting review by a roboticist

Your guides: Chloe · The Builder, Margaret · The Experimenter, Gideon · The Safety Engineer

MotivationH

Brilliant in simulation

A team trains a walking policy for a quadruped in simulation. After a few hours on a GPU it runs, turns, climbs steps and recovers from shoves, thousands of times without a fall. They copy the network onto the real robot, hold it up in a safety harness, lower it to the floor — and it shudders, stamps and topples. Nothing in the network is broken. The simulator was wrong in small ways, and the policy had learned to depend on them.

The Simulation lesson listed the usual culprits: contact and friction models, actuator dynamics and latency, unmodelled flexibility, sensor noise, and, for cameras, how a rendered image differs from a real one. The reinforcement-learning lesson showed why they bite so hard: an optimiser rewarded for performance will find any modelling error that helps and lean on it, just as it found the farmable reward in the corridor.

This lesson is about getting a policy across that gap. There are three complementary tools, and every successful system uses some mix of them:

  1. System identification: measure the real robot and make the simulator more like it.
  2. Domain randomisation: train on many deliberately different simulators, so that the real robot is just one more variation.
  3. Teacher–student training and adaptation: let the policy work out, from its own recent experience, which variation it is in.

You will implement each on a small problem, then see how they were combined in the results that made sim-to-real famous.

DefinitionsH

The reality gap, precisely

Write ξ\xi for the physical parameters a simulator needs: masses, inertias, friction coefficients, motor constants, delays, sensor noise levels, terrain properties. A simulator with parameters ξ\xi gives an expected return Jξ(π)J_\xi(\pi) for policy π\pi. The real robot behaves like some unknown ξ⋆\xi^\star — or, more honestly, like no value of ξ\xi at all, since every simulator leaves something out.

Definition— Reality gap

For a policy π\pi trained in a simulator with parameters ξ^\hat\xi, the reality gap is the difference Jξ^(π)−Jreal(π)J_{\hat\xi}(\pi) - J_{\text{real}}(\pi) between its performance in simulation and on the real system. A policy transfers zero-shot if it performs acceptably on the real system with no further training there.

The three tools attack different terms. System identification chooses ξ^\hat\xi to make the simulator's behaviour match logged real data,

ξ^=arg⁡min⁡ξ∑k∥ykreal−yksim(ξ)∥2,\hat\xi = \arg\min_\xi \sum_{k} \big\| y_k^{\text{real}} - y_k^{\text{sim}}(\xi) \big\|^2 ,

where yky_k are measured signals (joint velocities, positions, forces) under the same commands. Domain randomisation replaces the single simulator by a distribution p(ξ)p(\xi) and trains

πDR=arg⁡max⁡π Eξ∼p(ξ)[Jξ(π)],\pi_{\text{DR}} = \arg\max_\pi \ \mathbb{E}_{\xi \sim p(\xi)} \big[ J_\xi(\pi) \big],

hoping that the real system looks like a typical draw. Adaptation lets the policy depend on its history hth_t of observations and actions, at=π(ot,ht)a_t = \pi(o_t, h_t), so that it can behave differently in different ξ\xi: implicitly identifying the system while it acts.

ComputationC

System identification: fitting a joint to its log

Most of the reality gap of a legged robot or an arm sits in its actuators: the motor, the gearbox and the drive electronics. Catalogue values for inertia and friction are a starting point, but gearbox friction depends on lubrication, temperature and wear, and the reflected inertia of the rotor through the gearbox is easy to get wrong.

Consider one joint under torque (current) control. A simple model has rotor-plus-link inertia JJ, viscous friction bb and Coulomb friction τc\tau_c:

Jω˙=τ−b ω−τcsign⁡(ω).J \dot\omega = \tau - b\,\omega - \tau_c \operatorname{sign}(\omega).

Sampled every Δt\Delta t, this is linear in the unknowns (1/J, b/J, τc/J)(1/J,\ b/J,\ \tau_c/J):

ωk+1−ωkΔt=1J τk−bJ ωk−τcJsign⁡(ωk),\frac{\omega_{k+1} - \omega_k}{\Delta t} = \frac{1}{J}\,\tau_k - \frac{b}{J}\,\omega_k - \frac{\tau_c}{J}\operatorname{sign}(\omega_k),

so logging τk\tau_k and ωk\omega_k and solving one least-squares problem identifies all three. The program plays the role of the real joint (with a cogging ripple and encoder noise that the model does not include), excites it with a chirp, a sine wave sweeping from 0.2 Hz to about 3 Hz, fits the model, and then compares the catalogue model and the identified one on a different input, a sequence of torque steps.

python
import numpy as np

# The real joint, which we can only log: inertia J, viscous friction b, Coulomb friction tc,
# plus a small cogging ripple that no one put in the model.
J_TRUE, B_TRUE, TC_TRUE, COG = 0.012, 0.03, 0.15, 0.02
DT = 0.005                                                 # 200 Hz logging

def simulate(tau, J, b, tc, cog=0.0, rng=None, noise=0.0):
    th, w, out = 0.0, 0.0, []
    for t in tau:
        out.append(w + (rng.normal(0, noise) if rng is not None else 0.0))   # what the encoder reports
        drive = t - cog * np.sin(8 * th)
        if abs(w) < 1e-3 and abs(drive) <= tc:            # stuck: static friction holds it
            w = 0.0
        else:
            w = w + DT * (drive - b * w - tc * np.sign(w if abs(w) >= 1e-3 else drive)) / J
        th += w * DT
    return np.array(out)

rng = np.random.default_rng(0)
time = np.arange(0, 10, DT)
chirp = 0.5 * np.sin(2 * np.pi * (0.2 + 0.14 * time) * time)          # 0.2 Hz rising to about 3 Hz
w_log = simulate(chirp, J_TRUE, B_TRUE, TC_TRUE, COG, rng, noise=0.05)

# Equation error: (w[k+1] - w[k]) / dt = (1/J) tau - (b/J) w - (tc/J) sign(w), linear in (1/J, b/J, tc/J).
moving = np.abs(w_log[:-1]) > 0.2
Phi = np.stack([chirp[:-1], -w_log[:-1], -np.sign(w_log[:-1])], axis=1)[moving]
acc = (np.diff(w_log) / DT)[moving]
c = np.linalg.lstsq(Phi, acc, rcond=None)[0]
J_hat, b_hat, tc_hat = 1 / c[0], c[1] / c[0], c[2] / c[0]
print(f"identified: J = {J_hat:.4f} kg m^2 (true {J_TRUE}), b = {b_hat:.4f} N m s (true {B_TRUE}), "
      f"tc = {tc_hat:.3f} N m (true {TC_TRUE})")

# Validate on an input the fit never saw: a sequence of torque steps.
steps = np.repeat([0.4, 0.0, -0.3, 0.25, -0.4, 0.0], 400)
w_real = simulate(steps, J_TRUE, B_TRUE, TC_TRUE, COG)
for name, (J, b, tc) in [("datasheet model", (0.009, 0.015, 0.05)), ("identified model", (J_hat, b_hat, tc_hat))]:
    pred = simulate(steps, J, b, tc)
    print(f"{name:17s}: RMS speed error on held-out steps = {np.sqrt(np.mean((pred - w_real) ** 2)):.2f} rad/s"
          f"  (real speeds reach {np.abs(w_real).max():.1f} rad/s)")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

From ten seconds of data the fit recovers J=0.0118J = 0.0118 (true 0.012), b=0.0307b = 0.0307 (true 0.03) and τc=0.150\tau_c = 0.150 (true 0.15). On the held-out torque steps, the identified model predicts the joint's speed to within 0.12 rad/s RMS; the catalogue model, with friction a factor of two to three too low, is off by 8.19 rad/s — as large as the speeds themselves (up to 8.3 rad/s), because it predicts the joint spinning nearly three times too fast.

Four habits of practice are visible here. Excite the system: a chirp exercises a range of speeds and accelerations, while a constant torque would teach you nothing about inertia. Choose which data to believe: the fit skips every sample slower than 0.2 rad/s, where sign⁡(ω)\operatorname{sign}(\omega) flips with the noise and static friction holds the joint in a way the equation does not describe. Validate on different data: a model that fits its own training log can still be wrong. And know what you left out: the cogging ripple is not in the model, which is why the identified model is good but not perfect, and in a real joint backlash, temperature-dependent friction and current-loop dynamics would add more.

When the physics is too messy for a few parameters, learn it. Jemin Hwangbo and colleagues at ETH Zurich (2019) trained a small neural network on data logged from ANYmal's series-elastic actuators to predict torque from the recent history of position errors and velocities, and put that actuator network inside the simulator; policies trained with it transferred to the real robot where those trained with an idealised motor did not. Learned residual models, which correct a physics simulator using real data, follow the same idea.

ComputationC

Domain randomisation

Identification narrows the gap but never closes it, and some parameters (the friction of every floor a robot will ever walk on, the mass of every box it will lift) cannot be measured in advance. Domain randomisation accepts the uncertainty and trains for all of it. Josh Tobin and colleagues (OpenAI, 2017) randomised the visual appearance of simulated scenes — textures, lighting, camera position — until an object detector trained only on renders worked on real camera images. Xue Bin Peng and colleagues (2018) randomised the dynamics — masses, friction, damping, latency — and transferred a pushing policy to a real arm.

Here is the idea at small scale. A carriage on a rail must move 0.2 m and stop, under a PD policy u=kp(0.2−x)−kdvu = k_p(0.2 - x) - k_d v whose two gains are the "network" to be trained. Commands reach the motor 30 ms late, and the force is limited to 50 N. The engineers' model says the carriage has mass 1.0 kg and viscous friction 20 N s/m — the CAD model includes a heavily damped linear bearing. The real carriage, unknown to them, is 0.6 kg with friction 1.0 N s/m: the bearing was replaced with a smoother one. Both gains are trained with the cross-entropy method, a simple derivative-free optimiser: sample gains, score each in simulation, keep the best fifth, refit. One run trains on the nominal model only; the other on 16 robots whose mass is drawn uniformly from 0.5–2.0 kg and friction from 0.5–20 N s/m.

python
import numpy as np

# A carriage on a rail must move 0.2 m and stop. The policy is u = kp (0.2 - x) - kd v, computed every
# 10 ms and applied 30 ms late (sensing, computation and the bus), with a 50 N force limit.
# Unknown to the policy: the moving mass m (kg) and the viscous friction b (N s/m).
DT, DELAY, STEPS, UMAX, TARGET = 0.01, 3, 200, 50.0, 0.2

def simulate(kp, kd, m, b):
    """Vectorised over every argument. Returns the cost (time-weighted |error| plus a little effort)
    and the largest position reached, for each (gains, robot) pair."""
    kp, kd, m, b = np.broadcast_arrays(*map(np.asarray, (kp, kd, m, b)))
    x, v, cost, peak = (np.zeros(kp.shape) for _ in range(4))
    queue = [np.zeros(kp.shape)] * DELAY
    for _ in range(STEPS):
        queue.append(np.clip(kp * (TARGET - x) - kd * v, -UMAX, UMAX))
        u = queue.pop(0)                                   # the command issued DELAY steps ago
        v = v + DT * (u - b * v) / m
        x = x + DT * v
        cost += DT * (np.abs(TARGET - x) + 0.001 * np.abs(u))
        peak = np.maximum(peak, x)
    return cost, peak

def train(sample_robots, rng, iters=25, pop=60):
    """Cross-entropy method: sample gains, score each on the training robots, refit to the best fifth."""
    mu, sd = np.array([100.0, 10.0]), np.array([60.0, 8.0])
    m, b = sample_robots(rng, 16)                          # the 16 simulated robots used for training
    for _ in range(iters):
        G = np.abs(mu + sd * rng.standard_normal((pop, 2)))
        score = simulate(G[:, :1], G[:, 1:], m[None, :], b[None, :])[0].mean(axis=1)
        elite = G[np.argsort(score)[: pop // 5]]
        mu, sd = elite.mean(axis=0), elite.std(axis=0) + 0.5
    return mu

nominal = lambda rng, n: (np.full(n, 1.0), np.full(n, 20.0))          # the CAD model: 1 kg, heavy damping
randomised = lambda rng, n: (rng.uniform(0.5, 2.0, n), rng.uniform(0.5, 20.0, n))

rng = np.random.default_rng(0)
g_nom, g_dr = train(nominal, rng), train(randomised, rng)
print(f"trained on the nominal model:  kp = {g_nom[0]:6.1f}, kd = {g_nom[1]:5.1f}")
print(f"trained with randomisation:    kp = {g_dr[0]:6.1f}, kd = {g_dr[1]:5.1f}\n")
for name, (m, b) in [("nominal model     m=1.0 b=20", (1.0, 20.0)), ("the real robot    m=0.6 b=1.0", (0.6, 1.0)),
                     ("outside the range m=0.3 b=0.2", (0.3, 0.2))]:
    (c1, p1), (c2, p2) = simulate(*g_nom, m, b), simulate(*g_dr, m, b)
    print(f"{name}: cost {float(c1):.3f} vs {float(c2):.3f}   overshoot {100 * max(float(p1) / TARGET - 1, 0):5.1f}% "
          f"vs {100 * max(float(p2) / TARGET - 1, 0):5.1f}%")
m_test, b_test = randomised(np.random.default_rng(5), 500)                # 500 robots never seen in training
for name, g in [("nominal-trained", g_nom), ("randomised", g_dr)]:
    c = simulate(g[0], g[1], m_test, b_test)[0]
    print(f"{name:15s} on 500 unseen robots: mean cost {c.mean():.3f}, worst {c.max():.3f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The policy trained on the nominal model learned to lean on the simulated bearing: its derivative gain is only 8.9, because 20 N s/m of friction did most of the braking for it. In simulation it is excellent (cost 0.026). On the real carriage it overshoots the target by 92% and rings, at almost ten times its cost in simulation. The randomised policy doubles its derivative gain to 18.0, accepts a slightly slower response on the nominal model (cost 0.038 instead of 0.026 — the price of robustness), and on the real carriage overshoots by 20% with a cost of 0.049. Across 500 robots it never saw, its worst case (0.162) is less than half the nominal policy's (0.371).

nominal modelreal carriageoutside the range00.51.01.52.0mass m (kg)0.1110friction b (N s/m)
The space of simulated robots. The nominal model is a single point; domain randomisation trains on the shaded box (dots: the 16 robots actually sampled). The real carriage lies inside the box, though at lower friction than any training robot; the third test robot lies outside it, where neither policy is safe. Friction is on a logarithmic scale.

The third test is a warning. A carriage of 0.3 kg with almost no friction lies outside the randomised range, and both policies overshoot badly (323% and 80%). Randomisation buys robustness over the range you randomise, and a little beyond it — note that none of the 16 training robots had friction as low as the real carriage — but not everywhere. Choosing the ranges is the real engineering: too narrow and reality falls outside; too wide and the policy becomes timid, giving up performance on every robot to survive implausible ones. OpenAI's automatic domain randomisation (2019) widened each range gradually, only as far as the policy could still succeed, while Chebotar and colleagues (2019) went the other way, using a few real trials to adjust the randomisation distribution towards reality.

ComputationC

Teacher, student and privileged information

A randomised policy that cannot tell which robot it is on must use one compromise behaviour for all of them. A better policy would notice: the carriage accelerates more than expected, so it is light; slow down. Two ideas make this practical.

Privileged information. In simulation, everything is known: the true mass and friction, the exact terrain height under each foot, contact forces, the positions of objects behind the camera. A teacher policy given this privileged information is much easier to train with RL, because its problem is fully observed.

Distillation into a student. A student policy receives only what the real robot will have — proprioception, a camera, the recent history of its own commands and sensor readings — and is trained, still in simulation, to reproduce the teacher. The training is the imitation learning of the previous lesson, and because the teacher can be queried in any simulated state, DAgger works perfectly: run the student, label its states with the teacher's actions, retrain. Joonho Lee and colleagues used this to take ANYmal over rubble, mud and streams (2020); RMA (Ashish Kumar and colleagues, UC Berkeley and Carnegie Mellon, 2021) trained an adaptation module to estimate, from the last half-second of joint states and actions, a compact latent code of the environment that the teacher had been given directly, and ran a Unitree A1 on sand, oily surfaces and hiking trails.

simulation onlyprivileged statetrue m, b, terrainteachertrained by RLactionsensor historyjoints, IMU, past actionsadaptationmodulestudentactionmatchtheteacherdeployed on the real robot: the bottom row only
Teacher–student training. In simulation, the teacher is trained with RL using privileged information the real robot will never have. The student sees only onboard sensors and its recent history, and is trained to match the teacher (or the teacher's latent description of the environment). Only the student goes to the real robot.

The program below builds a minimal version on the carriage. The teacher is told mm and bb and chooses its gains to give every robot the same closed-loop behaviour: matching ms2+(b+kd)s+kpm s^2 + (b + k_d) s + k_p to m(s2+2ζωs+ω2)m(s^2 + 2\zeta\omega s + \omega^2) gives kp=mω2k_p = m\omega^2 and kd=2ζωm−bk_d = 2\zeta\omega m - b. The student is not told; its adaptation module looks at the last 15 commands and velocities it has seen, fits u=ma+bvu = m a + b v to them by least squares, and plugs its estimates into the teacher's formula. The robust policy is the randomised one from above.

python
import numpy as np

# The same carriage as above: move 0.2 m and stop, 30 ms delay, 50 N limit, unknown mass and friction.
DT, DELAY, STEPS, UMAX, TARGET = 0.01, 3, 200, 50.0, 0.2
W, ZETA = 14.0, 0.8                        # desired closed loop: natural frequency 14 rad/s, damping 0.8
DR_GAINS = (210.9, 18.0)                   # the robust gains found by domain randomisation above

def teacher_gains(m, b):
    """The teacher is told m and b (privileged information, available only in simulation) and places
    the closed-loop poles exactly: m s^2 + (b + kd) s + kp = m (s^2 + 2 zeta W s + W^2)."""
    return m * W ** 2, max(2 * ZETA * W * m - b, 0.0)

def run(m, b, who, window=15):
    x = v = cost = 0.0
    queue = [0.0] * DELAY
    us, vs = [], []
    m_hat, b_hat = 1.0, 5.0                # the student's guess before it has seen anything
    for k in range(STEPS):
        if who == "teacher":
            kp, kd = teacher_gains(m, b)
        elif who == "student":
            if len(vs) > window:           # adaptation module: estimate (m, b) from recent history only
                a = np.diff(vs[-window - 1:]) / DT                    # accelerations it caused
                vv, uu = np.array(vs[-window - 1:-1]), np.array(us[-window - 1:-1])
                # least squares for u = m a + b v, via the 2x2 normal equations
                M = np.array([[a @ a, a @ vv], [a @ vv, vv @ vv]])
                if np.linalg.det(M) > 1e-6 * np.trace(M) ** 2:       # only when the data is informative
                    m_hat, b_hat = np.linalg.solve(M, [a @ uu, vv @ uu])
                    m_hat, b_hat = max(m_hat, 0.2), max(b_hat, 0.0)
            kp, kd = teacher_gains(m_hat, b_hat)
        else:
            kp, kd = DR_GAINS
        queue.append(float(np.clip(kp * (TARGET - x) - kd * v, -UMAX, UMAX)))
        u = queue.pop(0)
        us.append(u); vs.append(v)
        v += DT * (u - b * v) / m
        x += DT * v
        cost += DT * (abs(TARGET - x) + 0.001 * abs(u))
    return cost, (m_hat, b_hat)

print("on the real robot (m = 0.6 kg, b = 1.0 N s/m):")
for who in ["teacher", "student", "robust"]:
    cost, est = run(0.6, 1.0, who)
    extra = f"   student's final estimate m = {est[0]:.2f}, b = {est[1]:.2f}" if who == "student" else ""
    print(f"  {who:8s} cost {cost:.3f}{extra}")

rng = np.random.default_rng(5)
robots = list(zip(rng.uniform(0.5, 2.0, 200), rng.uniform(0.5, 20.0, 200)))
print("\non 200 random robots from the training range:")
for who in ["teacher", "student", "robust"]:
    c = np.array([run(m, b, who)[0] for m, b in robots])
    print(f"  {who:8s} mean cost {c.mean():.3f}, worst {c.max():.3f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

On the real carriage the student's estimates converge to m=0.60m = 0.60, b=1.00b = 1.00, and its cost (0.029) is close to the teacher's (0.024) and well below the robust policy's (0.049). Over 200 random robots the student's worst case is 0.038 against the robust policy's 0.158: knowing which robot you are on beats being merely tolerable on all of them.

ExplanationH

The robots that crossed the gap

A few results, chosen because their papers are open and their ideas recur, show how the tools combine on real machines.

Legged locomotion (ETH Zurich and others, from 2019). ANYmal's learned controllers began with Hwangbo's actuator network plus dynamics randomisation (2019); Lee's teacher–student policies crossed rough natural terrain blind (2020); Miki and colleagues (2022) added perception, fusing a height map built from depth cameras or lidar with proprioception through a learned module that decides how far to trust the map — enough to finish an alpine hiking route in Switzerland. RMA showed rapid adaptation on a low-cost commercial quadruped from Unitree. As described in Legged locomotion, RL policies trained this way now run on commercial quadrupeds, and on humanoids: Ilija Radosavovic and colleagues (UC Berkeley, 2023) trained a transformer policy for Agility Robotics' Digit entirely in simulation, with heavy randomisation, and deployed it zero-shot outdoors.

In-hand manipulation (OpenAI, 2018–2019). A Shadow Dexterous Hand learned to reorient a block to commanded faces, using a policy trained entirely in a randomised simulator, with vision networks trained on randomised renders to estimate the block's pose. The follow-up solved a Rubik's Cube one-handed, using automatic domain randomisation and a recurrent policy whose memory let it adapt to perturbations it had never seen, including a glove on the hand and objects poking it. NVIDIA's DeXtreme (2022) reproduced cube reorientation on the cheaper Allegro hand with GPU simulation, and Tao Chen and colleagues at MIT (2022) reoriented previously unseen object shapes with a single depth camera, even with the hand facing downwards.

Drone racing (University of Zurich, 2023). The Swift system's policy was trained in simulation with RL, but the team did not trust the simulator blindly: they measured the gap on the real drone and fitted data-driven residual models of the perception and dynamics errors, which were added to the simulator for training. The result raced three human champions and won most of its races against them.

Note

What these projects have in common is less glamorous than the videos: weeks of careful actuator and latency measurement, randomisation ranges chosen and re-chosen, a second simulator used as a sanity check, and a staged route to hardware — first a robot on a stand, then in a harness or gantry that catches falls, then free. Whenever you test a learned policy for the first time, use the same staging on a small, slow robot, with a person at the emergency stop; Embedded control and safety describes the protective layers underneath.

As of 2026 the tooling has become standard. GPU-parallel simulators — NVIDIA's Isaac Lab, Google DeepMind's MuJoCo with its MJX and Playground frameworks, the open-source Genesis project and others — train locomotion policies for quadrupeds and humanoids in hours, and ship with randomisation and teacher–student recipes. The open problems are elsewhere: contact-rich manipulation of soft, articulated or unfamiliar objects, where simulators are least trustworthy; vision, where rendered and real images still differ in subtle ways; and evaluation, since you cannot know a policy has crossed the gap until you have tested it on the real robot many times (next lesson). Building simulations from real scans, so-called real-to-sim, is an active response to all three.

Common misconceptionsH

Common misconceptions

Misconception

"A more accurate simulator makes randomisation unnecessary." Identification shrinks the gap but cannot remove it, and many parameters vary in deployment (floors, payloads, wear, temperature). Good systems identify what can be measured and randomise what cannot.

Misconception

"Randomise everything as widely as possible, to be safe." Wider ranges cost performance on every robot, can make the task impossible to learn, and still offer no guarantee outside the ranges. The carriage policy paid 46% extra cost on the nominal model for its robustness, and both policies failed on the robot outside the range.

Misconception

"If it works in simulation across thousands of randomised robots, it will work on the real one." Only if reality is inside what was randomised and the simulator's structure is right. Unmodelled effects — a flexible link, a sensor that saturates, cable drag — are not covered by randomising the parameters of a model that lacks them.

Misconception

"The student is just a smaller copy of the teacher." It solves a harder problem: acting well without the privileged information. Its history-processing module is doing implicit system identification, and a student can never be better than what its observations allow it to infer.

Exercises

Exercises

1. Crossing the gap from simulation to reality

Warm-upHuman

A warehouse robot's wheel motors can be tested on a bench, but it will drive on floors whose friction varies from building to building and from day to day. Which combination is most appropriate before training its controller in simulation?

Choose one

2. Crossing the gap from simulation to reality

CoreComputational

Write identify(tau, w, dt) that fits the joint model Jω˙=τ−bω−τcsign⁡(ω)J\dot\omega = \tau - b\omega - \tau_c \operatorname{sign}(\omega) to logged torques tau and velocities w (NumPy arrays of equal length, sampled every dt seconds) and returns (J, b, tc). Use the equation-error least-squares method from the lesson, and ignore samples where ∣ω∣≤0.2|\omega| \le 0.2 rad/s.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

3. Crossing the gap from simulation to reality

Warm-upHuman

The lesson's teacher chooses PD gains so that ms2+(b+kd)s+kp=m(s2+2ζωs+ω2)m s^2 + (b + k_d) s + k_p = m(s^2 + 2\zeta\omega s + \omega^2). For a carriage with m=2m = 2 kg and b=4b = 4 N s/m, with ω=10\omega = 10 rad/s and ζ=0.7\zeta = 0.7, what is kdk_d (in N s/m)?

4. Crossing the gap from simulation to reality

CoreComputational

The lesson trained gains with the cross-entropy method. Write cem_update(samples, scores, elite_frac) that takes an array of candidate parameter vectors samples (shape (n, d)), their scores (shape (n,), lower is better), and the fraction to keep, and returns the new sampling mean and standard deviation (each shape (d,)) computed from the best int(n * elite_frac) samples.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

5. Crossing the gap from simulation to reality

CoreHuman

A quadruped's teacher policy is trained in simulation. Which of these inputs are privileged — available to the teacher in simulation but not to the deployed robot? Select all that apply.

Choose all that apply

6. Crossing the gap from simulation to reality

CoreHuman

A policy was trained with payload masses randomised between 0 and 5 kg. A customer asks whether it will work with an 8 kg payload. What is the best answer?

Choose one

7. Crossing the gap from simulation to reality

Warm-upHuman

A robot's controller runs at 200 Hz. Measurements show 25 ms between reading the sensors and the motor applying the resulting command. How many control periods of delay should the simulator's command queue hold to model this?

Further reading and resources