Skip to content
LumenaryAcademy
Frontier 85 min

Dexterous hands and touch

Multi-fingered hands and the kinematics of turning an object in the fingers; the tactile sensors that let robots feel, from camera-based gels to magnetic skins; a slip detector you build and test; and what learned dexterity can and cannot do as of late 2026.

Code tested · 5AI-draftedAwaiting review by a roboticist

Your guides: Chloe · The Builder, Margaret · The Experimenter, Gideon · The Safety Engineer

MotivationH

The last few centimetres

Take a coin out of a purse, turn a key, or push a button through its hole: your fingers slide, roll and regrip the object without your looking, and your grip tightens in step with the load, before anything slips. The best robots of 2026 pick boxes and bottles by the million with suction cups and two-jaw grippers (Manipulation and grasping), and the vision-language-action models of the first Frontier lesson fold laundry. But ask one to thread a nut onto a loosely held bolt, or to turn a pen end over end in its fingers, and the gap between robot and child is still wide.

Two things are missing. The first is the hand: enough well-placed, well-controlled joints to move an object within the grasp, in a package that survives being bumped and costs less than a car. The second is touch. Roland Johansson and Randall Flanagan review experiments in which people's fingertips were temporarily anaesthetised: although they could see perfectly well, they handled small objects clumsily, letting them slip or gripping far harder than needed. The hairless skin of one human hand has about 17,000 mechanoreceptive nerve units (Johansson and Vallbo, 1979); a typical robot gripper has none.

This lesson builds the core of both: the kinematics of turning an object in the fingers; how tactile sensors work; a model of the most useful thing touch does, detecting slip before it happens, tested in a simulated grip controller; and how dexterity is learned in 2026, what is repeatable and what is still open.

DefinitionsH

Hands: joints, actuators and what they cost

Definition— Degrees of freedom and of actuation

A hand's degrees of freedom (DoF) are its independent joint coordinates; its degrees of actuation (DoA) are the number of motors that drive them. A hand is fully actuated if DoA = DoF and underactuated if DoA < DoF, in which case some joints are coupled to others by tendons, linkages or springs and move in fixed or adaptive patterns.

Kinematic models of the human hand commonly use about 21 joint degrees of freedom, four in each finger and five in the thumb (27 with the wrist), moved by more than 30 muscles, most of them in the forearm, through tendons. The last two joints of each finger tend to move together, so people are partly underactuated too. Robot hands of 2026 span the range from that anatomy down to two jaws; some representative designs, with their makers' or authors' figures:

Hand (maker, country)FingersDoA / jointsDriveNotes
Shadow Dexterous Hand (Shadow Robot, United Kingdom)520 / 24tendons from forearm motorsthe hand of OpenAI's 2018–19 in-hand work
Allegro Hand (Wonik Robotics, South Korea)416 / 16a motor in every jointa workhorse of learning research
LEAP Hand (Carnegie Mellon University, United States, 2023)416 / 16off-the-shelf servosopen design; parts about USD 2,000
DEX-EE (Shadow Robot with Google DeepMind, United Kingdom, 2024)312tendons, several motors per jointbuilt to survive long learning experiments
RH56 series (Inspire Robots, China)56 / 12linear actuators and linkagescommon on humanoids
SharpaWave (Sharpa, Singapore, 2025)522not detailedcamera-based tactile fingertips; used with Gemini Robotics 2

Motors in the fingers (Allegro, LEAP) are simple, but small motors are weak and the fingers thick. Tendons (Shadow, DEX-EE, our own hands) keep fingers slim by putting big motors in the forearm, at the price of friction, stretch and wear. Few actuators with linkages (Inspire) are cheap and strong enough for the power and pinch grasps of factory pilots, but cannot move joints independently, which rules out most in-hand manipulation. And every hand must survive collisions, which is why the backdrivable, low-inertia drives of Force and impedance control matter in fingers as much as in legs.

Note

Small open hands such as LEAP make good first hardware, but even a small hand has pinch points between its fingers and can squeeze hard enough to hurt. Try new controllers in simulation first, then on the real hand with low force and speed limits, an emergency stop within reach, and your own fingers out of the grasp.

ComputationC

Moving an object with the fingers

Grasping was about holding. In-hand manipulation asks the converse: can the fingers move the object, and with which joint motions?

Let kk fingertips touch an object at points pip_i with sticking point contacts with friction, and let the object move with twist ν\nu (ν=(vx,vy,ω)\nu = (v_x, v_y, \omega) in the plane). The object's material point under fingertip ii moves with velocity v+ω×(pi−c)v + \omega \times (p_i - c), where cc is its reference point; stacked, these velocities are GTνG^{\mathsf T}\nu, where GG is the grasp matrix of Manipulation and grasping, the map from contact forces to net wrench. The fingertips move with velocities Jhq˙J_h \dot q, where the hand Jacobian JhJ_h has one finger Jacobian per finger on its diagonal. Sticking contacts make the two agree.

Definition— Contact constraint and manipulable grasp

For sticking point contacts, the joint velocities q˙\dot q and the object twist ν\nu satisfy

Jh(q) q˙=GTν.J_h(q)\,\dot q = G^{\mathsf T}\nu .

The grasp is manipulable if every object twist can be produced by some joint motion: range⁡(GT)⊆range⁡(Jh)\operatorname{range}(G^{\mathsf T}) \subseteq \operatorname{range}(J_h).

Theorem— How many joints a fingertip grasp needs

Consider a fingertip grasp in three dimensions: kk fingers, each touching the object at one sticking point contact with friction. The grasp is manipulable if and only if every touching finger's Jacobian JiJ_i (from its joint velocities to its fingertip velocity) has rank 3. So each touching finger needs at least three joints, and a three-fingered grasp at least nine. In the plane, the same argument gives rank 2 and two joints per finger.

Proof

Because each finger touches at one contact, JhJ_h is block-diagonal and range⁡(Jh)=range⁡(J1)×⋯×range⁡(Jk)\operatorname{range}(J_h) = \operatorname{range}(J_1) \times \cdots \times \operatorname{range}(J_k). Take a pure translation of the object, ν=(v,0)\nu = (v, 0): every contact point must then move with the same velocity vv, so GTν=(v,v,…,v)G^{\mathsf T}\nu = (v, v, \ldots, v). If the grasp is manipulable, this lies in range⁡(Jh)\operatorname{range}(J_h) for every v∈R3v \in \mathbb{R}^3, so each range⁡(Ji)\operatorname{range}(J_i) is all of R3\mathbb{R}^3: rank 3, which needs at least three columns, that is, three joints. Conversely, if every JiJ_i has rank 3, then range⁡(Jh)=R3k\operatorname{range}(J_h) = \mathbb{R}^{3k}, which contains range⁡(GT)\operatorname{range}(G^{\mathsf T}).

∎

Kenneth Salisbury's hand at Stanford (1982) had exactly three fingers of three joints. The program below does the planar version: two fingers of two joints hold a 5 cm disc by its sides, and the hand turns the disc about its centre with q˙=Jh−1GTν\dot q = J_h^{-1} G^{\mathsf T}\nu until a finger runs out of range.

q1q2q1q2left fingerright fingersqueezeω
Two two-jointed fingers hold a disc at opposite points. Turning the disc anticlockwise moves the right contact up and the left one down, so the right finger must extend and the left one curl. The squeezing forces along the line between the contacts produce no net wrench: they span the null space of the grasp matrix.
python
import numpy as np

# Two planar fingers (2 joints each) hold a disc by its sides, seen from the front.
L1, L2 = 0.06, 0.05                                   # link lengths (m)
BASES = [np.array([-0.04, 0.0]), np.array([0.04, 0.0])]
BEND = [-1, +1]                                       # left finger bends clockwise, right one anticlockwise
c, R = np.array([0.0, 0.08]), 0.025                   # disc centre and radius (m)

def tip(q, base):
    return base + np.array([L1 * np.cos(q[0]) + L2 * np.cos(q[0] + q[1]),
                            L1 * np.sin(q[0]) + L2 * np.sin(q[0] + q[1])])

def finger_jacobian(q):
    s1, c1, s12, c12 = np.sin(q[0]), np.cos(q[0]), np.sin(q[0] + q[1]), np.cos(q[0] + q[1])
    return np.array([[-L1 * s1 - L2 * s12, -L2 * s12], [L1 * c1 + L2 * c12, L2 * c12]])

def ik(p, base, bend):
    d = p - base
    q2 = bend * np.arccos((d @ d - L1**2 - L2**2) / (2 * L1 * L2))
    return np.array([np.arctan2(d[1], d[0]) - np.arctan2(L2 * np.sin(q2), L1 + L2 * np.cos(q2)), q2])

def grasp_matrix(contacts, centre):
    """3 x 2k: contact forces -> net wrench (fx, fy, torque) on the object; its transpose maps the
    object's twist (vx, vy, omega) to the contact points' velocities."""
    blocks = []
    for p in contacts:
        r = p - centre
        blocks.append(np.array([[1.0, 0.0], [0.0, 1.0], [-r[1], r[0]]]))
    return np.hstack(blocks)

# Material points of the disc touched by the fingertips (fixed in the disc: sticking contacts)
body_pts = [np.array([-R, 0.0]), np.array([R, 0.0])]
q = [ik(c + body_pts[i], BASES[i], BEND[i]) for i in range(2)]

G = grasp_matrix([c + b for b in body_pts], c)
_, s, Vt = np.linalg.svd(G)
print("grasp matrix rank:", np.linalg.matrix_rank(G), "of 3;  internal force (null space of G):",
      np.round(Vt[-1] / np.abs(Vt[-1]).max(), 3))

# Turn the disc anticlockwise about its centre, using only finger joint motions: qdot = J_h^-1 G^T nu.
nu = np.array([0.0, 0.0, 1.0])                        # (vx, vy, omega): pure rotation
dt = np.radians(0.1)                                  # turn in steps of 0.1 degree
k = 0
print(f"\n{'turned':>6} {'left finger q1, q2':>19} {'right finger q1, q2':>20} {'det J (cm^2) L, R':>18} {'contact error':>14}")
while True:
    theta = k * dt
    Rot = np.array([[np.cos(theta), -np.sin(theta)], [np.sin(theta), np.cos(theta)]])
    contacts = [c + Rot @ b for b in body_pts]
    dets = [np.linalg.det(finger_jacobian(qi)) for qi in q]
    err = max(np.linalg.norm(tip(q[i], BASES[i]) - contacts[i]) for i in range(2))
    deg = np.degrees(theta)
    bent = [abs(np.degrees(qi[1])) for qi in q]
    if k % 100 == 0:
        print(f"{deg:5.0f}° {np.degrees(q[0][0]):8.1f} {np.degrees(q[0][1]):7.1f}° {np.degrees(q[1][0]):9.1f} {np.degrees(q[1][1]):7.1f}°"
              f" {1e4 * dets[0]:9.1f} {1e4 * dets[1]:6.1f} {1e6 * err:10.1f} um")
    if min(bent) < 10 or max(bent) > 120:
        i = int(np.argmin(bent)) if min(bent) < 10 else int(np.argmax(bent))
        print(f"stop at {deg:.1f}°: the {'left' if i == 0 else 'right'} finger's second joint is at {bent[i]:.1f}° "
              f"({'nearly straight: singular' if bent[i] < 10 else 'at its flexion limit'})")
        break
    Gt = grasp_matrix(contacts, c).T                  # contact-point velocities = G^T nu
    v = Gt @ nu
    q = [q[i] + dt * np.linalg.solve(finger_jacobian(q[i]), v[2 * i:2 * i + 2]) for i in range(2)]
    k += 1
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

The grasp matrix has rank 3, and its null space (1,0,−1,0)(1, 0, -1, 0) is the squeeze: the fingers push towards each other and the disc does not move. As the disc turns, the right finger straightens (its Jacobian determinant falls from 29.9 to 10.8 cm² by 70°) until, at 75.9°, it is nearly singular and can no longer move its tip along its own length. Integrating velocities open-loop also lets the fingertips drift 37 µm from the contacts, which is why real controllers close the loop on measured joint angles and, better, on touch.

Real fingers have joint limits, fingertips roll, and the friction cones of a non-circular object tilt as it turns, so a full revolution needs more than finger motion with fixed contacts: finger gaiting (lift one finger and replace it while the others hold), rolling over curved fingertips, controlled sliding, and pivoting against gravity or the environment, which Nikhil Chavan-Dafle, Alberto Rodriguez and colleagues at MIT and Carnegie Mellon called extrinsic dexterity (2014). Each depends on knowing what the contacts are doing, which is what touch is for.

IntuitionH

Touch: how robots feel

Human fingertips carry four kinds of mechanoreceptor. Two adapt slowly and report sustained pressure and skin stretch; two adapt fast and report change, among them the Pacinian corpuscles, most sensitive to vibrations of a few hundred hertz, the signature of slip. Spatial detail is a millimetre or two, timing a few milliseconds. Robot tactile sensors trade these qualities against each other.

vision-basedgel, markers, camera, lightscapacitive taxelssqueezed gap, higher capacitancemagnetic skinmoving magnets, magnetometershearchip
Three ways to build a robot fingertip that feels. A vision-based sensor films the inside of a soft gel as it deforms. A capacitive taxel measures how a soft dielectric layer between two electrodes is squeezed. A magnetic skin measures how a magnetised elastomer moves above a magnetometer, which reveals shear as well as pressure.
  • Vision-based sensors. A camera inside the fingertip films a soft gel from behind. In GelSight (Edward Adelson's group at MIT, from 2009), a reflective skin lit from several directions turns the contact's 3-D shape into shading at micrometre scale, and printed markers show shear. Meta's DIGIT (2020) made the idea compact and cheap; Digit 360 (2024) adds vibration sensing up to 10 kHz, heat and processing on the fingertip. Bristol's TacTip films pins under a 3-D printed skin; Insight (Max Planck Institute for Intelligent Systems, Germany, 2022) maps force over a thumb-sized curved surface. The images are rich, but cameras run at tens of frames per second, gels wear, and the fingertips are bulky.
  • Capacitive and resistive arrays. Grids of taxels whose capacitance or resistance changes when pressed: thin, fast (hundreds of hertz to kilohertz) and able to cover large areas, like the skin of the iCub humanoid at the Italian Institute of Technology, but with millimetre resolution and weaker on shear.
  • Magnetic skins. A magnetised elastomer above Hall-effect or magnetometer chips: any deformation, normal or sideways, changes the field. XELA Robotics in Tokyo, a Waseda University spin-out, sells three-axis magnetic taxels that have been fitted to Allegro hands; ReSkin (Meta AI and Carnegie Mellon, 2021) and AnySkin (New York University, 2024) make the skin a cheap, replaceable layer. Nearby magnets and steel disturb them.
  • Vibration, multimodal and intrinsic sensing. Piezoelectric films catch slip and texture vibrations at kilohertz rates; SynTouch's BioTac, from research at the University of Southern California, combined pressure, vibration and temperature in a fluid-filled fingertip; and motor currents or joint torque sensors can estimate contact with no skin at all.
ComputationC

Feeling slip before it happens

Touch earns its keep by setting grip force. Too little and the object slips; too much and it is crushed or deformed. People keep their grip a modest margin above the slip limit and adapt it to each object's friction within about a tenth of a second of touching it (Johansson and Westling, 1984). How can a sensor know how close a grasp is to slipping, when the friction coefficient μ\mu is unknown and changes with wetness and dust?

Contact mechanics answers. Press a rounded elastic fingertip onto a surface and pull it sideways with less than the friction limit: its centre sticks while a ring around the edge already slips.

Theorem— Cattaneo–Mindlin partial slip

For a Hertzian (sphere-on-flat, elastic) contact of radius aa under normal force FnF_n, loaded monotonically by a tangential force Ft≤μFnF_t \le \mu F_n, the contact divides into a central stick zone of radius cc and a slip annulus c<r≤ac \lt r \le a, with

ca=(1−FtμFn)1/3.\frac{c}{a} = \left(1 - \frac{F_t}{\mu F_n}\right)^{1/3}.

Gross sliding begins when the stick zone vanishes, at Ft=μFnF_t = \mu F_n.

Cattaneo (1938) and Mindlin (1949) derived this from the elastic tractions in the contact; we take it as given. Its consequence is the point: the fraction of the available friction in use, ρ=Ft/(μFn)=1−(c/a)3\rho = F_t/(\mu F_n) = 1 - (c/a)^3, can be read from the size of the stick zone without knowing μ\mu. And a marker-based sensor can see the stick zone: markers there move with the object, while those in the annulus lag behind.

sideways load F_tstick zone, radius c:markers move with the objectslip annulus, c to a:markers lag behindc / a = (1 − F_t / μF_n)^(1/3)contact edge, radius a
Incipient slip seen by a marker-based tactile sensor. Each dot is a marker on the gel and its tail shows how far it has moved since the object was first touched. Inside the stick zone (radius c) the markers move together with the object; in the slip annulus they lag; outside the contact (radius a) they hardly move. As the sideways load grows, the stick zone shrinks towards its centre.

The program simulates a camera-based fingertip with markers every 0.5 mm over a 4 mm contact, under growing sideways load. It does not solve the elasticity problem; its displacement field merely has the right structure (a rigid stick zone of the Cattaneo–Mindlin size, a lagging annulus, 4 µm of tracking noise). An estimator that knows nothing of μ\mu labels each marker stuck or slipping and reports c/ac/a.

python
import numpy as np

rng = np.random.default_rng(5)

# Markers printed on the gel of a vision-based tactile sensor, on a 0.5 mm grid; contact radius a = 4 mm.
a, pitch, noise = 4.0, 0.5, 0.004                      # mm; tracking noise 4 micrometres
g = np.arange(-6.0, 6.0 + 1e-9, pitch)
X, Y = np.meshgrid(g, g)
pts = np.column_stack([X.ravel(), Y.ravel()])
r = np.linalg.norm(pts, axis=1)
in_contact = r <= a                                    # the contact patch is visible in the tactile image

def marker_field(rho, delta_max=0.4):
    """Simplified sideways marker displacements (mm) for a load ratio rho = F_t / (mu F_n).
    Cattaneo-Mindlin: the stuck centre has radius c = a (1 - rho)^(1/3) and moves rigidly by delta;
    in the slipping annulus c < r <= a the gel lags behind; outside the contact it relaxes."""
    c = a * (1 - rho) ** (1 / 3)
    delta = delta_max * (1 - (1 - rho) ** (2 / 3))     # Mindlin's tangential compliance curve
    u = np.where(r <= c, delta, delta * (1 - 0.6 * np.clip((r - c) / (a - c + 1e-12), 0, 1)))
    u = np.where(r > a, delta * 0.4 * np.exp(-(r - a) / 1.0), u)
    disp = np.column_stack([u, np.zeros_like(u)])      # the object is pulled along +x
    return disp + rng.normal(0, noise, disp.shape), c

def estimate_stick_ratio(disp, tol=0.012):
    """Markers that move together with the centre are stuck. Iterate: estimate the common
    translation from the markers believed stuck, then re-label every contact marker."""
    stuck = in_contact & (r <= a / 3)                  # start from the middle of the patch
    for _ in range(10):
        t = disp[stuck].mean(axis=0)
        new = in_contact & (np.linalg.norm(disp - t, axis=1) < tol + 0.02 * np.linalg.norm(t))
        if np.array_equal(new, stuck):
            break
        stuck = new
    return np.sqrt(stuck.sum() / in_contact.sum())     # area fraction -> radius fraction

print(f"{in_contact.sum()} markers inside the contact patch\n")
print(" load ratio rho   true c/a   estimated c/a   rho from the image")
for rho in [0.0, 0.2, 0.4, 0.6, 0.7, 0.8, 0.9, 0.95]:
    disp, c = marker_field(rho)
    est = estimate_stick_ratio(disp)
    print(f"{rho:12.2f} {c / a:12.3f} {est:13.3f} {1 - est ** 3:16.2f}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Near slip, where it matters, the image tracks the load ratio: 0.85 for a true 0.9, and 0.91 for 0.95. At low loads the slip annulus is thinner than the marker spacing and the estimate is poor (0.10 for a true 0.2), which is harmless, because a grasp far from slipping needs no correction. The estimated stick zone is always slightly too large: the sensor errs towards optimism.

A grip controller that feels

Now use it. A robot pinches a paper cup between two fingertips while 200 g of water is poured in, then lifts it briskly. The cup buckles if either finger presses harder than 8 N, and it may be dry (μ=0.9\mu = 0.9) or wet (μ=0.4\mu = 0.4). Five controllers compete: two fixed grips; one that measures the shear force and grips just enough for an assumed dry μ\mu; a slip reflex that watches a 1 kHz shear signal for the micro-vibrations of slip (the root-mean-square of its sample-to-sample changes over 10 ms) and squeezes harder when it finds them; and an incipient-slip controller that reads the stick ratio from a 60 Hz tactile camera, with the bias and noise we just measured, and scales the grip to hold ρ\rho at one half.

python
import numpy as np

G, DT = 9.81, 0.001                     # gravity (m/s^2), 1 kHz control loop
F_MIN, F_MAX, CRUSH = 1.0, 15.0, 8.0    # grip force per finger (N); the paper cup buckles above 8 N

def scenario(t):
    """Mass of the cup (kg) as water is poured in, and the arm's upward acceleration (m/s^2)."""
    m = 0.03 + 0.08 * np.clip(t - 0.5, 0.0, 2.5)                  # 30 g cup + 200 g of water
    ramp = lambda t0, T: np.clip((t - t0) / T, 0.0, 1.0)
    acc = 4.0 * (ramp(3.5, 0.03) - ramp(3.8, 0.03)) - 2.0 * (ramp(4.2, 0.1) - ramp(4.8, 0.1))   # a brisk lift
    return m, acc

def run(controller, mu_s, seed=0):
    rng = np.random.default_rng(seed)
    mu_k = 0.8 * mu_s                                               # sliding friction is lower
    F = 1.0
    v = slide = slip_d = peak_F = 0.0
    slips = 0
    readings, state = [], {"F_cmd": 1.0, "cooldown": 0, "ratio": 1.0}
    first_slip = detected = None
    for k in range(int(5.0 / DT)):
        t = k * DT
        m, acc = scenario(t)
        W = m * (G + acc)                                           # load the two fingers must carry
        if v <= 0.0 and W <= 2 * mu_s * F:                         # sticking
            v, F_t, slip_d = 0.0, W / 2, 0.0
        else:                                                       # sliding down between the fingers
            if slip_d == 0.0:
                slips += 1
            mu = mu_k + (mu_s - mu_k) * np.exp(-slip_d / 0.0005)    # friction falls over 0.5 mm of slip
            v = max(0.0, v + (W - 2 * mu * F) / m * DT)
            F_t = mu * F
            slip_d += v * DT + 1e-12
            first_slip = t if first_slip is None else first_slip
        slide += v * DT
        if slide > 0.03:
            return slide, peak_F, "dropped", first_slip, detected, slips
        # Tactile signals. Shear force at 1 kHz, with micro-vibrations while sliding (fast channel):
        vib = 0.05 * min(1.0, v / 0.005) * rng.normal() if v > 0 else 0.0
        readings.append(F_t + vib + rng.normal(0, 0.005))
        # Stick ratio c/a from the tactile camera at 60 Hz (slow channel), slightly optimistic:
        if k % 17 == 0:
            rho = F_t / (mu_s * F) if v == 0 else 1.0
            state["ratio"] = float(np.clip((1 - min(rho, 1.0)) ** (1 / 3) + 0.05 + rng.normal(0, 0.02), 0, 1))
        F_cmd, hit = controller(readings, state, F, k)
        if hit and detected is None and first_slip is not None:
            detected = t
        F += (np.clip(F_cmd, 0.0, F_MAX) - F) * DT / 0.03           # the finger drive lags by 30 ms
        peak_F = max(peak_F, F)
    outcome = "crushed" if peak_F > CRUSH else "held"
    return slide, peak_F, outcome, first_slip, detected, slips

def fixed(force):
    return lambda readings, state, F, k: (force, False)

def assumed_friction(mu_guess, rho_target=0.6):
    """Measure the shear force and grip just enough for a friction coefficient you assume."""
    return lambda readings, state, F, k: (max(F_MIN, readings[-1] / (mu_guess * rho_target)), False)

def vibration_reflex(readings, state, F, k, window=10, threshold=0.03):
    """Slip detector: RMS of the first difference of the shear signal over the last 10 ms."""
    hit = False
    if len(readings) > window:
        d = np.diff(readings[-window - 1:])
        hit = np.sqrt(np.mean(d ** 2)) > threshold
        if hit and state["cooldown"] <= 0:
            state["F_cmd"] = min(F_MAX, 1.4 * state["F_cmd"] + 0.3)   # squeeze harder, then wait
            state["cooldown"] = 40
    state["cooldown"] -= 1
    return state["F_cmd"], hit

def incipient_slip(readings, state, F, k, rho_target=0.5):
    """Keep the measured load ratio rho = 1 - (c/a)^3 at a target, without knowing mu."""
    if k % 17 == 0:
        rho_hat = 1 - state["ratio"] ** 3
        state["F_cmd"] = float(np.clip(F * rho_hat / rho_target, F_MIN, F_MAX))
    return state["F_cmd"], state["ratio"] < 0.05

controllers = [("fixed gentle grip, 2 N", fixed(2.0)), ("fixed firm grip, 10 N", fixed(10.0)),
               ("shear force, assumes mu = 0.9", assumed_friction(0.9)),
               ("vibration slip reflex", vibration_reflex), ("stick ratio (incipient slip)", incipient_slip)]
for mu, name in [(0.9, "dry cup, mu = 0.9"), (0.4, "wet cup, mu = 0.4")]:
    print(name)
    for label, ctrl in controllers:
        slide, peak, outcome, first, det, slips = run(ctrl, mu)
        lag = f" (first detected after {1000 * (det - first):.0f} ms)" if det is not None else ""
        print(f"  {label:30} {slips:2d} slips, slid {1000 * slide:4.1f} mm, peak grip {peak:4.1f} N: {outcome}{lag}")
Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.
  • Fixed grips fail in opposite ways: 2 N holds the dry cup and drops the wet one, and 10 N crushes both.
  • Shear force with an assumed μ\mu works on the dry cup. On the wet one, the first slip lowers the measured shear force (sliding friction is weaker), so the controller relaxes its grip and the cup falls: a wrong model, trusted, makes things worse.
  • The vibration reflex holds both cups with less force than the stick-ratio controller (2.7 N dry, 4.1 N wet), but only by slipping first: two slips on the dry cup and three on the wet one, the first caught 34 ms and 24 ms after it began. Each moved the cup only a fraction of a millimetre because the load grew gently; faster loads or slower finger drives mean longer slides.
  • The stick-ratio controller never lets the cup slip, because it acts while the stick zone shrinks. Its price is a firmer grip, 6.8 N on the wet cup, close to the 8 N at which the cup buckles. The honest conclusion is that two fingertips are the wrong grasp for a full, wet paper cup; a hand should wrap it.
ExplanationH

Contact-rich control

Holding is the simplest contact-rich task; insertion, screwing, wiping and reorientation depend on forces at several contacts that change within milliseconds. Four approaches are in use in 2026.

  • Impedance at the fingers and at the object. Each finger runs joint-level impedance control (Force and impedance control), and an object-level controller commands motion through GTG^{\mathsf T} while regulating the squeeze in the null space of GG, as Germany's DLR has done since the 2000s. Most learned policies sit on top of a layer like this.
  • Tactile servoing. The tactile image is the error signal: keep a contact centred, an edge aligned, or the stick ratio at a set-point. Nathan Lepora's group at Bristol has shown fingertips following unknown edges and surfaces this way.
  • Contact-implicit planning. Contacts are made and broken inside the optimisation. Because contact makes the dynamics non-smooth, MIT's Tao Pang, Terry Suh, Lujie Yang and Russ Tedrake (2022) smoothed a quasi-dynamic contact model so that planners could plan whole-hand and whole-arm manipulation; sampling-based predictive control in MuJoCo (World models) serves similar ends.
  • Industrial force control. Factory assembly runs today on a wrist force–torque sensor, low stiffness along the insertion axis, a spiral search when a peg misses its hole, and a check of the force signature that a connector is seated: unglamorous, reliable, and the bar a learned system must clear.

Since 2024, several groups have trained policies that output stiffness or force targets along with positions, soft while searching and firm while pushing; as of late 2026 there is no standard recipe.

ExplanationH

Learning dexterity: what works in 2026

In research, learning now drives most work on multi-fingered hands. The routes, with the evidence the labs report:

Reinforcement learning in simulation. The landmarks up to 2022 (OpenAI's Shadow Hand cube and Rubik's Cube, NVIDIA's DeXtreme, MIT's Visual Dexterity) are in Sim-to-real. Since then, Haozhi Qi and colleagues at Berkeley (2022) rotated many objects on an Allegro hand using proprioception alone; Zhao-Heng Yin and colleagues at UC San Diego (2023) did it with simple binary touch sensors and no vision; Qi and colleagues (2023) combined vision and touch; Bristol's AnyRotate (2024) turned unseen objects about arbitrary axes, with the hand facing any way, using dense tactile fingertips; and Berkeley's Twisting Lids Off with Two Hands (2024) took the method to two hands. DexterityGen (Berkeley and Meta, 2025) pretrained a low-level controller on many reinforcement-learned primitives and let a teleoperator steer it, reporting the use of a pen, a syringe and a screwdriver.

Human data. Teleoperating a dexterous hand is hard: the operator feels nothing, and the robot's hand is not theirs. Motion-capture gloves (Stanford's DexCap, 2024), virtual-reality hand tracking (Imitation learning) and human video supply data, and retargeting human finger motion onto a different hand is a research topic of its own.

Touch as a learned representation. Tactile images differ between sensors and even between two gels of one design. Meta's Sparsh (2024) pretrained self-supervised models on more than 460,000 tactile images from several sensors, and simulators such as TACTO render tactile images for training, though simulated gels still differ from real ones.

Touch in vision-language-action models. The VLAs of 2026 mostly see and do not feel. Research systems add touch as further input tokens to the transformer (see Attention and the transformer and Language models): Tactile-VLA (Tsinghua University and collaborators, 2025) feeds tactile and force signals into a VLA and outputs force targets as well as positions; OmniVTLA (Shanghai Jiao Tong University and the sensor maker PaXini, 2025) aligns tactile signals with visual and language concepts. No recipe has become standard. Among the few published success rates for five-fingered manipulation by a generalist model are Google DeepMind's for Gemini Robotics 2 (July 2026), on a humanoid with 22-degree-of-freedom hands: 92% for unscrewing a light bulb, 36% for screwing one in.

What is repeatable, and what is demonstration, as of late 2026:

Repeatable, with published protocolsDemonstrated, not yet reliableOpen
in-hand rotation and reorientation of many objects in the labautonomous tool use with dexterous hands outside the labhands that survive months of contact at low cost
tactile estimation of contact shape, force and slip on research sensorslong dexterous tasks at factory reliabilitytactile skins that do not wear, drift or need recalibration
force-controlled insertion and assembly in factorieshumanoid hand tasks in videos without stated trial countstouch data at scale, and a representation shared across sensors
power and pinch grasps on humanoids in pilotsgeneral manipulation of deformable objectssimulating soft contact and tactile sensors well enough to transfer

When you read a dexterity claim, ask whether the video is autonomous or teleoperated, at what playback speed, how many trials were run with what success rate, and whether the objects and scenes were seen in training. Responsible labs report all of these.

Common misconceptionsH

Common misconceptions

Misconception

"More degrees of freedom make a more dexterous hand." Joints without sensing, control and robustness add weight, cost and failure points. In-hand manipulation does need independent joints, as the counting theorem shows, but they are necessary, not sufficient: a six-actuator hand that feels can beat a twenty-joint hand that cannot on many factory tasks.

Misconception

"A tactile sensor measures force." Most measure deformation or geometry and infer force by calibration or learning. Touch also reports contact location, shape, the distribution of shear, vibration and temperature, and in manipulation the distribution often matters more than the total.

Misconception

"With good cameras, touch is unnecessary." At the moment of contact the fingers hide the contact from the cameras, friction is invisible, and slip begins in micrometres and milliseconds. People with numbed fingertips see their hands perfectly well and still fumble.

Misconception

"To stop slipping, detect slip and squeeze." By the time slip is detected the object has moved, which matters for a pen or a key, and squeezing hard enough to be safe from every slip crushes fragile things. A margin read from the stick zone, with a fast reflex as backup, does better.

Misconception

"Sim-to-real has solved in-hand manipulation." It has made in-hand rotation and reorientation repeatable on research hands. Tool use, deformable objects, long tasks and production reliability remain open, partly because soft contact and tactile sensors are still hard to simulate.

History & global context

A short history

Tokuji Okada's three-fingered hand at Japan's Electrotechnical Laboratory (1979) was among the first computer-controlled multi-fingered hands. Salisbury's Stanford/JPL hand (1982) set the three-finger, three-joint pattern, and Stephen Jacobsen's tendon-driven Utah/MIT Dexterous Hand (mid-1980s) had four fingers with sixteen joints. DLR's hands in Germany, the Shadow hand, NASA's Robonaut hand and the high-speed hand of Masatoshi Ishikawa's laboratory at the University of Tokyo followed. Tactile sensing grew alongside, from piezoresistive pads to the iCub skin, the BioTac and GelSight (2009); learning arrived at scale with OpenAI's in-hand cube in 2018.

Exercises

Exercises

1. Dexterous hands and touch

Warm-upHuman

A robot holds a 400 g jar upright between two fingertips that squeeze horizontally, with friction coefficient μ=0.5\mu = 0.5 at each contact and g=9.81 m/s2g = 9.81\ \text{m/s}^2. What is the smallest normal force per finger, in newtons (rounded to the nearest whole number), that keeps the jar from slipping while it is held still?

2. Dexterous hands and touch

CoreHuman

A rounded fingertip presses on an object with normal force Fn=4F_n = 4 N, friction coefficient μ=0.5\mu = 0.5, and carries a tangential load Ft=1.5F_t = 1.5 N. Using the Cattaneo–Mindlin result, what is the radius of the stick zone as a percentage of the contact radius, 100 c/a100\, c/a, rounded to the nearest whole number?

3. Dexterous hands and touch

CoreHuman

A hand holds an object in three dimensions with four fingertips, each touching at one sticking point contact with friction. What is the smallest total number of joints in the four touching fingers for which the grasp can be manipulable, that is, for which the fingers can give the object any small motion while the contacts stick?

4. Dexterous hands and touch

CoreHuman

A gripper should notice the onset of slip from the micro-vibrations it causes, which in this task are strongest between 100 and 400 Hz. Which sensor is best suited to that signal?

Choose one

5. Dexterous hands and touch

CoreHuman

Why can a controller that reads the stick-zone radius from a marker-based tactile sensor keep a grasp away from slipping without knowing the friction coefficient μ\mu?

Choose one

6. Dexterous hands and touch

CoreHuman

A company posts a video of its humanoid's five-fingered hand peeling a clementine. Which facts do you need before concluding what the hand can do? (Choose all that apply.)

Choose all that apply

7. Dexterous hands and touch

CoreComputational

Write the fast slip detector of the lesson's grip controller. detect_slip(signal, window, threshold) takes a sequence of shear-force readings (one per millisecond) and returns the index ii of the first sample, with i≥i \ge window, at which the root-mean-square of the window sample-to-sample differences of signal[i - window : i + 1] exceeds threshold; it returns None if that never happens. The tests check that noise and a smooth change of load do not trigger it, that slip vibrations do, and what happens with a sudden step.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

8. Dexterous hands and touch

CoreComputational

Write grasp_matrix(points, centre) for planar point contacts. For contacts at points pip_i and torques taken about centre cc, it returns the 3×2k3 \times 2k matrix GG that maps the stacked contact forces (f1x,f1y,f2x,f2y,…)(f_{1x}, f_{1y}, f_{2x}, f_{2y}, \ldots) to the net wrench (Fx,Fy,τ)(F_x, F_y, \tau) on the object, with τ=rxfy−ryfx\tau = r_x f_y - r_y f_x for r=pi−cr = p_i - c. The tests also check that GTG^{\mathsf T} maps an object rotation to the contact-point velocities.

Runs entirely in your browser, in an isolated worker. Nothing is sent to a server.

Further reading and resources