The last few centimetres
Take a coin out of a purse, turn a key, or push a button through its hole: your fingers slide, roll and regrip the object without your looking, and your grip tightens in step with the load, before anything slips. The best robots of 2026 pick boxes and bottles by the million with suction cups and two-jaw grippers (Manipulation and grasping), and the vision-language-action models of the first Frontier lesson fold laundry. But ask one to thread a nut onto a loosely held bolt, or to turn a pen end over end in its fingers, and the gap between robot and child is still wide.
Two things are missing. The first is the hand: enough well-placed, well-controlled joints to move an object within the grasp, in a package that survives being bumped and costs less than a car. The second is touch. Roland Johansson and Randall Flanagan review experiments in which people's fingertips were temporarily anaesthetised: although they could see perfectly well, they handled small objects clumsily, letting them slip or gripping far harder than needed. The hairless skin of one human hand has about 17,000 mechanoreceptive nerve units (Johansson and Vallbo, 1979); a typical robot gripper has none.
This lesson builds the core of both: the kinematics of turning an object in the fingers; how tactile sensors work; a model of the most useful thing touch does, detecting slip before it happens, tested in a simulated grip controller; and how dexterity is learned in 2026, what is repeatable and what is still open.
Hands: joints, actuators and what they cost
Definition— Degrees of freedom and of actuation
A hand's degrees of freedom (DoF) are its independent joint coordinates; its degrees of actuation (DoA) are the number of motors that drive them. A hand is fully actuated if DoA = DoF and underactuated if DoA < DoF, in which case some joints are coupled to others by tendons, linkages or springs and move in fixed or adaptive patterns.
Kinematic models of the human hand commonly use about 21 joint degrees of freedom, four in each finger and five in the thumb (27 with the wrist), moved by more than 30 muscles, most of them in the forearm, through tendons. The last two joints of each finger tend to move together, so people are partly underactuated too. Robot hands of 2026 span the range from that anatomy down to two jaws; some representative designs, with their makers' or authors' figures:
| Hand (maker, country) | Fingers | DoA / joints | Drive | Notes |
|---|---|---|---|---|
| Shadow Dexterous Hand (Shadow Robot, United Kingdom) | 5 | 20 / 24 | tendons from forearm motors | the hand of OpenAI's 2018–19 in-hand work |
| Allegro Hand (Wonik Robotics, South Korea) | 4 | 16 / 16 | a motor in every joint | a workhorse of learning research |
| LEAP Hand (Carnegie Mellon University, United States, 2023) | 4 | 16 / 16 | off-the-shelf servos | open design; parts about USD 2,000 |
| DEX-EE (Shadow Robot with Google DeepMind, United Kingdom, 2024) | 3 | 12 | tendons, several motors per joint | built to survive long learning experiments |
| RH56 series (Inspire Robots, China) | 5 | 6 / 12 | linear actuators and linkages | common on humanoids |
| SharpaWave (Sharpa, Singapore, 2025) | 5 | 22 | not detailed | camera-based tactile fingertips; used with Gemini Robotics 2 |
Motors in the fingers (Allegro, LEAP) are simple, but small motors are weak and the fingers thick. Tendons (Shadow, DEX-EE, our own hands) keep fingers slim by putting big motors in the forearm, at the price of friction, stretch and wear. Few actuators with linkages (Inspire) are cheap and strong enough for the power and pinch grasps of factory pilots, but cannot move joints independently, which rules out most in-hand manipulation. And every hand must survive collisions, which is why the backdrivable, low-inertia drives of Force and impedance control matter in fingers as much as in legs.
Note
Small open hands such as LEAP make good first hardware, but even a small hand has pinch points between its fingers and can squeeze hard enough to hurt. Try new controllers in simulation first, then on the real hand with low force and speed limits, an emergency stop within reach, and your own fingers out of the grasp.
Moving an object with the fingers
Grasping was about holding. In-hand manipulation asks the converse: can the fingers move the object, and with which joint motions?
Let fingertips touch an object at points with sticking point contacts with friction, and let the object move with twist ( in the plane). The object's material point under fingertip moves with velocity , where is its reference point; stacked, these velocities are , where is the grasp matrix of Manipulation and grasping, the map from contact forces to net wrench. The fingertips move with velocities , where the hand Jacobian has one finger Jacobian per finger on its diagonal. Sticking contacts make the two agree.
Definition— Contact constraint and manipulable grasp
For sticking point contacts, the joint velocities and the object twist satisfy
The grasp is manipulable if every object twist can be produced by some joint motion: .
Theorem— How many joints a fingertip grasp needs
Consider a fingertip grasp in three dimensions: fingers, each touching the object at one sticking point contact with friction. The grasp is manipulable if and only if every touching finger's Jacobian (from its joint velocities to its fingertip velocity) has rank 3. So each touching finger needs at least three joints, and a three-fingered grasp at least nine. In the plane, the same argument gives rank 2 and two joints per finger.
Proof
Because each finger touches at one contact, is block-diagonal and . Take a pure translation of the object, : every contact point must then move with the same velocity , so . If the grasp is manipulable, this lies in for every , so each is all of : rank 3, which needs at least three columns, that is, three joints. Conversely, if every has rank 3, then , which contains .
∎
Kenneth Salisbury's hand at Stanford (1982) had exactly three fingers of three joints. The program below does the planar version: two fingers of two joints hold a 5 cm disc by its sides, and the hand turns the disc about its centre with until a finger runs out of range.
import numpy as np
# Two planar fingers (2 joints each) hold a disc by its sides, seen from the front.
L1, L2 = 0.06, 0.05 # link lengths (m)
BASES = [np.array([-0.04, 0.0]), np.array([0.04, 0.0])]
BEND = [-1, +1] # left finger bends clockwise, right one anticlockwise
c, R = np.array([0.0, 0.08]), 0.025 # disc centre and radius (m)
def tip(q, base):
return base + np.array([L1 * np.cos(q[0]) + L2 * np.cos(q[0] + q[1]),
L1 * np.sin(q[0]) + L2 * np.sin(q[0] + q[1])])
def finger_jacobian(q):
s1, c1, s12, c12 = np.sin(q[0]), np.cos(q[0]), np.sin(q[0] + q[1]), np.cos(q[0] + q[1])
return np.array([[-L1 * s1 - L2 * s12, -L2 * s12], [L1 * c1 + L2 * c12, L2 * c12]])
def ik(p, base, bend):
d = p - base
q2 = bend * np.arccos((d @ d - L1**2 - L2**2) / (2 * L1 * L2))
return np.array([np.arctan2(d[1], d[0]) - np.arctan2(L2 * np.sin(q2), L1 + L2 * np.cos(q2)), q2])
def grasp_matrix(contacts, centre):
"""3 x 2k: contact forces -> net wrench (fx, fy, torque) on the object; its transpose maps the
object's twist (vx, vy, omega) to the contact points' velocities."""
blocks = []
for p in contacts:
r = p - centre
blocks.append(np.array([[1.0, 0.0], [0.0, 1.0], [-r[1], r[0]]]))
return np.hstack(blocks)
# Material points of the disc touched by the fingertips (fixed in the disc: sticking contacts)
body_pts = [np.array([-R, 0.0]), np.array([R, 0.0])]
q = [ik(c + body_pts[i], BASES[i], BEND[i]) for i in range(2)]
G = grasp_matrix([c + b for b in body_pts], c)
_, s, Vt = np.linalg.svd(G)
print("grasp matrix rank:", np.linalg.matrix_rank(G), "of 3; internal force (null space of G):",
np.round(Vt[-1] / np.abs(Vt[-1]).max(), 3))
# Turn the disc anticlockwise about its centre, using only finger joint motions: qdot = J_h^-1 G^T nu.
nu = np.array([0.0, 0.0, 1.0]) # (vx, vy, omega): pure rotation
dt = np.radians(0.1) # turn in steps of 0.1 degree
k = 0
print(f"\n{'turned':>6} {'left finger q1, q2':>19} {'right finger q1, q2':>20} {'det J (cm^2) L, R':>18} {'contact error':>14}")
while True:
theta = k * dt
Rot = np.array([[np.cos(theta), -np.sin(theta)], [np.sin(theta), np.cos(theta)]])
contacts = [c + Rot @ b for b in body_pts]
dets = [np.linalg.det(finger_jacobian(qi)) for qi in q]
err = max(np.linalg.norm(tip(q[i], BASES[i]) - contacts[i]) for i in range(2))
deg = np.degrees(theta)
bent = [abs(np.degrees(qi[1])) for qi in q]
if k % 100 == 0:
print(f"{deg:5.0f}° {np.degrees(q[0][0]):8.1f} {np.degrees(q[0][1]):7.1f}° {np.degrees(q[1][0]):9.1f} {np.degrees(q[1][1]):7.1f}°"
f" {1e4 * dets[0]:9.1f} {1e4 * dets[1]:6.1f} {1e6 * err:10.1f} um")
if min(bent) < 10 or max(bent) > 120:
i = int(np.argmin(bent)) if min(bent) < 10 else int(np.argmax(bent))
print(f"stop at {deg:.1f}°: the {'left' if i == 0 else 'right'} finger's second joint is at {bent[i]:.1f}° "
f"({'nearly straight: singular' if bent[i] < 10 else 'at its flexion limit'})")
break
Gt = grasp_matrix(contacts, c).T # contact-point velocities = G^T nu
v = Gt @ nu
q = [q[i] + dt * np.linalg.solve(finger_jacobian(q[i]), v[2 * i:2 * i + 2]) for i in range(2)]
k += 1The grasp matrix has rank 3, and its null space is the squeeze: the fingers push towards each other and the disc does not move. As the disc turns, the right finger straightens (its Jacobian determinant falls from 29.9 to 10.8 cm² by 70°) until, at 75.9°, it is nearly singular and can no longer move its tip along its own length. Integrating velocities open-loop also lets the fingertips drift 37 µm from the contacts, which is why real controllers close the loop on measured joint angles and, better, on touch.
Real fingers have joint limits, fingertips roll, and the friction cones of a non-circular object tilt as it turns, so a full revolution needs more than finger motion with fixed contacts: finger gaiting (lift one finger and replace it while the others hold), rolling over curved fingertips, controlled sliding, and pivoting against gravity or the environment, which Nikhil Chavan-Dafle, Alberto Rodriguez and colleagues at MIT and Carnegie Mellon called extrinsic dexterity (2014). Each depends on knowing what the contacts are doing, which is what touch is for.
Touch: how robots feel
Human fingertips carry four kinds of mechanoreceptor. Two adapt slowly and report sustained pressure and skin stretch; two adapt fast and report change, among them the Pacinian corpuscles, most sensitive to vibrations of a few hundred hertz, the signature of slip. Spatial detail is a millimetre or two, timing a few milliseconds. Robot tactile sensors trade these qualities against each other.
- Vision-based sensors. A camera inside the fingertip films a soft gel from behind. In GelSight (Edward Adelson's group at MIT, from 2009), a reflective skin lit from several directions turns the contact's 3-D shape into shading at micrometre scale, and printed markers show shear. Meta's DIGIT (2020) made the idea compact and cheap; Digit 360 (2024) adds vibration sensing up to 10 kHz, heat and processing on the fingertip. Bristol's TacTip films pins under a 3-D printed skin; Insight (Max Planck Institute for Intelligent Systems, Germany, 2022) maps force over a thumb-sized curved surface. The images are rich, but cameras run at tens of frames per second, gels wear, and the fingertips are bulky.
- Capacitive and resistive arrays. Grids of taxels whose capacitance or resistance changes when pressed: thin, fast (hundreds of hertz to kilohertz) and able to cover large areas, like the skin of the iCub humanoid at the Italian Institute of Technology, but with millimetre resolution and weaker on shear.
- Magnetic skins. A magnetised elastomer above Hall-effect or magnetometer chips: any deformation, normal or sideways, changes the field. XELA Robotics in Tokyo, a Waseda University spin-out, sells three-axis magnetic taxels that have been fitted to Allegro hands; ReSkin (Meta AI and Carnegie Mellon, 2021) and AnySkin (New York University, 2024) make the skin a cheap, replaceable layer. Nearby magnets and steel disturb them.
- Vibration, multimodal and intrinsic sensing. Piezoelectric films catch slip and texture vibrations at kilohertz rates; SynTouch's BioTac, from research at the University of Southern California, combined pressure, vibration and temperature in a fluid-filled fingertip; and motor currents or joint torque sensors can estimate contact with no skin at all.
Feeling slip before it happens
Touch earns its keep by setting grip force. Too little and the object slips; too much and it is crushed or deformed. People keep their grip a modest margin above the slip limit and adapt it to each object's friction within about a tenth of a second of touching it (Johansson and Westling, 1984). How can a sensor know how close a grasp is to slipping, when the friction coefficient is unknown and changes with wetness and dust?
Contact mechanics answers. Press a rounded elastic fingertip onto a surface and pull it sideways with less than the friction limit: its centre sticks while a ring around the edge already slips.
Theorem— Cattaneo–Mindlin partial slip
For a Hertzian (sphere-on-flat, elastic) contact of radius under normal force , loaded monotonically by a tangential force , the contact divides into a central stick zone of radius and a slip annulus , with
Gross sliding begins when the stick zone vanishes, at .
Cattaneo (1938) and Mindlin (1949) derived this from the elastic tractions in the contact; we take it as given. Its consequence is the point: the fraction of the available friction in use, , can be read from the size of the stick zone without knowing . And a marker-based sensor can see the stick zone: markers there move with the object, while those in the annulus lag behind.
The program simulates a camera-based fingertip with markers every 0.5 mm over a 4 mm contact, under growing sideways load. It does not solve the elasticity problem; its displacement field merely has the right structure (a rigid stick zone of the Cattaneo–Mindlin size, a lagging annulus, 4 µm of tracking noise). An estimator that knows nothing of labels each marker stuck or slipping and reports .
import numpy as np
rng = np.random.default_rng(5)
# Markers printed on the gel of a vision-based tactile sensor, on a 0.5 mm grid; contact radius a = 4 mm.
a, pitch, noise = 4.0, 0.5, 0.004 # mm; tracking noise 4 micrometres
g = np.arange(-6.0, 6.0 + 1e-9, pitch)
X, Y = np.meshgrid(g, g)
pts = np.column_stack([X.ravel(), Y.ravel()])
r = np.linalg.norm(pts, axis=1)
in_contact = r <= a # the contact patch is visible in the tactile image
def marker_field(rho, delta_max=0.4):
"""Simplified sideways marker displacements (mm) for a load ratio rho = F_t / (mu F_n).
Cattaneo-Mindlin: the stuck centre has radius c = a (1 - rho)^(1/3) and moves rigidly by delta;
in the slipping annulus c < r <= a the gel lags behind; outside the contact it relaxes."""
c = a * (1 - rho) ** (1 / 3)
delta = delta_max * (1 - (1 - rho) ** (2 / 3)) # Mindlin's tangential compliance curve
u = np.where(r <= c, delta, delta * (1 - 0.6 * np.clip((r - c) / (a - c + 1e-12), 0, 1)))
u = np.where(r > a, delta * 0.4 * np.exp(-(r - a) / 1.0), u)
disp = np.column_stack([u, np.zeros_like(u)]) # the object is pulled along +x
return disp + rng.normal(0, noise, disp.shape), c
def estimate_stick_ratio(disp, tol=0.012):
"""Markers that move together with the centre are stuck. Iterate: estimate the common
translation from the markers believed stuck, then re-label every contact marker."""
stuck = in_contact & (r <= a / 3) # start from the middle of the patch
for _ in range(10):
t = disp[stuck].mean(axis=0)
new = in_contact & (np.linalg.norm(disp - t, axis=1) < tol + 0.02 * np.linalg.norm(t))
if np.array_equal(new, stuck):
break
stuck = new
return np.sqrt(stuck.sum() / in_contact.sum()) # area fraction -> radius fraction
print(f"{in_contact.sum()} markers inside the contact patch\n")
print(" load ratio rho true c/a estimated c/a rho from the image")
for rho in [0.0, 0.2, 0.4, 0.6, 0.7, 0.8, 0.9, 0.95]:
disp, c = marker_field(rho)
est = estimate_stick_ratio(disp)
print(f"{rho:12.2f} {c / a:12.3f} {est:13.3f} {1 - est ** 3:16.2f}")Near slip, where it matters, the image tracks the load ratio: 0.85 for a true 0.9, and 0.91 for 0.95. At low loads the slip annulus is thinner than the marker spacing and the estimate is poor (0.10 for a true 0.2), which is harmless, because a grasp far from slipping needs no correction. The estimated stick zone is always slightly too large: the sensor errs towards optimism.
A grip controller that feels
Now use it. A robot pinches a paper cup between two fingertips while 200 g of water is poured in, then lifts it briskly. The cup buckles if either finger presses harder than 8 N, and it may be dry () or wet (). Five controllers compete: two fixed grips; one that measures the shear force and grips just enough for an assumed dry ; a slip reflex that watches a 1 kHz shear signal for the micro-vibrations of slip (the root-mean-square of its sample-to-sample changes over 10 ms) and squeezes harder when it finds them; and an incipient-slip controller that reads the stick ratio from a 60 Hz tactile camera, with the bias and noise we just measured, and scales the grip to hold at one half.
import numpy as np
G, DT = 9.81, 0.001 # gravity (m/s^2), 1 kHz control loop
F_MIN, F_MAX, CRUSH = 1.0, 15.0, 8.0 # grip force per finger (N); the paper cup buckles above 8 N
def scenario(t):
"""Mass of the cup (kg) as water is poured in, and the arm's upward acceleration (m/s^2)."""
m = 0.03 + 0.08 * np.clip(t - 0.5, 0.0, 2.5) # 30 g cup + 200 g of water
ramp = lambda t0, T: np.clip((t - t0) / T, 0.0, 1.0)
acc = 4.0 * (ramp(3.5, 0.03) - ramp(3.8, 0.03)) - 2.0 * (ramp(4.2, 0.1) - ramp(4.8, 0.1)) # a brisk lift
return m, acc
def run(controller, mu_s, seed=0):
rng = np.random.default_rng(seed)
mu_k = 0.8 * mu_s # sliding friction is lower
F = 1.0
v = slide = slip_d = peak_F = 0.0
slips = 0
readings, state = [], {"F_cmd": 1.0, "cooldown": 0, "ratio": 1.0}
first_slip = detected = None
for k in range(int(5.0 / DT)):
t = k * DT
m, acc = scenario(t)
W = m * (G + acc) # load the two fingers must carry
if v <= 0.0 and W <= 2 * mu_s * F: # sticking
v, F_t, slip_d = 0.0, W / 2, 0.0
else: # sliding down between the fingers
if slip_d == 0.0:
slips += 1
mu = mu_k + (mu_s - mu_k) * np.exp(-slip_d / 0.0005) # friction falls over 0.5 mm of slip
v = max(0.0, v + (W - 2 * mu * F) / m * DT)
F_t = mu * F
slip_d += v * DT + 1e-12
first_slip = t if first_slip is None else first_slip
slide += v * DT
if slide > 0.03:
return slide, peak_F, "dropped", first_slip, detected, slips
# Tactile signals. Shear force at 1 kHz, with micro-vibrations while sliding (fast channel):
vib = 0.05 * min(1.0, v / 0.005) * rng.normal() if v > 0 else 0.0
readings.append(F_t + vib + rng.normal(0, 0.005))
# Stick ratio c/a from the tactile camera at 60 Hz (slow channel), slightly optimistic:
if k % 17 == 0:
rho = F_t / (mu_s * F) if v == 0 else 1.0
state["ratio"] = float(np.clip((1 - min(rho, 1.0)) ** (1 / 3) + 0.05 + rng.normal(0, 0.02), 0, 1))
F_cmd, hit = controller(readings, state, F, k)
if hit and detected is None and first_slip is not None:
detected = t
F += (np.clip(F_cmd, 0.0, F_MAX) - F) * DT / 0.03 # the finger drive lags by 30 ms
peak_F = max(peak_F, F)
outcome = "crushed" if peak_F > CRUSH else "held"
return slide, peak_F, outcome, first_slip, detected, slips
def fixed(force):
return lambda readings, state, F, k: (force, False)
def assumed_friction(mu_guess, rho_target=0.6):
"""Measure the shear force and grip just enough for a friction coefficient you assume."""
return lambda readings, state, F, k: (max(F_MIN, readings[-1] / (mu_guess * rho_target)), False)
def vibration_reflex(readings, state, F, k, window=10, threshold=0.03):
"""Slip detector: RMS of the first difference of the shear signal over the last 10 ms."""
hit = False
if len(readings) > window:
d = np.diff(readings[-window - 1:])
hit = np.sqrt(np.mean(d ** 2)) > threshold
if hit and state["cooldown"] <= 0:
state["F_cmd"] = min(F_MAX, 1.4 * state["F_cmd"] + 0.3) # squeeze harder, then wait
state["cooldown"] = 40
state["cooldown"] -= 1
return state["F_cmd"], hit
def incipient_slip(readings, state, F, k, rho_target=0.5):
"""Keep the measured load ratio rho = 1 - (c/a)^3 at a target, without knowing mu."""
if k % 17 == 0:
rho_hat = 1 - state["ratio"] ** 3
state["F_cmd"] = float(np.clip(F * rho_hat / rho_target, F_MIN, F_MAX))
return state["F_cmd"], state["ratio"] < 0.05
controllers = [("fixed gentle grip, 2 N", fixed(2.0)), ("fixed firm grip, 10 N", fixed(10.0)),
("shear force, assumes mu = 0.9", assumed_friction(0.9)),
("vibration slip reflex", vibration_reflex), ("stick ratio (incipient slip)", incipient_slip)]
for mu, name in [(0.9, "dry cup, mu = 0.9"), (0.4, "wet cup, mu = 0.4")]:
print(name)
for label, ctrl in controllers:
slide, peak, outcome, first, det, slips = run(ctrl, mu)
lag = f" (first detected after {1000 * (det - first):.0f} ms)" if det is not None else ""
print(f" {label:30} {slips:2d} slips, slid {1000 * slide:4.1f} mm, peak grip {peak:4.1f} N: {outcome}{lag}")- Fixed grips fail in opposite ways: 2 N holds the dry cup and drops the wet one, and 10 N crushes both.
- Shear force with an assumed works on the dry cup. On the wet one, the first slip lowers the measured shear force (sliding friction is weaker), so the controller relaxes its grip and the cup falls: a wrong model, trusted, makes things worse.
- The vibration reflex holds both cups with less force than the stick-ratio controller (2.7 N dry, 4.1 N wet), but only by slipping first: two slips on the dry cup and three on the wet one, the first caught 34 ms and 24 ms after it began. Each moved the cup only a fraction of a millimetre because the load grew gently; faster loads or slower finger drives mean longer slides.
- The stick-ratio controller never lets the cup slip, because it acts while the stick zone shrinks. Its price is a firmer grip, 6.8 N on the wet cup, close to the 8 N at which the cup buckles. The honest conclusion is that two fingertips are the wrong grasp for a full, wet paper cup; a hand should wrap it.
Contact-rich control
Holding is the simplest contact-rich task; insertion, screwing, wiping and reorientation depend on forces at several contacts that change within milliseconds. Four approaches are in use in 2026.
- Impedance at the fingers and at the object. Each finger runs joint-level impedance control (Force and impedance control), and an object-level controller commands motion through while regulating the squeeze in the null space of , as Germany's DLR has done since the 2000s. Most learned policies sit on top of a layer like this.
- Tactile servoing. The tactile image is the error signal: keep a contact centred, an edge aligned, or the stick ratio at a set-point. Nathan Lepora's group at Bristol has shown fingertips following unknown edges and surfaces this way.
- Contact-implicit planning. Contacts are made and broken inside the optimisation. Because contact makes the dynamics non-smooth, MIT's Tao Pang, Terry Suh, Lujie Yang and Russ Tedrake (2022) smoothed a quasi-dynamic contact model so that planners could plan whole-hand and whole-arm manipulation; sampling-based predictive control in MuJoCo (World models) serves similar ends.
- Industrial force control. Factory assembly runs today on a wrist force–torque sensor, low stiffness along the insertion axis, a spiral search when a peg misses its hole, and a check of the force signature that a connector is seated: unglamorous, reliable, and the bar a learned system must clear.
Since 2024, several groups have trained policies that output stiffness or force targets along with positions, soft while searching and firm while pushing; as of late 2026 there is no standard recipe.
Learning dexterity: what works in 2026
In research, learning now drives most work on multi-fingered hands. The routes, with the evidence the labs report:
Reinforcement learning in simulation. The landmarks up to 2022 (OpenAI's Shadow Hand cube and Rubik's Cube, NVIDIA's DeXtreme, MIT's Visual Dexterity) are in Sim-to-real. Since then, Haozhi Qi and colleagues at Berkeley (2022) rotated many objects on an Allegro hand using proprioception alone; Zhao-Heng Yin and colleagues at UC San Diego (2023) did it with simple binary touch sensors and no vision; Qi and colleagues (2023) combined vision and touch; Bristol's AnyRotate (2024) turned unseen objects about arbitrary axes, with the hand facing any way, using dense tactile fingertips; and Berkeley's Twisting Lids Off with Two Hands (2024) took the method to two hands. DexterityGen (Berkeley and Meta, 2025) pretrained a low-level controller on many reinforcement-learned primitives and let a teleoperator steer it, reporting the use of a pen, a syringe and a screwdriver.
Human data. Teleoperating a dexterous hand is hard: the operator feels nothing, and the robot's hand is not theirs. Motion-capture gloves (Stanford's DexCap, 2024), virtual-reality hand tracking (Imitation learning) and human video supply data, and retargeting human finger motion onto a different hand is a research topic of its own.
Touch as a learned representation. Tactile images differ between sensors and even between two gels of one design. Meta's Sparsh (2024) pretrained self-supervised models on more than 460,000 tactile images from several sensors, and simulators such as TACTO render tactile images for training, though simulated gels still differ from real ones.
Touch in vision-language-action models. The VLAs of 2026 mostly see and do not feel. Research systems add touch as further input tokens to the transformer (see Attention and the transformer and Language models): Tactile-VLA (Tsinghua University and collaborators, 2025) feeds tactile and force signals into a VLA and outputs force targets as well as positions; OmniVTLA (Shanghai Jiao Tong University and the sensor maker PaXini, 2025) aligns tactile signals with visual and language concepts. No recipe has become standard. Among the few published success rates for five-fingered manipulation by a generalist model are Google DeepMind's for Gemini Robotics 2 (July 2026), on a humanoid with 22-degree-of-freedom hands: 92% for unscrewing a light bulb, 36% for screwing one in.
What is repeatable, and what is demonstration, as of late 2026:
| Repeatable, with published protocols | Demonstrated, not yet reliable | Open |
|---|---|---|
| in-hand rotation and reorientation of many objects in the lab | autonomous tool use with dexterous hands outside the lab | hands that survive months of contact at low cost |
| tactile estimation of contact shape, force and slip on research sensors | long dexterous tasks at factory reliability | tactile skins that do not wear, drift or need recalibration |
| force-controlled insertion and assembly in factories | humanoid hand tasks in videos without stated trial counts | touch data at scale, and a representation shared across sensors |
| power and pinch grasps on humanoids in pilots | general manipulation of deformable objects | simulating soft contact and tactile sensors well enough to transfer |
When you read a dexterity claim, ask whether the video is autonomous or teleoperated, at what playback speed, how many trials were run with what success rate, and whether the objects and scenes were seen in training. Responsible labs report all of these.
Common misconceptions
Misconception
"More degrees of freedom make a more dexterous hand." Joints without sensing, control and robustness add weight, cost and failure points. In-hand manipulation does need independent joints, as the counting theorem shows, but they are necessary, not sufficient: a six-actuator hand that feels can beat a twenty-joint hand that cannot on many factory tasks.
Misconception
"A tactile sensor measures force." Most measure deformation or geometry and infer force by calibration or learning. Touch also reports contact location, shape, the distribution of shear, vibration and temperature, and in manipulation the distribution often matters more than the total.
Misconception
"With good cameras, touch is unnecessary." At the moment of contact the fingers hide the contact from the cameras, friction is invisible, and slip begins in micrometres and milliseconds. People with numbed fingertips see their hands perfectly well and still fumble.
Misconception
"To stop slipping, detect slip and squeeze." By the time slip is detected the object has moved, which matters for a pen or a key, and squeezing hard enough to be safe from every slip crushes fragile things. A margin read from the stick zone, with a fast reflex as backup, does better.
Misconception
"Sim-to-real has solved in-hand manipulation." It has made in-hand rotation and reorientation repeatable on research hands. Tool use, deformable objects, long tasks and production reliability remain open, partly because soft contact and tactile sensors are still hard to simulate.
A short history
Tokuji Okada's three-fingered hand at Japan's Electrotechnical Laboratory (1979) was among the first computer-controlled multi-fingered hands. Salisbury's Stanford/JPL hand (1982) set the three-finger, three-joint pattern, and Stephen Jacobsen's tendon-driven Utah/MIT Dexterous Hand (mid-1980s) had four fingers with sixteen joints. DLR's hands in Germany, the Shadow hand, NASA's Robonaut hand and the high-speed hand of Masatoshi Ishikawa's laboratory at the University of Tokyo followed. Tactile sensing grew alongside, from piezoresistive pads to the iCub skin, the BioTac and GelSight (2009); learning arrived at scale with OpenAI's in-hand cube in 2018.
Exercises
1. Dexterous hands and touch
Warm-upHumanA robot holds a 400 g jar upright between two fingertips that squeeze horizontally, with friction coefficient at each contact and . What is the smallest normal force per finger, in newtons (rounded to the nearest whole number), that keeps the jar from slipping while it is held still?
2. Dexterous hands and touch
CoreHumanA rounded fingertip presses on an object with normal force N, friction coefficient , and carries a tangential load N. Using the Cattaneo–Mindlin result, what is the radius of the stick zone as a percentage of the contact radius, , rounded to the nearest whole number?
3. Dexterous hands and touch
CoreHumanA hand holds an object in three dimensions with four fingertips, each touching at one sticking point contact with friction. What is the smallest total number of joints in the four touching fingers for which the grasp can be manipulable, that is, for which the fingers can give the object any small motion while the contacts stick?
4. Dexterous hands and touch
CoreHumanA gripper should notice the onset of slip from the micro-vibrations it causes, which in this task are strongest between 100 and 400 Hz. Which sensor is best suited to that signal?
5. Dexterous hands and touch
CoreHumanWhy can a controller that reads the stick-zone radius from a marker-based tactile sensor keep a grasp away from slipping without knowing the friction coefficient ?
6. Dexterous hands and touch
CoreHumanA company posts a video of its humanoid's five-fingered hand peeling a clementine. Which facts do you need before concluding what the hand can do? (Choose all that apply.)
7. Dexterous hands and touch
CoreComputationalWrite the fast slip detector of the lesson's grip controller. detect_slip(signal, window, threshold) takes a sequence of shear-force readings (one per millisecond) and returns the index of the first sample, with window, at which the root-mean-square of the window sample-to-sample differences of signal[i - window : i + 1] exceeds threshold; it returns None if that never happens. The tests check that noise and a smooth change of load do not trigger it, that slip vibrations do, and what happens with a sudden step.
8. Dexterous hands and touch
CoreComputationalWrite grasp_matrix(points, centre) for planar point contacts. For contacts at points and torques taken about centre , it returns the matrix that maps the stacked contact forces to the net wrench on the object, with for . The tests also check that maps an object rotation to the contact-point velocities.
Further reading and resources
- Coding and use of tactile signals from the fingertips in object manipulation tasks · Roland S. Johansson and J. Randall Flanagan · 2009How human fingertips signal contact, slip and load, and how the brain uses those signals to set grip force. Nature Reviews Neuroscience 10.
- Modern Robotics: Mechanics, Planning, and Control · Kevin M. Lynch and Frank C. Park · 2017Chapter 12 covers contact kinematics, the grasp matrix and closure; the preprint is free.
- GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force · Wenzhen Yuan, Siyuan Dong and Edward H. Adelson · 2017The vision-based tactile sensor from MIT, with its shape, force and slip algorithms; open access.
- DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation · Mike Lambeta et al. (Meta AI) · 2020
- Digitizing Touch with an Artificial Multimodal Fingertip · Mike Lambeta et al. (Meta FAIR) · 2024Digit 360, a fingertip that adds vibration, heat and other channels to a vision-based core, with on-device processing.
- The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies · Benjamin Ward-Cherrier et al. (Bristol Robotics Laboratory) · 2018
- A soft thumb-sized vision-based sensor with accurate all-round force perception · Huanbo Sun, Katherine J. Kuchenbecker and Georg Martius (Max Planck Institute for Intelligent Systems) · 2022Insight, a camera-based sensor that reports a force distribution over its whole curved surface.
- ReSkin: versatile, replaceable, lasting tactile skins · Raunaq Bhirangi, Tess Hellebrekers, Carmel Majidi and Abhinav Gupta · 2021
- AnySkin: Plug-and-play Skin Sensing for Robotic Touch · Raunaq Bhirangi et al. · 2024
- LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning · Kenneth Shaw, Ananye Agarwal and Deepak Pathak · 2023
- DEX-EE · Shadow RobotA three-fingered research hand designed with Google DeepMind to survive long learning experiments.
- In-Hand Object Rotation via Rapid Motor Adaptation · Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma and Jitendra Malik · 2022
- Rotating without Seeing: Towards In-hand Dexterity through Touch · Zhao-Heng Yin, Binghao Huang, Yuzhe Qin, Qifeng Chen and Xiaolong Wang · 2023
- General In-Hand Object Rotation with Vision and Touch · Haozhi Qi et al. · 2023
- AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch · Max Yang et al. (University of Bristol) · 2024
- Twisting Lids Off with Two Hands · Toru Lin, Zhao-Heng Yin, Haozhi Qi, Pieter Abbeel and Jitendra Malik · 2024
- DexterityGen: Foundation Controller for Unprecedented Dexterity · Zhao-Heng Yin et al. (UC Berkeley and Meta FAIR) · 2025
- DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation · Chen Wang et al. (Stanford) · 2024
- Sparsh: Self-supervised touch representations for vision-based tactile sensing · Carolina Higuera et al. (Meta FAIR) · 2024
- TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors · Shaoxiong Wang, Mike Lambeta, Po-Wei Chou and Roberto Calandra · 2020
- Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models · Tao Pang, H.J. Terry Suh, Lujie Yang and Russ Tedrake (MIT) · 2022
- Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization · Jialei Huang et al. · 2025
- OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensing · Zhengxue Cheng et al. (Shanghai Jiao Tong University and PaXini) · 2025
- Gemini Robotics 2 brings whole-body intelligence to robots · Google DeepMind · 2026Includes success rates for five-fingered manipulation tasks on a humanoid.