Lumenary Scientia
Lumenary Robotics
Robotics from a first moving robot to the learning-based humanoids of 2026: motion, control, perception, planning, real robot systems, robot learning and the frontier.
Lumenary Robotics is the fourth subject of Lumenary Scientia, free and open source like the others. It starts with no experience at all — what makes a machine a robot — and climbs through the mathematics of motion, control, perception and state estimation, and planning, to how real robots are built and how robots learn, ending with the vision-language-action models and humanoids that research labs and companies are building in 2026.
Every lesson teaches with ideas first and code second. Robots are simulated in Python with NumPy, and every block with a Run in browser button runs there, with nothing to install: you will drive simulated robots, fuse noisy sensors, plan paths and train small policies yourself. A lesson says Code tested only when all of its code passed. Lumenary Robotics builds on Lumenary CS for programming, algorithms and machine learning, and on Lumenary EE for circuits, motors and control.
Real robots can hurt people. Lessons that touch hardware say what is safe to try — small, slow, battery-powered robots — and why industrial machines are fenced, rated and certified.
Every lesson was drafted with AI assistance and is awaiting review by a human roboticist. The text, exercises and code are original and licensed CC BY-SA 4.0 (code Apache-2.0). Where a lab, company or project has published the primary source — a paper, a dataset, a design document — the lesson links to it rather than retelling it.
- Chloe · The Builder
“How do we get a robot to do this?” - Margaret · The Experimenter
“What happens when we simulate it?” - Gideon · The Safety Engineer
“What could go wrong?”
Start · 5 lessons
What robots are
No experience needed, from about age 11. What makes a machine a robot, how robots sense the world and move in it, your first robot program in a simulated world, and how robots and people work safely together.
- 1What is a robot?Machines that sense, think and act, from factory arms to Mars rovers, and the 1920 play that gave them their name. 40 min
- 2How robots sense the worldWheel counters, bump switches, echoes, lasers and cameras, and why a careful robot never trusts a single reading. 45 min
- 3How robots moveWheels, tracks, legs, propellers and arms; the motors and gears that drive them; and the batteries that run out. 45 min
- 4Your first robot programA robot in a grid world, in Python — moving, turning, feeling for walls, escaping a maze and steering round obstacles. 60 min
- 5Robots and peopleWorking safely beside robots, robots in care and at home, jobs, privacy, weapons, and who is responsible when a robot gets it wrong. 45 min
Motion · 7 lessons
The mathematics of motion
Describing where things are and how they move: frames and rigid transforms, rotations and quaternions, wheeled robots and odometry, forward and inverse kinematics of arms, Jacobians, and dynamics.
- 6Frames and rigid transformsWhere is that, and according to whom? Points, frames and the 3×3 and 4×4 matrices that carry coordinates from a sensor to a robot to the world. 60 min
- 7Rotations in 3-D and quaternionsNine numbers, three degrees of freedom, and many ways to write them down. Rotation matrices, Euler angles and their gimbal lock, axis–angle, and the quaternions that most robot software uses. 70 min
- 8Wheeled robots and odometryHow wheel speeds become motion, why a car cannot drive sideways yet can still park, and how counting wheel turns tells a robot where it is, for a while. 65 min
- 9Forward kinematics of robot armsFrom joint angles to where the gripper is. Kinematic chains, Denavit–Hartenberg parameters, the product of exponentials, workspaces, and the URDF files that describe real robots. 70 min
- 10Inverse kinematicsWorking backwards from where the hand must be to how every joint must turn — and why the answer may be none, two, eight or infinitely many. 75 min
- 11Jacobians, velocity and singularitiesOne matrix turns joint speeds into tool speeds and tool forces into joint torques — and tells you exactly where an arm goes weak. 75 min
- 12Robot dynamicsWhy kinematics is not enough — the equations of motion of an arm, derived, simulated and checked against the conservation of energy. 80 min
Control · 5 lessons
Control
Making a robot do what we mean: feedback and PID, state-space models, LQR and observers, trajectories and tracking, force and impedance control, and model predictive control.
- 13Feedback and PID controlMeasure, compare, correct, a hundred times a second — the loop that makes a robot joint go where it is told and stay there. 70 min
- 14State space, LQR and observersDescribe a robot by its state, balance an inverted pendulum with one gain matrix, choose that matrix optimally, and estimate what you cannot measure. 80 min
- 15Trajectories and trackingPlan how a joint should move through time, within its speed and torque limits — then make the arm follow, using its model as well as its errors. 75 min
- 16Force, impedance and compliant controlWhen a robot touches the world, controlling position is not enough — control how it pushes back, and build actuators that can give way. 75 min
- 17Model predictive controlLook ahead, plan within the limits, take one step, look again — the controller behind walking robots, autonomous cars and agile drones. 80 min
Perception · 6 lessons
Perception and state estimation
Knowing the world under uncertainty: Bayes filters, Kalman filters, particle filters and localisation, cameras and computer vision, depth sensing and point clouds, and simultaneous localisation and mapping.
- 18Uncertainty and Bayes filtersA robot never knows exactly where it is. It can know how unsure it is, and update that every time it moves and every time it looks. 60 min
- 19Kalman filtersWhen beliefs are Gaussian, the Bayes filter becomes a few lines of linear algebra. Tracking, tuning, nonlinear sensors, consistency checks and fusing an IMU with wheel odometry. 75 min
- 20Particle filters and localisationRepresent a belief by thousands of guesses, keep the ones the sensors agree with, and a robot can find itself in a building that looks the same in every direction. 70 min
- 21Cameras and computer vision for robotsFrom the pinhole model to pixels, from markers to poses, and from detected features to the first step of visual odometry, with a look at what deep learning now does for robot vision. 80 min
- 22Depth sensing and point cloudsGetting the third dimension back. Stereo, structured light, time of flight and lidar; point clouds and how to align them with ICP; and occupancy maps built one beam at a time. 75 min
- 23Simultaneous localisation and mappingBuilding a map while working out where you are in it. Why it is hard, how EKF-SLAM did it, how pose graphs and factor graphs do it today, and where neural scene representations fit in 2026. 80 min
Planning · 5 lessons
Planning and decision-making
Deciding where to go and what to do: search on graphs and grids, sampling-based planners, configuration spaces and collision checking, trajectory optimisation, and task planning with behaviour trees.
- 24Planning on graphs and gridsFrom a map to a route — occupancy grids as graphs, A* with the right heuristic, costmaps that keep a robot off the walls, replanning when the world changes, and lattices for robots that cannot turn on the spot. 75 min
- 25Sampling-based planningWhen the space is too big to grid — probabilistic roadmaps, rapidly-exploring random trees, RRT-Connect and RRT*, measured in code, with the narrow passages that defeat them and the libraries that run them on real arms. 75 min
- 26Configuration space and collision checkingTurning a robot with a shape into a point in a space of its own — the configuration space of an arm, Minkowski sums, and the bounding volumes, separating axes, GJK and distance fields that answer "does it touch?" millions of times a second. 75 min
- 27Trajectory optimisationPlanning as optimisation — costs, constraints and dynamics; shooting and collocation; a CHOMP-style optimiser that pushes a path out of obstacles; the fastest way to drive a fixed path; and why the initial guess decides the answer. 80 min
- 28Task planning and behaviour treesFrom where to go to what to do — symbolic planning with STRIPS and PDDL, a breadth-first planner for a kitchen, behaviour trees that react and retry, task and motion planning, and what language models can and cannot do as planners. 75 min
Systems · 8 lessons
Building real robots
From parts to working machines: ROS 2 and robot software, simulation, embedded control and functional safety, manipulation and grasping, legged robots, aerial robots, autonomous vehicles, and teams of robots.
- 29ROS 2 and robot softwareMany programs, many clocks, real hardware. How robot software is organised into nodes that talk through topics, services and actions, and how teams log, replay and test it. 70 min
- 30SimulationA robot that lives in a computer. How physics engines step time forward, why the choice of integrator decides whether energy leaks or grows, how contact and friction are faked, and how far to trust the result. 70 min
- 31Embedded control and functional safetyBelow the robot's main computer sit microcontrollers, motor drives and fieldbuses running loops thousands of times a second, and a separate safety system whose only job is to stop the machine before it hurts someone. 75 min
- 32Manipulation and graspingPicking things up is the oldest job in robotics and still one of the hardest. Grippers, friction cones, a test for whether a grasp can resist any push, and the pipeline that turns a cluttered bin into a stream of successful picks. 70 min
- 33Legged locomotionWalking is falling, caught in time. Support polygons, the zero-moment point, the linear inverted pendulum and the capture point, gaits, the controllers inside today's walking robots, and the motors that made them possible. 70 min
- 34Aerial robotsFour propellers, no wings, and an unstable machine that a computer keeps upright hundreds of times a second. Thrust and motor mixing, cascaded flight control, attitude estimation, flight stacks, navigation with and without GNSS, fixed wings and VTOL, and flying within the law. 70 min
- 35Autonomous vehicles and field robotsRobots that drive among people, and robots that work on farms, in mines and around industrial plants. The driving stack, the SAE levels, a path-tracking controller built and tested, sensors, safety cases, where robotaxis stand in 2026, and the field robots already at work. 70 min
- 36Multi-robot systemsMany robots, one job. Who decides, who does which task, how a group agrees and moves together, how swarms emerge from simple rules, how robots share corridors without collisions, and what warehouses with thousands of robots have learned. 70 min
Learning · 4 lessons
Robot learning
Robots that learn from experience and from demonstration: reinforcement learning for control, imitation learning, crossing the gap between simulation and reality, and the large datasets behind today's generalist policies.
- 37Reinforcement learning for robot controlLearning a controller from reward and experience — Markov decision processes, Q-learning, policy gradients, PPO and SAC, the art of writing rewards, and why robots do their practising in simulation. 80 min
- 38Imitation learningTeaching a robot by showing it — behaviour cloning, why small mistakes snowball, DAgger, demonstrations that disagree, action chunking and diffusion policies, and the teleoperation rigs that collect the data. 80 min
- 39Crossing the gap from simulation to realityPolicies trained in simulation meet a real robot that is heavier, slower, stickier and later than the model. System identification, domain randomisation and teacher–student training — and the robots that crossed the gap. 75 min
- 40Robot data at scaleThere is no internet of robot actions. How robot data is collected, the shared datasets of 2023–2026, training one policy on many robots, what the scaling evidence does and does not show, and why measuring a robot policy is harder than training it. 75 min
Frontier · 5 lessons
The frontier of robotics
Where robotics stands in 2026, with links to the labs' own papers: vision-language-action models, humanoid robots, world models, dexterous hands and touch, and how learned robots are evaluated and deployed safely.
- 41Vision-language-action modelsHow a vision-language model learns to move a robot — action tokens and their error, flow-matching action experts, co-training on web and robot data — and what the generalist policies of 2026 can and cannot do. 85 min
- 42Humanoid robotsWhy build robots in our shape; what is inside them — actuators, hands, batteries — and what their energy budget allows; how they are controlled; who builds them, where they work in 2026, and what is still missing. 80 min
- 43World modelsLearned models of how the world responds to actions — dynamics models and model-based reinforcement learning, video models as world simulators for data and evaluation — and planning with a learned model by random-shooting MPC, implemented, with its limits. 80 min
- 44Dexterous hands and touchMulti-fingered hands and the kinematics of turning an object in the fingers; the tactile sensors that let robots feel, from camera-based gels to magnetic skins; a slip detector you build and test; and what learned dexterity can and cannot do as of late 2026. 85 min
- 45Evaluation, safety and deployment of learned robotsHow generalist robots are measured and how sure a measurement can be; how a learned policy is wrapped in safety filters and runtime monitors; the standards and laws that apply in late 2026; and the people who supervise robots, work beside them, and whose work they change. 85 min
Further reading and resources
- MIT OpenCourseWare: Mechanical Engineering and EECSFree lecture notes and videos, including robotics, control and underactuated robotics courses.
- Underactuated Robotics · Russ TedrakeA free online textbook on the control of walking, flying and swimming robots.
- Modern Robotics · Kevin M. Lynch and Frank C. ParkA textbook with free preprint, videos and software, on kinematics, dynamics, planning and control.
- ROS 2 documentationThe Robot Operating System, the open-source software framework most research robots use.
- MuJoCoAn open-source physics engine widely used for robot simulation and learning.