← All articles

Mikhail Polyakov

A robotic hand has learned to walk on its own fingers. Part 1: what it can do and how reliably

Level: BeginnerReading time: 7 min

Topics: Robots, AI, Science

A metal five-fingered robotic hand standing on its fingertips on a laboratory bench, one finger lifted mid-step, with a small module on the back of the hand

Short answer: researchers at the Soft Robotics Lab at ETH Zurich took an off-the-shelf five-fingered robotic hand, changed neither its fingers nor its built-in controller, added an 80-gram module with a battery, a tilt sensor and a tiny computer to the back of it, and trained it in four skills: crawling on its fingertips, steering, recovering from falls, and working with objects using those same fingers. The result is a robot weighing 818 grams that moves untethered and without an external computer. The average speed across the measured paths is about 9 centimetres per second. The work was posted as a preprint on 15 September 2026, and on 8 October IEEE Spectrum covered it. Below: what it can do, with which numbers, and what it cannot. There is a separate continuation about how the training itself works.

Why a hand would walk

The authors explain the idea as follows. A hand with mobility of its own could work in confined spaces without requiring the robot arm that normally carries it to follow. A larger robot could place such a hand on a support surface near a restricted opening; from there it could move to the control or object it needs, do the job and return to be retrieved.

No separate locomotion mechanism is needed for this: the same fingers that later grasp objects do the walking. The authors themselves compare the idea to Thing from The Addams Family.

The difficulty is just as clear: the very same fingers have to move the body, support its weight and interact with the environment at the same time. The authors note this in their review of other work as well: a leg used for manipulation is no longer available for support. With their hand the fingers provide all the body support, so every key press or object push has to be combined with keeping balance.

What this robot is built from

An off-the-shelf WUJI right hand was used, and that is an important part of the idea: its fingers and its built-in position controller were not modified.

  • The hand weighs 738 grams. The module on its back adds 80 grams, so the robot comes to 818 grams.
  • The module holds a Raspberry Pi Zero 2 W single-board computer, a BNO085 sensor (it measures tilt and rotation) and a four-cell lithium-polymer battery. That is what makes the robot untethered.
  • The hand has 20 actuated joints, four per finger.
  • The control program runs on that computer 50 times a second. Crawl commands arrive over a separate radio link from a gamepad.

Each task has its own trained policy: crawling separately, fall recovery separately, key pressing separately, object pushing separately. All of them were trained in a simulator and then transferred to hardware.

What it can do: the numbers

Here it matters what was obtained on the real robot and what in simulation. The authors distinguish the two, and so do I.

Crawling (hardware). The hand crawled untethered across 14 indoor and outdoor surfaces: from smooth flooring to metal grating, grass and gravel. The authors themselves call these runs a qualitative demonstration and show them in the supplementary video, which means no success rates per surface are given there.

Speed (hardware). The average speed across 21 measured paths was 0.093 metres per second, that is, about 9 centimetres per second. Steering was evaluated on a 1.3-metre mat, with position measured by an overhead camera.

Steering (hardware). Without a steering command the hand drifts right by about 6 degrees per second, and a constant correction produces nearly straight travel. In five heading-step trials the hand reached the target heading for 15-degree steps in both directions and for a 30-degree right turn. The 30-degree left turns fell short, and the reason is stated plainly: the offset that cancels the rightward drift also adds to left-turn commands.

Fall recovery (hardware). Successful in 21 cases out of 25. For falls onto the thumb side, 11 out of 14; onto the wrist side, 10 out of 11. In the four failures the fingers caught on each other and the hand got stuck.

Fall recovery (simulation). The hand rights itself in 28 cases out of 32, with a median time of 6.1 seconds. In the remaining four the upright detector did not fire within the 20 seconds allowed.

Key pressing (hardware). 29 correct presses out of 32 successive commands in 72.5 seconds, without realigning the keyboard in between. The median latency from command to press is 0.25 seconds, and the greatest tilt of the hand during this is 7.7 degrees. All three errors fell on the "up" command, where a neighbouring key was pressed instead.

Object pushing (hardware). A PLA plastic cube with a 40-millimetre side and a mass of 41.4 grams. 15 deliveries over distances from 10 to 40 centimetres, with a mean final position error of 17 millimetres, ranging from 5 to 37. A delivery counted as follows: enter a circle of 2-centimetre radius, then stay within 5 centimetres for one second.

What it cannot do

This is the most useful part, and the authors do not hide it in footnotes.

  • It has no camera of its own. It presses keys without vision: there are four learned press locations, and before a block of trials an operator aligns the keyboard to those locations using trial presses. If the keyboard is moved, manual realignment is needed.
  • Object pushing relies on an overhead camera. The camera is external, so outside its working area the task is not solved.
  • 30-degree left turns fall short, for the reason described above.
  • The speed is modest. 0.093 metres per second is about 5.6 metres per minute.

In their conclusions the authors name directly what is missing: extending the range of commanded heading changes and integrating onboard perception, which would broaden where the hand can operate. Visual registration could automate keyboard alignment, while onboard tracking of the hand and the object could support pushing beyond the overhead camera's workspace.

It is also worth remembering that this is a preprint: the work was posted by the authors on 15 September 2026, and the arXiv page gives no indication that it has been peer reviewed in a journal.

What is interesting here beyond robotics

Two things worth taking away even if robots are of no use to you.

First: a part of a robot can be a robot. The familiar arrangement is a body with arms hanging off it. Here one part takes on both roles instead, and no separate locomotion mechanism is needed. That is a different way of thinking about a machine: not "what to add" but "what is already there and could work differently".

Second: the hardware was not modified. They took an off-the-shelf hand with its fingers and its built-in controller and obtained new behaviour through training. That is a different framing of the task: not "design a new device" but "control the existing one differently".

If you are weighing such a development against a task of your own, the order is simple. First watch the video the authors supplied and compare it with the numbers from the preprint: a demonstration across fourteen surfaces and measured trials are different things. Then compare your own conditions with the four limitations above: do you need operation without an external camera, do you need more than 5.6 metres per minute, are four failed recoveries out of twenty-five acceptable. If even one does not pass, compare the available options by those same four yardsticks: travel speed, dependence on an external camera, the precision with which your task gets solved, and the share of failures you can tolerate. The winner is whatever passes on all four, and you can return to this development when the authors close the gaps they themselves have named.

How recordings of real human actions used to train robots are collected, I went through in the article how data for training robots is collected. There is another piece on a scientific set of human motion recordings: the HiPHI dataset.

In summary

Researchers at ETH Zurich have shown that an off-the-shelf robotic hand can become a robot in its own right: 818 grams including battery and computer, crawling on its fingertips across 14 surfaces, fall recovery in 21 cases out of 25, 29 correct key presses out of 32, and 15 cube deliveries to a target with a mean error of 17 millimetres. The fingers serve as both legs and hands, and the design of the hand was not changed. The limitations are named honestly too: no vision of its own, pushing relies on an external camera, 30-degree left turns fall short, and the speed is about 9 centimetres per second.

How exactly it was taught all this, and how the training reward is built, I go through in the continuation: Part 2: how it was taught to walk.

I work on AI agents and automation. If you would like to see my projects or discuss your own task, take a look at my portfolio.

Sources