ETH Zurich researchers made a commercial robotic hand capable of walking by equipping it with a battery and a Raspberry Pi Zero 2 W. The twenty-joint hand uses its fingers like legs with neural network-based software; it moves across terrain, pushes lightweight objects toward a target, and presses keyboard keys.
Trained using reinforcement learning with a simulation model, the system walked faster than an adapted quadrupedal locomotion model. The hand was trained to move straight ahead, turn, recover from a fall, and press keys, but it cannot use a keyboard on its own because it has no camera. It righted itself within twenty seconds in 21 of 25 tests. Because it drifted to the right while walking, constant correction was required; nevertheless, it traversed fourteen different surfaces, ranging from a rubber mat to gravel and grass.
Why it matters
The study opens up discussion of the limitations of designs focused solely on grasping tasks by showing that hand and locomotion capabilities can be combined on the same robotic platform. For the researchers, the key criterion is that tasks such as moving across different surfaces, recovering after a fall, and interacting with objects can be carried out within a single system; these results indicate that these capabilities can be evaluated together. However, the absence of a camera prevents the system from perceiving its surroundings and using a keyboard, while its drift to the right requires constant correction during movement. The open question, therefore, is to what extent learned movements can become autonomous and precise when combined with perception capabilities.