Watch a Robotic Hand Play the Piano With a More Human Touch

https://www.nytimes.com/2018/12/19/science/piano-robot-hand.html

Version 0 of 1.

Scientists and engineers have been building and programming piano-playing robots for decades. But there’s something different about a new robotic hand that tickles the ivories with techniques usually reserved for humans.

Rather than relying on a maze of motors, as decades of piano bots have, the hand — introduced Wednesday in a paper in Science Robotics — operates passively, meaning its fingers are not individually connected to any motors. Instead, a simple mechanical arm controls only the wrist, and the rest of the hand follows in a design inspired by human anatomy.

“Our aim is to move away from the traditional approach in robotics, where one motor gives one behavior, because that doesn’t scale,” said Josie Hughes, a researcher in the Department of Engineering at the University of Cambridge who led the development of the hand.

Her hand, for now, is far from mastering Chopin or Debussy. But it’s capable of performing different styles and dynamics, and it can eke out a decent “Jingle Bells:”

The research is another step in the effort to design soft robots that move with the nuance of bodies, said Cecilia Laschi, a professor at the BioRobotics Institute of Scuola Superiore Sant’Anna in Italy, who was not involved in the study. Developments in this field might someday lead to robots that can explore natural environments or perform medical diagnoses by touch.

[Like the Science Times page on Facebook. | Sign up for the Science Times newsletter.]

For inspiration, Ms. Hughes and her collaborators turned to the human hand.

“We don’t just have intelligence in our brain — we have intelligence distributed throughout the whole of our body,” Ms. Hughes said, adding that much of “intelligence arises from physical structure and mechanics.” For that reason, her team designed a mechanical hand with bones and ligaments placed as they are in nature.

Next, they evaluated how hard or soft different components needed to be. In the absence of motors, these varying levels of stiffness generate complexity. “We wanted to replicate how the human hand is strong and powerful, but also capable of performing light and delicate movements,” Ms. Hughes said.

The researchers fabricated the hand with a 3-D printer that blended hard plastic and soft rubber in different ratios to create ligaments and joints with varying degrees of stiffness. Then they attached the hand to a robot arm that is commonly used in industrial assembly lines.

Her team broke piano playing into three types of movement: single finger strikes, jumps and sliding thumb movements. They used three musical excerpts to gauge their robot’s abilities.

The first, from “Toccata,” by the Italian composer Scarlatti, requires a series of punctuated, or staccato, raps, usually performed with the index finger. The engineers found that keeping this forefinger very stiff came closest to approximating how a human would play the notes. They made the surrounding digits less stiff, to ensure that the robot could also play progressions that were soft and slurred, or legato.

Next, the researchers used “Alligator Crawl,” by the American jazz pianist Fats Waller, to study their robot’s ability to play octaves. Big jumps, they found, require stretchy ligaments in the valleys between fingers but moderately stiff joints in the fingers themselves to ensure clear notes on landing.

Finally, Ms. Hughes’s team turned to the thumb, an especially complex digit. “Rhapsody in Blue,” by the American composer George Gershwin, helped them model how a thumb skates across keys to create a slide in pitch, called a glissando. Low stiffness in the thumb joints — the lowest the researchers tested — helped create a smooth glissando. It also allowed the index finger to cross over the thumb, a tricky maneuver that pianists employ often to flow between keys more easily.

Compared with many previous piano bots, this one showed flexibility in volume control, thumb movements and playing both staccato and legato, Ms. Hughes said. She next plans to add sensors, motors and components that act like tendons, which will help create a wider range of behaviors.

Ultimately, she hopes to create a robot that can play piano expressively, rather than just focusing on rigid accuracy.

“We’re going beyond simply treating notes as a list of instructions,” she said. “Piano playing is an art. We’re trying to introduce that complexity, depth and artistry into our robot’s behavior.”