The Washington PostDemocracy Dies in Darkness

Watch a human musician and his robots improvise together

This performance by Shimon and the Shimi Robots showcases the PhD research of Georgia Tech doctoral student Mason Bretan on machine improvisation, path planning and embodied cognition. (Video: Mason Bretan/YouTube)
Placeholder while article actions load

Rest assured that when our future robotic overlords come on the scene, they'll have a sweet sense of rhythm.

The Robotic Musicianship Group at Georgia Tech has been working on Shimon, a musical robot that can improvise melodic accompaniment, for about six years now. And for three years, they've added Shimi — a small, smartphone-connected bot that can respond to music with dance and sound — to the mix.

Shimon and the Shimis (which is a great band name, by the way) are showcased in the above video, in which they jam along with one of their creators, PhD student Mason Bretan. He gave them an arrangement of what he'd be playing and recorded some tracks and cues for them, but in between (when you hear funky electronic noises) they're doing their own thing based on his chord progressions. And the mallet solo in the middle is completely robot-improvised.

When Bretan joined the lab five years ago, Shimon was being taught how to compose jazz music on the fly based on music theory. "I jumped right in," Bretan said. "And with Shimi — which don't just generate music, we call them 'robotic musical companions' because you can talk to them and use them to interact with your playlist — with Shimi I've been there from the start."

"I'm always trying something new with the robots, and sometimes they surprise me with something that's sort of out there or pretty cool," he added.

His dissertation, which he hopes to turn in by the end of 2015, centers around teaching the robots to understand their physical constraints and abilities.

"So the goal is that if you gave the same input to a robot with 20 arms, it would perform differently than an eight-armed robot because it would be optimizing its performance," he said."Combined with the new algorithm we have for jazz music improvisation, these skills really allow them to more optimally achieve musical goals."

And while he certainly doesn't want to replace human musicians like himself with robots, he's excited about the mechanical abilities they have that we don't.

"I mean, Shimon already has four arms and can hold eight mallets," he said, "So it can already do something a person can't."