Kinect-powered orchestra

Could a symphony orchestra keep it together without the guy in the penguin suit waving his hands around in front of them? Probably. But with the Computer Orchestra, a project by a trio of students at ECAL, an arts and design school in Switzerland, the human conductor’s presence is vital. In fact, since all the players are laptops, the conductor is the only human involved.

The project, created by Simon de Diesbach, Jonas Lacôte, and Laura Perrenoud, involves one conductor, one Kinect, and up to 24 laptops each loaded with a single sample. Those fragments, which can be selected from a preloaded library or snapped up from an in-the-works crowdsourced platform, are triggered purely by the movements of the conductor’s hands. Each instrument is mapped to a section of 3-D space — move your hands through them and the samples play; hold a hand in one zone and that computer will continue to play with increasing volume. It’s a chance to see how easy — or, probably, how hard — it really is to do the job, though in this case the system was designed with harmonious output in mind. “No special skills are required to make a great performance,” the students explain. “Anyone could do it.”

But the project doesn’t just re-imagine the orchestra from the conductor’s perspective. It offers a unique listening experience, too. The designers hope to exhibit the project live at some point in the future, because seeing it in person, they say, is completely different than watching it on a screen. “Each sound being played by one computer; it’s an orchestra you can walk through, that you can experience from the inside,” the team explains. “The sound will be completely different depending where you stand in the room.”

Getting all the components just right, from the motion detection zones to the overall sound design, was a challenge, but the hardest part was making the final video to show it off. “We had to borrow computers from friends, keep them charged all the day long, install the app on each post, build the setup … it was a huge mess to organize,” they explain. Wrangling any 24 computers isn’t an easy feat, no less 24 in the same outfit.