Sound Bytes: Techno-Debate Is Just Sosolimited

By David Segal
Washington Post Staff Writer
Tuesday, October 14, 2008

NEW YORK -- It's become a cliche to talk about the modern political debate as a carefully stage-managed production filled with talking points and scripted zingers. To get beyond the artifice, voters have for years been forced to rely on their ears, eyes and common sense.

Isn't that quaint?

Well, it seems quaint after an evening with Sosolimited, a trio of MIT-educated software engineers-slash-culture jammers who have developed a program that slices, dices and remixes the words and images of live television broadcasts, in real time, in ways that are intended to be revealing. The group's wonkery can be applied to any TV show with closed captioning, but their focus is now on the presidential debates, which they have digitally split apart and reassembled before live audiences, first in Boston and then here. On Wednesday, they bring "ReConstitution 2008" to the Corcoran Gallery of Art for the third and final round of Obama vs. McCain.

"It's about looking at the debate in terms of underlying content, as opposed to the polished image that's typically coming through your TV," says Sosolimited's John Rothenberg, 29. "It's about taking away the makeup and showing you the hidden layers."

So how do Rothenberg and his mischievous, data-mining buddies -- Eric Gunther and Justin Manor, both 30 -- show you the hidden layers? At the Art Directors Club in Chelsea last week, they sat in front of a few hundred viewers, beneath a massive video screen, wearing suits and sunglasses and mutely fiddling with their laptops. Throughout the debate, they meddled, often in ways that were distracting, sometimes in ways that were annoying and rarely in ways that were particularly revealing.

They turned each candidate's words into stacks of text and then, on a split screen, attached a number corresponding to the number of times each word had been uttered by the candidate. (At one point, 6 was attached to "senator" and 4 to "values.") They pixelated the faces of the candidates and deepened their voices, turning them into witness-protection talking heads, a la "60 Minutes." A running transcript ran next to their smudgy heads, with occasional words blacked out, as though redacted. In another segment -- there are nine altogether -- Sosolimited tallied the number of times each candidate referred to himself, his opponent and to the voters. We learned, for instance, that McCain uttered the word "Obama" 45 times, "senator" 45 times and "country" on 13 occasions.

Here's an obvious question: Who cares?

Why does it matter how many times Obama refers to himself? Why does it matter how many times McCain refers to Obama? This is one of those rare instances where adding information might detract from comprehension. At best, knowing that by the end of the evening McCain had said "I" 112 times and Obama had said "I" 99 times is simply irrelevant. At worst, it could lead you to conclude . . . well, something.

Most intrusively, Sosolimited turned the closed captioning into colored pixels that covered the screen, which effectively made it impossible to see either candidate. And as one segment transitioned to another, there were brief moments when both sound and audio disappeared. If McCain and Obama shook hands at the outset of the night, nobody in the audience at the Art Directors Club saw it. It happened during one of those segues.

So let's be clear: If you're really interested in watching Wednesday's debate, keep away from "ReConstitution 2008." You'll find it maddening. Yes, you'll be awed by the technology. The event, if that is the right word, works as a piece of high-tech performance art. But the intention is to enhance the viewers' understanding of what is being said.

That, at least, is what Gunther, Manor and Rothenberg had in mind during the five years it took to create the software. The three met in college and now work at the same design firm in Cambridge, Mass., which creates interactive installations for places such as the Museum of Sex and the Nobel Peace Center. Sosolimited started as an after-hours side project, and until presidential debate season rolled around, it has mostly involved DJ gigs with a multimedia twist.

"This is the first time we've hooked up with an entertainment manager," Rothenberg says. "We're so focused on the performance side, we never would have had the time to order the folding chairs and hire bartenders."

Members of the audience, which skewed to the 25-to-30ish range, seemed unfazed by all the extraneous information caroming around this production. Maybe the IM generation is accustomed to a certain level of visual white noise.

"When they highlight a phrase like 'health care,' I'm visualizing health care in a way I wouldn't if I didn't see it on-screen," said Leslie Michael, a medical student. "When I'm just listening to words, I feel like I'm missing something."

Erik Larson, who works in the financial services biz, gave "ReConstitution" a thumbs-up for highlighting the absurdity of the debate. "It contextualized everything," he said. "It's as though you are able to see where you are, what you're doing, just how staged the whole debate is."

The members of Sosolimited said one attendee talked about being forced by all the bits-and-bytes shenanigans to focus harder on what the candidates were saying. But you could get the same effect by watching in a kennel full of yapping dogs, couldn't you? How about adding some dimension of understanding, or even changing a mind? Until software can do that, voters will have to stick to the oldest low-tech contraption in the book: the gut.

"Right now, I think most people's reaction is 'Wow, I haven't seen anything like that,' " Manor said. "This is a whole new visual language that we're asking people to be comfortable with, and I don't think anyone can use it as a visual tool yet. For now, we can't expect more than a broad sense that there's more to be done with a television signal than anyone thought."

© 2008 The Washington Post Company