The Washington PostDemocracy Dies in Darkness

Inside the surprisingly high-stakes quest to design a computer program that ‘gets’ sarcasm online


Sarcasm on the Internet, the old Web adage goes, is kind of like winking on the phone.

It’s pointless. It’s invisible. It’s almost always misunderstood.

And for linguists, pollsters, marketers, stockbrokers, law enforcement and anyone else with a vested interest in knowing what people say (and mean) online, it’s become one of modern computing’s most vexing puzzles: Could we ever teach a program to recognize sarcasm — a human quirk that even humans mess up half the time?

“Sarcasm detection is a very difficult computational problem,” stresses David Bamman, a computational linguist and an assistant professor at UC Berkeley’s School of Information. His most recent stab at solving it — sponsored by the National Science Foundation, and published earlier this year — correctly IDed sarcasm on Twitter about 85 percent of the time, which is still a long way from ideal.

[The Secret Service wants software that detects social media sarcasm. Yeah, sure it will work.]

Part of the struggle lies in the fact that sarcasm, far from “the lowest form of wit,” is actually pretty sophisticated. Saying (or typing!) the opposite of what you mean is a form of what linguists sometimes call implicit speech: It’s deliberately difficult to detect, especially on the Internet.

See, most of the cues that people have developed to signal sarcasm — a louder volume, a slower tempo, a suspicious lack of eye contact — don’t translate well to written text (… a fact that any frequent sender of sarcastic e-mails can probably confirm with some embarrassment). In one 2006 study, readers correctly identified sarcastic e-mails less than 60 percent of the time. In another, three adults were asked to judge whether a set of 270 tweets was sarcastic — and they disagreed on roughly half of them.

In part because sarcasm is so devilishly ambiguous, there’s a surprising degree of demand for an automated tool or an algorithm that could reliably detect it. Not necessarily to save you the trauma of missing a joke, mind you, but to more accurately measure public opinion in its many, exhausting Internet iterations.

[New firms read between the tweets to help companies understand what their customers think]

Sentiment-analysis is already a booming industry: Dozens of firms peddle software that claims to gauge how much social media users like your special interest, or your candidate, or your new line of discount hairspray. But given that snark’s basically the lingua franca of the Web, there’s plenty they could miss. Presidential campaigns are already finding, for instance, that sarcasm can singlehandedly wreck their best estimates of voter sentiment. And what should Netflix make of a one-star review that reads “Give Nicholas Cage an Oscar for this”?

There are graver applications, too: Several government defense agencies, including the Secret Service and the intelligence group that conducts research for the NSA and CIA, have lately solicited proposals for sarcasm-detection software, or funded studies in that vein.

Presumably, the most immediate applications for those programs would involve gauging the seriousness of online threats — statements like “I want to kill Obama” or “I think I’ma SHOOT UP A KINDERGARTEN,” the Facebook joke that briefly landed a Texas 19-year-old in jail. (“If you saw the full context, it obviously reads like sarcasm,” said Justin Carter’s lawyer, who’s still fighting to get prosecutors to drop the two-year-old charges against his client.)

[Texas man faces decades in prison over ‘sarcastic’ Facebook comment]

And yet, given that people have so much trouble determining tone online, how much are we expecting a computer to do? Minus the overt winks we’ve developed to signal online sarcasm to our more oblivious friends — the #haha, the drawn-out soooooo, the ~ironic tilde~ — irony rarely signals itself in a way an algorithm can read.

Computer scientists have tried to address the problem by feeding massive batches of “sarcastic” data — frequently tweets with the #sarcasm hashtag — to self-learning, pattern-seeking programs that look for recurring words, phrases and topics that people tend to reference when they’re being sarcastic. Mathieu Cliche, who developed a public tool called The Sarcasm Detector when he was a Ph.D. candidate at Cornell, can reel off lists of words that skew sarcastic or sincere: “just what,” “just love,” “a blast” and “shocker” tend toward the ironic — but tweeters rarely talk about their mothers unless they mean it.

Alas, this approach has some pretty obvious holes. If I type “I just love it when the office is quiet in the morning” into Cliche’s Sarcasm Detector — a true statement that I mean, in all sincerity! — it spits out a “sarcasm score” of 71 out of 100.

In other words, it’s pretty much positive that I’m being sarcastic, simply because I used a statistically sarcastic word; the algorithm has no way of even conceiving of a quiet office, let alone the fact that a quiet office is a good thing or that I might appreciate one.

“In 2015, computers aren’t so bad at understanding language,” explains Christopher Manning, a professor of computer science and linguistics at Stanford. “But they’re still pretty bad at understanding the world.”

In order for sarcasm detection to really, truly work, Manning says — for Netflix or the Secret Service or anyone else — the technology will have to move past the mere words we use to be sarcastic, and to begin to understand the lived human experience.

[Should sarcasm get its own font?]

There are some glimmers on that front: In the past six months, in fact, two papers have proposed ways for sarcasm-detection algorithms to account for more than just words. Bamman’s recent attempt at gauging sarcasm on Twitter — the one that scored 85 percent accuracy — sought to understand the speaker, his audience and the relationship between them by also ingesting contextual information from their past tweets and Twitter bios. (Fun fact: Being unverified, male and American, tweeting about art and TV, and having the words chemistry or #atheist in your bio are all strong predictors of irony.)

Another similar paper about Reddit, published just last month, found that sarcasm-detection becomes far more accurate if the algorithm knows not only what was said, but where — as in, did someone drop an “I love Obama” in r/conservative, r/liberal or r/obamacare?

This is still small fries, of course — but it’s a step toward teaching computer programs the complex, multivariate relationships between things and ideas in the real world. And that, in turn, is a step toward deciphering Internet sarcasm. And that’s a step toward … well, who knows.

Maybe a machine that can not only process concepts like silence and noise, or morning and afternoon, but that can also conceive of feelings like distraction and stress and how they might relate to a quiet newsroom.

“A true sarcasm detector will need to understand people — what they like, what they think,” Manning said. “We’ve already made enormous advances in things like speech recognition, things we once thought of as artificial intelligence.”

Maybe, if researchers have their way, snark recognition is next.

Liked that? Try these!