President Obama's reelection campaign famously sent different e-mails to randomized groups of followers to determine which language would generate the best response. It also, to a lesser extent, helped popularize the idea of Web site testing — diverting a fraction of a site's visitors to an alternate version whose performance can be measured against the standard.
Now we've reached the next evolution of that political strategy: applying those same sorts of tests to social media.
Honing your every tweet this way might seem like overkill. But for an industry where even the thinnest margins can have an outsized impact, gaining an edge on social media is crucial. So it's only natural that strategists' love affair with data would extend to their interactions with online as well as off-line followers.
"Testing is critical, especially for smaller clients," said Serenety Hanley, a former Republican National Committee technology director who now runs a boutique social media consulting firm. "The smaller the client, the more vital it is to maximize their dollars."
Serenety works with nonprofits and campaigns designing targeted Facebook ads and promoted posts. She then compares the performance of those posts against one another using tools like Google Analytics and Facebook Insights — information that helps campaigns understand which tactics are the most effective.
Advisors like Serenety are relatively rare; even though A/B testing in general has become mainstream among political strategists, it's still only beginning to affect social media campaigns. Only one commercial shop, ShareProgress, is so far available to liberals. Other groups have resorted to building their own social testing programs in-house.
Among them is MoveOn.org, the progressive advocacy organization. Here's how their system works. Embedded within every link the group shares on Facebook is a set of identifiers. When someone shares the link, it sends information back to MoveOn about who that person is, which version of the item (A or B) is being rebroadcast and how far along the chain of retransmission that person is. While it doesn't seem as though Facebook supports segmenting your audience for more targeted tests, MoveOn can still pool together the aggregate data on links A and B and compare them at a high level.
As a campaign, the immediate goal isn't always to draw as many people as possible to your side. Mostly, it's to create something shareable so that instead of an organization expending its own effort to attract new loyalists, ordinary people become the recruiters, said Milan de Vries, MoveOn's director of analytics.
"We've gotten really good over the last few years at how to broadcast through e-mail with one big megaphone," he said. "But here, if we can harness the dynamics of the networks our members are a part of, we can broadcast with hundreds of megaphones at once."
There are two parts to how this works. The first part is what shows up by default when a user opts to share a campaign's content — things like the headline, graphic and a descriptive blurb. Tweaking the default text and images can sometimes have a profound effect on how users behave. Adding "sign the petition!" to social media posts, said ShareProgress founder Jim Pugh, had the consistent effect of increasing petition signatures. Maybe that sounds obvious. But to a social media manager being pulled in different directions all the time, even the smallest guidance is useful.
The second part involves giving users a choice to customize the headline text and other elements to their own liking. Together, the one-two punch helps content get noticed in the first place and draws potential supporters even closer by making the cause participatory. Personalized messages go an especially long way toward enhancing virality.
To the untrained, virality might just imply the number of times someone watched a YouTube video or liked a Facebook post. But if the objective is to directly compare two social media messages in an attempt to determine which is better, then what you need is something a little more scientific. For that, MoveOn came up with what de Vries called "the coefficient of virality." For every person that sees an item from a MoveOn campaign, how many more people will see and share the same thing?
All these techniques converged in 2011, when a college student named Zach Wahls got up before the Iowa state legislature to oppose a bill to end civil unions. Wahls came from a family with gay parents, and after his speech was posted online, it gained reasonable traction with the headline "Zach Wahls speaks about family." But it wasn't until de Vries and a handful of others updated the headline ("Two lesbians had a baby and this is what they got") and optimized the post for Facebook sharing that it really took off. The Wahls video surged to 17 million views in the wake of the treatment, up from just under a million beforehand, said de Vries.
Facebook has emerged as the most important platform for A/B testing social media, almost to the exclusion of other social networks.
"Twitter is generally pretty bad for this," said Taren Stinebrickner-Kauffman, the founder of the corporate watchdog group SumofUs. "For whatever reason, you don't get many new people through Twitter. There's no other social network yet that I've seen generate anywhere near the level of sharing and click-through and engagement that Facebook does."
The reliance on Facebook carries some new dangers. The company routinely updates its product in ways that breaks the A/B testing tools used by campaigns. And the Facebook audience is by nature different from those on other services, potentially raising concerns about data bias. But the frustrations are trivial compared to the dark days when "testing" meant launching your fully-fledged petition for real, seeing the results, taking the whole thing down, building it back up, and then launching again — to your entire audience.
In short, campaigns are no longer firing shots into the dark. They're learning faster than ever before. And while social media messaging isn't about to eclipse e-mail, that old standby, there's a sense that the environment's changing.
The things that worked six and a half years ago are not the same things that work now," said Stinebrickner-Kauffman.