Five Myths
Challenging everything you think you know

Five myths about political polls

When Mitt Romney was trailing in public polls before the first presidential debate — particularly in swing states — his campaign manager was dismissive, contending that, according to his camp’s superior internal data, the race was “inside the margin of error.” After the debate, when Romney grabbed the advantage in some public surveys, it was the Obama campaign arguing that “polls don’t matter.”

Well, polls do matter. And it matters how they’re conducted and scrutinized. When trying to make sense of the numbers, here are a few myths to keep in mind.

1. A campaign’s internal polls are more accurate than public polls.

The message from both sides, when they’re slumping, is consistent: Campaign polls are better at assessing voters’ intentions than the polls produced by news organizations and universities.

Five Myths

A feature from The Post’s Outlook section that dismantles myths, clarifies common misconceptions and makes you think again about what you thought you already knew.

Archive

More from Outlook

Assad is too delusional to make peace

Assad is too delusional to make peace

U.S. can’t count on his good faith, says the first American journalist to interview him in two years.

Thermostat wars

Thermostat wars

How cooling the planet could heat up world conflict.

Why your cheating team can do no wrong

Why your cheating team can do no wrong

The NFL’s DeflateGate and the psychology of sports fandom.

Five myths about Super Bowl ads

Five myths about Super Bowl ads

Yes, they are worth the money.

Boko Haram is just as vicious. Why does the Islamic State get all the headlines?

Boko Haram is just as vicious. Why does the Islamic State get all the headlines?

The discrepancy in coverage reflects a certain hypocrisy.

Watching my father vanish before my eyes

Watching my father vanish before my eyes

The grief and helplessness of dementia.

The joys — and benefits — of touch

The joys — and benefits — of touch

David J. Linden shows why touching is good for babies, apes and athletes.

Why sorority culture contributes to the problem of rapes on campus

Why sorority culture contributes to the problem of rapes on campus

Instead of blaming the sisters, blame should go to the institutions’ skewed values.

How American is your team?

How American is your team?

Football teams with centrist fans are more likely to win the Super Bowl.

The National Weather Service’s worst week

The National Weather Service’s worst week

Predictions of a historic storm turned out to be hyperbole.

Gallery

But there is a central flaw in private polling, at least what we get to see of it: Most campaign surveys are presented with a heavy dose of spin. The goals are also different, with a premium on testing messages and anticipating the effects of strategic decisions, often with tenuous assumptions about “likely voters” that may prove wrong.

Nor are campaigns reliable interpreters. Testifying under oath in the trial of former presidential contender John Edwards, Harrison Hickman, Edwards’s onetime pollster, said that the campaign used public polls as “propaganda.” Even though he privately counseled that Edwards had almost no chance of winning the 2008 Iowa caucuses, he said he monitored all the polls and sent “the ones that were most favorable because [campaign aides] wanted to share them with our supporters.”

“Out of a big stack of acorns,” Hickman said, “I was trying to pick out a few good ones that they could pass along to other people, you know, to keep them working.”

Election analysis should look at the big picture, not a few acorns.

2. Polls prove that the first presidential debate upended the race.

Immediately after the debate, coverage focused on polls that moved in Romney’s direction. After all, a Gallup survey showed that 72 percent of debate-watchers said Romney did a better job, the most lopsided debate readout Gallup has ever recorded.

But there is little evidence that the debate decisively moved the needle in key swing states. In six state surveys released Thursday by two well-regarded polling partnerships — NBC-Wall Street Journal-Marist and CBS-New York Times-Quinnipiac — there were virtually no shifts for either candidate compared with pre-debate polls.

Nationally, the debate effect may have quickly faded. In Washington Post-ABC News polling after last week’s face-off, voters had more positive reactions to Romney on the first two nights after the debate than on the next two. In Gallup tracking, the post-debate tally is nearly identical to what it was in the preceding days. And one new Florida poll shows momentum for Romney, but another doesn’t.

The presidential race has long been characterized as tightly competitive and voters as overwhelmingly locked-in. The first debate seems to have done little to alter these basic, well-documented story lines.

3. The best polls are those that correctly predict an election’s outcome.

Getting elections “right” is a necessary but insufficient reason to put great stock in polls. Some surveys could be well-modeled — adjusted to previous elections or to hunches, including, some surmise, tweaks to agree with other polls. (Few pollsters relish being an outlier.)

Both the Republican-leaning Rasmussen Reports and the Democratic-leaning Public Policy Polling have track records that some view favorably, but both use automated phone calls with recorded voices to collect data (“Press 1 for Obama, 2 for Romney,” etc.), revealing a basic flaw: It’s against federal law to have computers dial cellphones, so most “robopolls” ignore about 30 percent of U.S. adults.

At a minimum, excluding cellphones means these polls are increasingly reliant on statistical weighting. They might make up for a lack of young people, for instance, by counting several times the answers from the few younger respondents. Peter Miller, a recent president of the American Association for Public Opinion Research, points out the root problem with weighting: “The more you rely on weights, the more you risk being wrong for reasons you don’t understand.”

It is always better to look at polls’ methodology, not just their results. What’s more, the “horse race” findings are often among the least important parts of a good poll. What issues motivate voters? And who is motivated to vote? These are more crucial to understanding an election than whether a candidate is up two, three or six percentage points.

4. Polls are skewed if they don’t interview equal numbers of Democrats and Republicans.

Political junkies quickly scan each poll release to see the “party ID split” — the proportions of self-identified Democrats, Republicans and independents in the sample. It makes sense: With party loyalty as strong as it is, the numbers of Democratic and Republican respondents in a survey make a huge difference. But to think, as many do, that there should be equal numbers from each party is off-base, particularly when looking at polls meant to assess the general population’s views.

Though voters seem split down the middle, there have generally been more Democrats than Republicans in the country since pollsters started tracking partisan allegiances after World War II. There is often more parity in election tallies, as Republicans tend to be more reliable voters than Democrats, but that also depends on the year.

Artificially evening out a poll’s partisan balance or arbitrarily setting it to match someone’s guess at turnout on Election Day may make one side a bit happier — but at the expense of distorting the current state of play.

5. News outlets are biased in presenting polls favorable toward President Obama.

Media-bashing is at a peak among conservatives, and some on the right routinely dismiss any poll showing Obama ahead of Romney. (And some on the left reflexively scorn polls in which Romney leads.) Before recent surveys turned in Romney’s direction, one conservative blogger went so far as to set up UnskewedPolls.com, a site offering “adjusted” public polls, tilted to favor Romney by manipulating the samples to be more Republican.

In truth, news organizations have only one bias — toward news.

Nothing demonstrates this better than the recent coverage of the Pew Research Center’s post-debate national poll, which showed big, across-the-board movements from Obama toward Romney. The Washington Post splashed the poll on its front page, and the survey provided days of fodder for cable news, illustrating that the media gravitate toward a competitive race, big shifts and conflict much more than one political party or the other.

cohenj@washpost.com

Jon Cohen is The Washington Post’s polling director.

Read more from Outlook:

Five myths about Latino voters

Five myths about presidential debates

Five myths about the 47 percent

Friend us on Facebook and follow us on Twitter.

 
Read what others are saying