The Washington Post

Deja vu for New Hampshire polls?

A “fiasco,” one analyst called it. Another observer called it a “snafu.”

In short, the pre-election polls before the New Hampshire primary in 2008 were a disaster. The numbers had anticipated a clear result: It would be a second, major win for then-Sen. Barack Obama over Sen. Hillary Rodham Clinton, who had been the overwhelming favorite for the Democratic nomination.

Instead, Clinton won by about two percentage points, setting off the polling industry’s biggest reevaluation since the infamous “Dewey defeats Truman” headlines of 1948.

With voters in the Granite State again set to cast ballots in the country’s first primary, are we in for a surprise similar to the one greeting poll watchers four years ago? After all, even before the 2008 debacle, The Washington Post dubbed the state “a snowy graveyard for pols and pollsters.”

A big shock is unlikely, at least at the top of the heap — where former Massachusetts governor Mitt Romney has a large, consistent lead — but that’s not necessarily because the polls are better this time around.

The New Hampshire polls in 2008 sent shock waves not mainly for their numerical inaccuracy — pre-primary polls are notoriously imprecise — but because most presaged the wrong winner. On average, the polls before the Iowa caucuses were just as off-target, but the final result lined up with expectations gleaned from most polling there.

And this year’s Iowa polls were no better than they were four years ago, with an average miss on the top two candidates equal to the Clinton and Obama differences in 2008.

After so many surveys got the winner wrong in New Hampshire last time, there was intensely negative attention on political polls. The American Association for Public Opinion Research — an industry organization — responded by creating a task force to dissect the data from New Hampshire and four other states. (Disclosure: I served on AAPOR’s executive council from 2009 to 2011.)

The resulting report — published in early 2009 — revealed no clear reasons for the New Hampshire slip-up. But a number of likely culprits emerged, including the compressed period between the Iowa caucuses and the New Hampshire primary, and a large influx of first-time voters that might have spoiled likely voter models tuned to past behavior or elections.

David Paleologos, who heads up the survey center at Suffolk University and is again conducting daily tracking polls in New Hampshire, says he is now using stricter criteria to identify probable voters. He used looser ones four years ago because of the great “fluidity between Democratic and Republican primary voters.” (New Hampshire voters who are not registered with a political party can opt to participate in either one.)

Without a meaningful Democratic primary this time around, one big polling hurdle is gone. Paleologos also now includes cellphones in Suffolk’s polls, as do some others. The AAPOR report said the lack of cellphone samples was unlikely to have caused the errors in 2008, but the proportion of New Hampshire adults who have only a cellphone has about doubled since then.

But the biggest difference this year is contextual — Romney’s advantage over his rivals is about double Obama’s best from four years ago. The errors would have to be that much greater to spoil most prognostications about the No. 1 spot.

Although polling in general might survive public shame in New Hampshire this year, what hasn’t changed in political polling is a continued lack of basic disclosure, pointing to continued risk.

AAPOR’s main finding about the 2008 polls focused on transparency, not polling acumen. Delayed and incomplete information about 2008 limited the researchers’ ability to draw conclusions about what happened, and why. And without such knowledge, the possibilities for similar errors remain.

The Freddie Gray case

Please provide a valid email address.

You’re all set!

Campaign 2016 Email Updates

Please provide a valid email address.

You’re all set!
Show Comments
The Republicans debated Saturday night. The New Hampshire primary is Feb. 9. Get caught up on the race.
Highlights from Saturday's GOP debate
Except for an eminent domain attack from Bush, Trump largely avoided strikes from other candidates.

Christie went after Rubio for never having been a chief executive and for relying on talking points.

Carson tried to answer a question on Obamacare by lamenting that he hadn't been asked an earlier question about North Korea.
The GOP debate in 3 minutes
Play Video
We have all donors in the audience. And the reason they're booing me? I don't want their money!
Donald Trump, after the debate crowd at St. Anselm's College booed him for telling Jeb Bush to be "quiet."
Play Video
New Hampshire polling averages
Donald Trump holds a commanding lead in the next state to vote, but Marco Rubio has recently seen a jump in his support, according to polls.
New Hampshire polling averages
A victory in New Hampshire revitalized Hillary Clinton's demoralized campaign in 2008. But this time, she's trailing Bernie Sanders, from neighboring Vermont. She's planning to head Sunday to Flint, Mich., where a cost-saving decision led to poisonous levels of lead in the water of the poor, heavily black, rust-belt city. 
55% 38%
Upcoming debates
Feb. 11: Democratic debate

on PBS, in Wisconsin

Feb 13: GOP debate

on CBS News, in South Carolina

Feb. 25: GOP debate

on CNN, in Houston, Texas

Campaign 2016
State of the race

To keep reading, please enter your email address.

You’ll also receive from The Washington Post:
  • A free 6-week digital subscription
  • Our daily newsletter in your inbox

Please enter a valid email address

I have read and agree to the Terms of Service and Privacy Policy.

Please indicate agreement.

Thank you.

Check your inbox. We’ve sent an email explaining how to set up an account and activate your free digital subscription.