| || || ||
| ||ONLINE EXTRAS/What Americans Think|
Surveys Way Off the Mark for N.H. Primary
By Richard Morin
Washington Post Polling Director
Monday, February 7, 2000
George W. Bush wasn't the only one to hit a bump in the road in New Hampshire. Most of the tracking polls conducted immediately before the election also stumbled and bumbled to the finish line.
But what else is new?
To the credit of each, every major tracking poll had the winner either ahead of or within the margin of sampling error. But few surveys foreshadowed Arizona Sen. John McCain's wipeout win over Bush, and many had Vice President Gore with double-digit leads over former senator Bill Bradley on the Democratic side.
This primary's "Dewey Defeats Truman" moment came courtesy of the American Research Group (ARG). Its final poll, completed last Monday night, put Bush ahead by a 38 percent to 36 percent margin, within the sampling error of plus or minus 4 percentage points, but miles from the final mark, 49 percent for McCain to Bush's 31 percent.
CBS News got the Republican winner right, but just barely: CBS reported a 39 percent to 35 percent McCain advantage among "definite" voters, based on polling conducted through the Sunday before the primary.
Gallup for USA Today and CNN came closest. Its final tracking conducted last Sunday and Monday found McCain up a dozen points over Bush: 44 percent to 32 percent. Zogby International for Reuters/WHDH-TV reported a 10-point McCain lead (41 percent to 31 percent) and released on election day the results of a special 10-day tracking poll that had McCain up by 12, an estimate "derived by factoring in leaners and the distribution of undecided voters." (Say what you will, John Zogby's got mojo.) The University of Massachusetts had McCain up by 9 (37 percent to 28 percent) and WMUR/Fox had McCain ahead by 7 (41 percent to 34 percent).
On the Democratic side, ARG made a praiseworthy recovery by nailing the margin, reporting a five-point Gore advantage in their final polling. CBS had a bad night on both sides of the aisle, predicting a 16-point Gore landslide (55 percent to 39 percent) in a race the vice president won by a 52 percent to 47 percent margin.
Gallup/CNN/USA Today had Gore up by a dozen, 54 percent to 42 percent. Reuters/WHDH also had Gore up by 12, 52 percent to 42 percentnot too far off, given margin of sampling error for these surveys. The University of Massachusetts and WMUR/Fox had the race extremely close, well within the margin of error, which is about where it ended up.
Some caveats: Primary elections are notoriously difficult to predict, particularly in New Hampshire, where independents can vote in either party's primary. Tracking polls are fragile things. Results are based on interviews of as few as 100 "likely voters" each night, with large margins of sampling error for daily results. Also, reporting the size of the margina 12-point lead, for exampleoften is misleading and tends to hype any change in the overall results. That's because a one-point change in a two-person race produces a two-point change in the margin. Getting the margin right with large numbers of don't-knows may be meaningless and perhaps just lucky.
All that said, let's get real: Small samples or large samples, final tracking numbers are read as predictions, and rightly so. Hiding behind "margin of sampling error" doesn't work if you've got Bush ahead (albeit by a statistically insignificant margin) and he gets waxed by 18 points. Likewise, the size of the lead is how the politicos, pundits and real people talk about tracking poll results. The bold boys and girls who elect to track the horse race should expect their results to be compared with the actual voting, for better or for worse.
In non-predictive polls, reporting margins are to be avoided, or at least ignored in favor of simply reporting the overall results. Not so for election-eve tracking polls: It's a different game, with different rules. Some differences between election results and election-eve tracking polls is understandable, perhaps even inevitable. About one out of six Republican and Democratic primary voters made up their minds on election day, making even election-eve polling out of date if these late deciders break big for a candidate.
That's exactly what happened in New Hampshire, but in a puzzling way. Democrats who made up their minds on election day disproportionately voted for Bradley, not Gore. That suggests those inflated Gore margins in pre-election polls probably were slightly closer to the mark than they might otherwise appear. That's good news for trackers.
Here's the bad news: Republican late deciders broke big for Bush and Steve Forbesnot McCain. This suggests the difference between the tracking polls and election results would have been much larger had these procrastinating GOPers voted in similar proportions as those who made up their minds earlier.
One final New Hampshire curiosity. The pre-election polls have suggested in recent months that Bradley led or was tied with Gore. But VNS exit polling found that among voters who said they made up their minds sometime before the past week, Gore led Bradley, and by double-digit margins. Is there something wacky about those who decided to vote? Perhaps problems with identifying likely voters? The Curse of New Hampshire? Go figure.
While the tracking polls are getting their lumps, the television network exit poll consortium is getting kudos from clients after a generally smooth New Hampshire primary.
"New Hampshire is always difficult," says Murray Edelman, editorial director of Voter News Service. "It did go smoothly. It was my favorite kind of election: The winner on one side was winning by big margins, and on the other side it was too close to call."
His impression of the Election-from-Hell: where one candidate has a lead that's just large enough "that you're thinking about calling, but you're just not sure."
Archive: What Americans Think