An article in last Sunday's Magazine misstated the frequency with which public opinion polls fall outside the margin of error. The correct figure is one out of 20. (Published 09/19/1999)
Margins Of Error
Back in 1982, Lance Tarrance was the pollster for the then-
governor of Texas, Bill Clements, an irascible Republican who was running for reelection to a second term. It was a lousy year to be a Republican, given the national recession, but right up until the weekend before the vote, Tarrance's polls showed his client cruising toward victory.
When Election Day came, however, Clements got swamped by a tidal wave of Democratic voters. A few weeks later, Tarrance saw the vanquished governor at a meeting of the Republican Governors' Association and coined a memorable line when asked how their first post-election session went. "It was like being Napoleon's intelligence chief after Waterloo," Tarrance said.
Tarrance's remarkable candor offers a benchmark as the campaign of 2000 heats up. Over the next 14 months, a small army of pollsters will chart the mood of America and offer insights into the psyches of every imaginable demographic slice of the electorate to candidates hungry for the clues that will turn a close contest into victory. The luckiest among them will end up with the unofficial title of "pollster to the president," which will bring fame and no little fortune.
But the law of averages says that somewhere along the way, some of these pollsters will get it wrong -- a single poll perhaps, a strategic insight almost certainly, even possibly a whole race, although the odds weigh heavily against that, given the round-the-clock sampling that marks the conclusion of any truly competitive race. You wouldn't guess it from talking to them, however.
I made a round of calls to some of the best-known names
in the polling business recently to ask what happens when things go wrong. The calls produced few confessional tales of the race that got away or the poll that went awry. As Mark Mellman,a Democratic pollster, put it, "More often than not it's a matter of interpretation. The numbers aren't usually wrong." Even though every pollster knows that one poll in five is likely to be outside the margin of error, the bad ones mostly have disappeared into the mists of memory.
If these pollsters had a story to tell, it was usually on somebody else. One remembered a congressional district poll done by a competitor that showed the incumbent congressman not doing very well back home -- until the candidate himself took a look at the internals of the poll. These towns aren't even in the district, the congressman complained. The pollster barely blinked. This is for wallpaper, he said. The real poll is coming back next week.
It's clear that even the science of polling can't eliminate all the gremlins that can contaminate findings. Polls with samples that have too many Republicans or too many Democrats can throw off results. Sometimes, my experts said, a poll comes back that doesn't pass the smell test. What do they do in that case?
"I hate to advertise this, but we usually do it again, or at least part of it," Mellman said. Geoff Garin, also a Democratic pollster, said he occasionally redoes a poll that looks wrong -- at his own expense. Bob Teeter, a Republican who was President George Bush's pollster, said badly worded questions often lead to confusing results. He recalled a poll he did years ago in Missouri that asked something like, "In what areas do you think new consumer protection laws are needed?" It came back with answers like "Kansas City,"
"Columbia" or "Springfield."
The biggest blunder in polling history came in 1936, when the Literary Digest predicted that Republican Alf Landon would defeat President Franklin D. Roosevelt.
It was based on a telephone poll of voters at a time when many households didn't have phones, and the polling stopped weeks before the election. Roosevelt won in one of the biggest landslides in history.
Pollsters say that kind of mistake is nearly impossible today, given improvements in methodology. But just 20 years ago, Garin recalled, his firm wrongly predicted the outcome of an Iowa Senate race, based on the final poll for the campaign. "But in 1978, our last poll was done the first week of October, which was pretty common," he said. "There were four weeks of campaigning between that poll and Election Day. Nowadays we're doing 3,000 interviews in the last three weeks of a big race."
But if pollsters say they are rarely surprised by the outcome of races in which they're involved, the same can't be said for candidates. Bob Dole has been particularly hard on his pollsters -- or vice versa, to hear him describe it. For years he publicly bad-mouthed his 1988 pollster, Richard Wirthlin, whom he blamed for too-rosy predictions in New Hampshire, which Dole eventually lost to Bush.
Eight years later, Dole sacked his pollsters after another losing campaign in the New Hampshire primary, leading David Letterman to quip, "Being in charge of polls for Bob Dole -- that position has about as much job security as being a Saddam Hussein son-in-law." Maybe that's why they bury their mistakes.
Dan Balz covers national politics for The Post. Al Kamen will be back In the Loop next week.