It will be a few more weeks before we know exactly what went wrong with the 2004 exit polls. But this much we know right now: The resulting furor was the best thing that could have happened to journalism, to polling and to the bloggers who made this year's Election Day such a cheap thrill.
That's because the 2004 election may have finally stripped exit polling of its reputation as the crown jewel of political surveys, somehow immune from the myriad problems that affect telephone polls and other types of public opinion surveys. Instead, this face-to-face, catch-the-voters-on-the-way-out poll has been revealed for what it is: just another poll, with all the problems and imperfections endemic to the craft.
_____Outlook Live Online_____
Richard Morin fields your questions and comments on this article on Monday, Nov. 22, 11 a.m. ET.
It's also time to make our peace with those self-important bloggers who took it upon themselves to release the first rounds of leaked exit poll results. Those numbers showed Democrat John F. Kerry with a narrow lead, which ignited premature celebrations in one camp and needless commiseration in the other -- until the actual votes showed President Bush had won.
If a few hours on the roller coaster of ecstasy and agony were all that anyone had to endure, only the political junkies would be interested in the whys and wherefores of the exit poll confusion. But the false picture had real impact: The stock market plummeted nearly 100 points in the last two hours of trading, and the evening news was replete with veiled hints of good news to come for the Kerry campaign. Since then, some disappointed and angry Bush-bashers have seized upon the early numbers as evidence of something amiss in the outcome. You can read it on the Internet -- the election was stolen, the early exit poll numbers were right.
But rather than flog the bloggers for rushing to publish the raw exit poll data on their Web sites, we may owe them a debt of gratitude. A few more presidential elections like this one and the public will learn to do the right thing and simply ignore news of early exit poll data. Then perhaps people will start ignoring the bloggers, who proved once more that their spectacular lack of judgment is matched only by their abundant arrogance.
It seems clear now that the 2004 exit polls were rife with problems, most of them small but none trivial. Skewed samples, technical glitches and a woefully inept question that included the undefined term "moral values" in a list of concrete issues all combined to give exit polling its third black eye in as many elections.
The sampling errors gave a boost to Kerry, who led in all six releases of national exit poll results issued on Election Day by the National Election Pool (NEP), the consortium of the major TV networks and the Associated Press that sponsored the massive survey project. (The Post received exit poll data as an NEP subscriber.)
In the first release, at 12:59 p.m. on Election Day, Kerry led Bush 50 percent to 49 percent, which startled partisans on both sides. That statistically insignificant advantage grew to three percentage points in a late-afternoon release, where it remained for hours, even as the actual count began to suggest the opposite outcome. It was only at 1:33 a.m. Wednesday that updated exit poll results showed Bush ahead by a point.
Even more curious numbers were emerging from individual states. The final Virginia figures showed Bush with a narrow lead. Exit poll data from Pennsylvania, which was held back for more than an hour, showed Kerry ahead by nine percentage points. The actual results: Bush crushed Kerry in Virginia by nine points, while Kerry took Pennsylvania by just a two-point margin.
In a review of 1,400 sample precincts, researchers found Kerry's share of the vote overstated by 1.9 percentage points -- which, unhappily for exit pollsters, was just enough to create an entirely wrong impression about the direction of the race in a number of key states and nationally.
It's hardly unexpected news that the exit polls were modestly off; exit polls are never exactly right. The networks' 1992 national exit poll overstated Democrat Bill Clinton's advantage by 2.5 percentage points, about the same as the Kerry skew. But Clinton won, so it didn't create a stir. In 1996 and 2000, the errors were considerably smaller, perhaps just a whiff more Democratic than the actual results. That suggests to some that exit polls are more likely to misbehave when their insights are valued most -- in high-turnout, high-interest elections such as 1992 and this year.
I learned early in my Washington Post career that exit polls were useful but imperfect mirrors of the electorate. On election night in 1988, we relied on the ABC News exit poll to characterize how demographic subgroups and political constituencies had voted. One problem: The exit poll found the race to be a dead heat, even though Democrat Michael Dukakis lost the popular vote by seven percentage points to Dubya's father. (The dirty little secret, known to pollsters, is that discrepancies in the overall horse race don't affect the subgroup analyses. Whether Dukakis got 46 percent or 50 percent didn't change the fact that nine of 10 blacks voted for him, while a majority of all men didn't. The exit poll may have under- or over-sampled either group, producing an incorrect national total, but the within-group voting patterns remain accurate.)
In practice, there are many separate exit polls, not just one. This year, there was a national one based on interviews at 250 randomly selected polling places around the country by Joseph Lenski and Warren Mitofsky under contract with NEP. Then there were separate exit polls in each state. The number of precincts sampled in these states ranged from 14 in Alabama to 52 in Florida.
In theory, the voting pattern in these precincts should reflect the national and statewide votes. If the exit poll results differ from the actual vote -- say, the sample precincts nationally showed Kerry ahead by three points while he ended up losing by three -- then something was wrong with the sample.