But it's hard to resist. Donald Trump can't not talk about general-election polls, even when the polls demonstrate the opposite of what he says. There's something about getting a poll number back — Kasich 44, Sanders 43, hmm, interesting — that's too tantalizing to pass up. That despite the fact that a lot of people have never heard of Ohio Gov. John Kasich or Sen. Bernie Sanders of Vermont and that we still have more than 200 days of campaigning before voters actually have to make up their minds.
Last fall, we warned readers not to pay attention to general-election polling for precisely those reasons. We figured it was worth revisiting the subject to see whether the polls were any better now than they were then.
So we took all of RealClearPolitics' collected polls of the general election for 2004, 2008 and 2012 and plotted them on three dimensions. First, how far before Election Day the poll was conducted. Second, how the margin between the Democrat and the Republican compared with the actual margin. (In other words, if Mitt Romney lost to Barack Obama by 3.8 points but the poll had Obama up by 1, that's a poll that's 2.8 points more friendly to Romney than it should have been to be accurate.) And, third, how many of the candidates in the poll actually ended up being nominated. After all, a lot of early polls — including head-to-head polls that are coming out now — include one or two people that will never actually be on the ballot in November.
Here's what that looks like in the stretch from 300 to 200 days before Election Day. (The dots are scaled by size to represent how many eventual nominees were included and colored by cycle. The closer to the middle the dot is, the closer the poll margin was to the final result that year.)
You don't need to parse every dot to get the important point here. Between 300 and 200 days before the general election, polls were still all over the place. If you'd isolated any one of those dots and tried to use it to predict the election, you'd have been hard-pressed to know which dot would get you closest.
But let's extend that outward. Let's see how it looked for every day from 700 days before the election until Election Day. Like so.
The dots start to cluster closer to the middle over time, as you'd expect. But there's still a great deal of variation, even right before the election — after a long campaign and with the two actual candidates well-known.
We can look at this another way. We averaged the margins between the two candidates in 100-day chunks for each possible matchup for which we had data. (See the list at the bottom below.) For most contests, the general election being polled never came to be, so the lines fade away before the election arrives. For the three contests that did happen — John Kerry vs. George W. Bush, Obama vs. John McCain and Obama vs. Romney — we made the averages stand out.
The results were unexpected.
Averaging polls in each 100-day period pitting Romney against Obama in 2012 usually came out to at or just below a three-point difference. Sometimes Romney was leading in polls; usually he wasn't. But the aggregated accuracy didn't change much. In 2004 and 2008, the patterns weren't as predictable.
This is why the most useful tool for predicting election results has been a running average of the polling. That's not what we're doing here. This is a raw average of a lot of polls of varying quality.
To make a different point. Isolating one particular head-to-head poll can be risky even right before the election. Polling is an instrument imprecise enough that we calculate and share the built-in margin of error to ensure that people understand the ways in which it might vary from the final results.
So if the election were held today, would Kasich actually beat Sanders by a point? One poll said he would, but the world in which the election is today is a very different world than the one in which we see that poll (and the polls that disagree with it). What we can say for now is this: When candidates claim that they'll necessarily win in November because the polls prove it, you can ignore them.