|Page 2 of 2 <|
5 Tips for Decoding Those Election Polls
If you have the time to drill down, do. Look for biased questions and cherry-picked or hyped analyses. Watch for big headlines about small differences and reckless analysis of small subgroups.
3. Watch for consistent change and a meaningful narrative.
Change over time is important, especially when it's consistent, with a clear narrative of what's happening and why. Knowing, say, former Arkansas governor Mike Huckabee's current level of support in Iowa is woefully incomplete without also knowing its trajectory over time -- from 8 percent in our July poll to 24 percent in November to 35 percent in December.
"Trend is your friend," pollsters say. Look at repeat polls from the same organization to gauge movement over time. And, again, look beyond the horse race to other measures: the levels of commitment and enthusiasm from a candidate's supporters, the groups that are more or less fired up, the factors motivating their support. It takes willpower to trudge off to an hours-long Iowa caucus on a dark winter's night. Who's inspired? How? Why?
Look at preferences on the issues; examine views of candidates' various attributes; consider differences among voter groups. The evolution of these views puts the story in context -- elevating mere numbers into something of greater value. Let's call it intelligence.
4. Don't be seduced by averages.
Poll averages are all the rage this year, even ones that purport to show candidates' standings measured down to tenths of a percentage point. (Don't get us started.) This may fill political junkies' seemingly insatiable desire for a minute-by-minute assessment, as if this were the Nasdaq average or our rich Uncle Leo's EKG. In fact, there's a lot less to these averages than meets the eye.
Averaging across polls with different methodologies can easily obscure rather than clarify. If you take a state with few polls -- one good-quality survey, say, and three methodological clunkers -- averaging may well do more harm than good. Averaging polls done across different time periods, with different sampling methodologies, different procedures to estimate "likely voters" (some reasonable, some not) and different numbers of alleged "undecideds" all assumes that these differences make no difference. With this approach, you might as well throw a little Ouija in as well.
The reality is that a good poll is a good estimate. All else being equal (and it never is), a collection of good polls will be an even better estimate, but a collection of good and bad polls won't.
5. Be skeptical of post-election scorecards.
As surely as night follows day, you can bet that some lucky pollster will spring up after each caucus or primary with a chest-beating announcement about how his or her estimate was the most accurate. Chill. Good technique matters, but post-election assertions that this or that poll was the "most accurate" are, by and large, hokum. Polls tend to converge as Election Day approaches, but their paths to the endpoint vary dramatically. Some pollsters weight their data to previous turnout, some build in high numbers of undecideds and then arbitrarily allocate them to one side or another, others do Lord-knows-what. Sometimes it works, and a career is born.
At the end of the day, we're on solid ground judging polls by their inputs, not their outputs; a lucky guess is not the same thing as a high-quality survey. It takes rigorous, proven sampling methods, well-crafted questions and intelligent analysis to produce a valid, reliable and meaningful understanding of an election. A good estimate will bring us within a few points of the final outcome, but pinpoint accuracy in pre-election polling is a myth. And for a few of us, at least, the aim of the enterprise is not simply to win the horse-race lottery.
One last suggestion: Relax, unless you're one of the candidates. We'll know what the voters decided soon enough -- and, with the help of good polling, we'll even know why.
Gary Langer is director of polling at ABC News.
Jon Cohen is director of polling at The Washington Post.