A Politico poll released Monday finds Republicans with a healthy 41 to 34 percent lead in this year’s “most competitive races” for the House and Senate among likely voters.
The poll is among the first trying to home in on voters in the most decisive contests this fall, but as with any specialized sample it’s a quandary how to square results with national surveys. Politico’s Alexander Burns wrote “both [Barack] Obama’s job approval and the partisan ballot matchup are markedly more negative for Democrats in this poll than other national surveys.” While apparently true, most national polls have reported results among registered voters interviewed by telephone while Politico’s are among likely voters interviewed online, a far from apples to apples comparison.
Here are some tips for parsing results from specialized battleground polls.
How are battle-ground voters chosen?
Every battleground poll is different, varying chiefly on what Senate and congressional races a pollster defines to be “competitive.” The Politico survey used a broad definition -- any Senate or House races not ranked as “safe Democrat” or “safe Republican” by the University of Virginia’s Center for Politics. By contrast, an April survey by the Democratic-aligned strategy group Democracy Corps focused only on House districts (not Senate), including a set of 76 congressional districts (versus Politico’s 68).
The samples themselves are drawn in very different ways. Politico’s poll draws its sample from GfK’s Knowledge Panel, where respondents are recruited through traditional means (phone, mail) to take surveys online. Respondents are sampled according to their home address, which is known ahead of time. The Democracy Corps poll relied on a registration-based telephone sample, where phone numbers are appended to voter files.
National poll comparisons are often confounded by other survey differences
Battleground polls hold big potential, appearing to offer a look at how voters in areas with legitimately competitive elections compare with the nation overall. But designing such a sample is difficult, and often involves different methodologies from most national polls which produce predictably different results. When asking about congressional or Senate vote choice, the Politico online survey offered an explicit "don't know" option, which 25 percent of respondents selected. Even after they were probed for which way they leaned, 16 percent insisted they didn't know. By contrast, telephone polls typically accept "don't know" only as a volunteered option, yielding results in the single or low double digits.
Even comparing between similar sample sources sources isn't a sure bet. Politico says its poll was sampled using the same research firm that produces surveys for the Associated Press. AP polling director Jennifer Agiesta cautioned against the equation in an e-mail, writing that "any survey's methodology involves many choices beyond sample source, and on those measures, the polls are quite different."
Likely voter screening matters, especially in midterms
How likely voters are selected is a major source of variation in election polls, and it’s a key reason the Politico and Democracy Corps surveys find Republicans leading by 7 and 5 points while many national polls of registered voters show an even split. The April Washington Post-ABC News poll helps explain the gap, which found Democrats one point ahead among registered voters (45 vs. 44 percent) but five points behind Republicans who are “certain” to vote (44 vs. 49 percent). Even that definition is a very "loose" likely voter screen in 2014, indicating Republicans’ turnout advantage could shift results even further.
House, Senate, or what?
A poll question's wording is always important to examine, and even more so when surveys ask about vague concepts such as the generic congressional ballot. The so-called generic House ballot is an intentionally vague poll question, since voters cast ballots for actual candidates and not simply the “Democratic candidate” or “Republican candidate.” Indeed, the Politico survey asked some respondents how they would vote in a Senate election and some in a House race, depending on how they were sampled. Results may differ in states where a Senate or House candidate is especially well liked or disliked.
How much worse are battlegrounds for Democrats in 2014?
A few percentage points. Obama won 48 percent of the vote in states the Politico survey identifies as Senate battlegrounds compared with 51 percent of the national popular vote.
Remember: Trend is your friend
You can cut through the huge array of differences in poll methodology by looking at the same pollster’s survey over time, and writing off any changes smaller than the margin of sampling error. This is not possible yet in 2014, but will be if Politico and others publish additional waves of battleground state surveys.
Peyton M. Craighill contributed to this report.