Something is amiss in Granite State polls…again. Last Tuesday, Bloomberg News released a poll whose analysis began “Mitt Romney holds a commanding lead in the New Hampshire Republican presidential primary, more than double the support for his nearest rival, Texas congressman Ron Paul.” Only three days later, the New Hampshire Journal declared “Romney and Gingrich in a statistical dead heat for the January 10th primary.”
Say what?
New Hampshire has long earned a reputation as the Bermuda Triangle for pollsters, most recently failing to forecast an impending victory by Hillary Rodham Clinton in the 2008 Democratic primary, an episode that prompted the American Association for Public Opinion Research to produce a 100-page report investigating the fiasco. So what’s happening this time around?
The numbers
The Bloomberg News poll conducted Nov. 10-12 pegged Romney’s support at 40 percent and Gingrich’s at 11 percent, while the New Hampshire Journal poll, an automated survey conducted Nov. 15-16 by the Republican firm Magellan strategies, reported Romney and Gingrich at 29 and 27 percent, respectively. That marks an 11-point drop for Romney and a 16-point rise for Gingrich in the span of a single week.
While Gingrich’s numbers have been on the rise, it’s nothing to the order of 17 percentage points in a window of days, and there’s not a strong reason why Gingrich would surge in New Hampshire. He’s not a favorite son or neighbor like Romney (he’s from Georgia), and while he has visited the state 14 times since June, he’s been there only once in the last 30 days, according to The Washington Post’s ultra-useful Republican primary tracker. And while a majority of voters have been contacted by the Romney and Ron Paul campaigns according to the Bloomberg poll, Gingrich’s campaign has only reached one in five.
Suffice to say, other factors besides a shift in voter attitudes may be at play.
Party and ideology
One clue to the difference in polls is the partisan and ideological makeup of the “likely” electorates. New Hampshire hosts an open primary where both registered Republicans and those who are not registered with a state party — undeclared voters — are allowed to vote in the GOP contest.
Registered Republicans outnumber undeclared voters by a 60 to 40 percent margin in the Magellan poll, while the Bloomberg survey found Republican and undeclared voters making up 47 and 53 percent of the likely electorate, respectively. The Magellan poll weighted its results by the firm’s 2012 projections of turnout by party registration, gender and age.
Besides weighting the statewide sample by Census figures for age and sex, the Bloomberg poll did not weight to match pre-determined turnout estimates. Instead, the poll defined likely voters as registered Republicans and independents who said they would “definitely” or “probably” vote in the GOP primary.
Weighting by projected turnout numbers is a controversial subject among pollsters. J. Ann Selzer, who conducted the Bloomberg poll, said in an e-mail she “would never weight to past election demographics. That blinds you to any change in the electorate.” Magellan President David Flaherty argues that their poll is more in line with past Republican primaries and the midterm elections. Some 61 percent of Republican primary voters in 2008 were registered with the GOP, as were 63 percent in 2000, according to exit polls.
The difference doesn’t seem to matter much. Romney performs better among undeclared voters in the Journal poll, but even when the Magellan results are adjusted to match Bloomberg’s party registration makeup, there’s essentially no difference. Romney’s edge over Gingrich increases from two to four percentage points under that scenario, closer to the Bloomberg poll but not nearly enough to explain the large discrepancies in candidate support.
Same goes for ideology and age, where likely voters in Magellan’s poll are somewhat more conservative and older than Bloomberg respondents. After adjusting these results to match the Bloomberg poll on both of these measures, neither makes a big difference in the poll result.
(On a side note, both pollsters deserve big kudos for disclosing a host of demographic information about their sample, including crosstabs from Magellan, which allows this kind of analysis.)
Automated versus live-interviewers
With both these hypotheses dead in the water (just like those Granite State polls of yore), another alternative seems possible: respondents to the Magellan automated survey respondents may be systematically different from those from live interviewer polls like Bloomberg. Magellan made phone calls using a recorded voice with touch-tone instructions, while the Bloomberg survey was conducted using live interviewers calling both landlines and cellphones. Automated surveys are thought to have substantially lower response rates than live interviewer surveys, and most automated polls (including Magellan) do not call cellphones due to legal restrictions on auto dialers.
In addition to New Hampshire’s traditional challenges, differences between increasingly prevalent automated polls and those conducted by live interviewers may add another wrinkle. So far in 2011, nearly half of all polls in early primary and caucus states (Iowa, New Hampshire, South Carolina and Florida) have been conducted using automated methodology. Along with a highly fluid GOP race, the differences across survey modes is worth scrutiny.
Sign up for Post Polling e-mails
