Coming into Virginia’s state elections last November, it looked like Republican gubernatorial candidate Ed Gillespie was cutting into Democrat Ralph Northam’s lead. Gillespie, whose track record was distinctly moderate, had edged out a hard-right candidate in the Republican primary and was embracing President Trump’s rhetoric on crime and immigration in an effort to make up the difference in his race.
As Nov. 7 approached, the RealClearPolitics average of polls in the state showed Northam’s lead sliding from seven points to around three points. Election Day seemed like it might be a jump ball.
It wasn’t. It was a blowout. The final polling average figured Northam would win by about three points; he won by nearly nine. He won, in fact, by a wider margin than the average of polls had ever indicated.
National polling missed the 2016 presidential election by only a narrow margin, with RCP’s average figuring that Hillary Clinton would win the popular vote by three points. She won by two.
Where polls were off in 2016 was at the state level. The RCP average had Clinton up two points in Pennsylvania, but she lost narrowly. She was up three in Michigan but lost there by a small margin, too. The biggest miss was in Wisconsin, where she was up over six points but, again, was edged out by Trump.
In other words, the miss in Virginia was worse than the miss in Pennsylvania, Michigan and, by a wide margin, nationally. But as data firm TargetSmart’s Tom Bonier noted at the time, the polls “got it right” — that is, they expected a Northam win, and that’s what they got, margin notwithstanding. The miss on the actual number would be somewhat buried. There was precedent for that in 2016, too: The Los Angeles Times-USC poll consistently had Trump winning the popular vote. He didn’t, but he did win the presidency — so the Times touted the “accuracy” of its poll.
There was a glut of analysis after 2016 about why those state polls were wrong. Broadly speaking, polls overall weren’t unusually wrong, as we wrote in June — they were just wrong in specific places that drew a lot of attention. The New York Times’s Nate Cohn distilled those misses to three factors in an article last year: late movement among undecided voters, a failure to weight responses by education (missing the split on Trump between those with and without a college degree) and a boost in turnout among Trump voters.
That last point is important. For polls to be accurate, pollsters have to accurately estimate who’s likely to vote. The pool of “likely voters” who make up final polls can be cobbled together in various ways, and that cobbling can affect the results of the poll. In Virginia in 2017, for example, we looked at three different ways the electorate might look: based on self-identified certainty to vote, looking at the 2013 gubernatorial turnout in the state or on a weighted combination of these two factors.
The most accurate result came from the likely-vote pool that emphasized self-identified certainty to vote. Put another way: the pool that emphasized enthusiasm.
In 2016, one of the problems in state polls was that Trump voters turned out more heavily than many pollsters expected. In 2017, one of the problems with the polls in Virginia was that Democrats turned out more heavily than many pollsters expected. Those Democrats elected not only Northam but a slew of Democrats at the state level, dramatically shifting Virginia politics to the left.
There’s been a lot of focus on those inaccurate 2016 polls over the past few weeks, in part thanks to Trump and his supporters reminding Americans that he and his party had been underestimated before. All of these projections about how the Republicans were going to lose the House? Well, remember when pollsters said that Trump was going to lose Michigan?
Those results from Virginia offer the opposite lesson. All of those projections about how the Republicans are going to hold or expand their leads in the Senate? Well, remember when pollsters thought that the Virginia gubernatorial race might end up close?
As of writing, there are four Senate races in which the margin is less than two percentage points in the RCP average: Arizona, Indiana, Missouri and Nevada. The Republican candidates in Tennessee and Texas have leads of five or six points. If the RCP poll averages are off as widely as they were in Virginia last year, all of those races could swing to the Democrats, probably giving the party control of the Senate.
It all comes down to turnout, as they say. It’s a throwaway motto that candidates use to boost their turnout efforts and that the media uses to acknowledge the uncertainty in election predictions.
It’s also a motto that probably gives a lot of pollsters nightmares.
Scott Clement contributed to this report.