About two weeks before Election Day 2016, the New York Times tweeted its most recent forecast for the outcome of the presidential election. Hillary Clinton had a 93 percent chance of winning, the forecast suggested, leaving Donald Trump a less than 1 in 10 chance of success.

That forecast and ones from the same period have become emblematic of the failures of polling to a large number of Americans. A poll showed Clinton with a 93 percent chance of winning! And we're supposed to take polls seriously?

The proper response to that, of course, is that the forecast wasn’t a poll but, instead, a model of the likely results of the election based on existing state-level polls. What’s more, during the last two weeks of the campaign, Clinton’s odds narrowed significantly, until Trump had a better than 1 in 7 chance of winning. As it turns out, some state polling missed Trump’s eventual margins of victory, particularly in three states in the Rust Belt and the Midwest that he won narrowly. Those state polls overestimated Clinton’s chances, and so did the forecasts.

AD
AD

That election was certainly humbling for those who try to explain what’s happening in American politics by looking at public opinion polling. Polling still remains largely accurate, and, in fact, national polling in 2016 nailed the eventual popular vote margin. But predicting a nearly 3-million-vote win nationally isn’t very useful if you miss the 78,000 votes in Michigan, Pennsylvania and Wisconsin that actually swing the electoral vote.

The Times’s Nate Cohn seems determined not to be caught off guard again. In July, he wrote a detailed article exploring how the results of the 2020 election could diverge even further than they did three years ago, with the Democrat earning a bigger popular-vote margin and Trump getting a bigger margin in the electoral college.

Why?

AD

“Trump’s approval rating has been stable even after seemingly big missteps,” Cohn wrote. “And if it improves by a modest amount — not unusual for incumbents with a strong economy — he could have a distinct chance to win re-election while losing the popular vote by more than he did in 2016, when he lost it by 2.1 percentage points."

AD

On Monday, Cohn and the Times released the results of new head-to-head polling, conducted with Siena College, looking at six states that seem likely to be pivotal next year. The results of those polls are not what Democrats would like to see: Among likely voters in those states (the three that Trump swung in 2016 plus Arizona, North Carolina and Florida), Trump and former vice president Joe Biden are essentially tied in each, with Biden holding a narrow edge in five. Sen. Bernie Sanders (I-Vt.) leads Trump in only one, though the margins are under three points in another three. Sen. Elizabeth Warren (D-Mass.) trails Trump in five states, three of them by four points, and is tied with him in the sixth.

If Trump wins all of those states, as he did in 2016, Democrats will have a tough time taking back the White House.

AD

There are a number of things that are important to remember about this — and any other — poll. The obvious one is that we’re 364 days from next year’s election. (If, for example, unemployment in these states continues to rise, it undercuts that assumption about the “strong economy.”) Another is that the pivotal states in 2016 weren’t necessarily ones that were expected to be, a function both of what the electorate looked like and that Trump’s path to victory required his overperforming somewhere unexpected.

AD

The main thing to remember, though, is a broader one related to what the 2016 electorate looked like.

One of the most valuable polling-related articles the Times published in 2016 was an experiment by Cohn. In it, he gave raw national polling data to a small group of pollsters, each of whom was asked to determine who was winning. One pollster figured that Trump was up by a point. Another figured Clinton was up by four. The rest were somewhere in the middle.

AD

Why the variance? Because the pollsters made certain assumptions about who would turn out. The pollster who had Trump leading was expecting that more white people would turn out to vote — a group with which Trump did well. The pollster who had Clinton leading by four also had the lowest density of white voters.

The nature of polling is that not everyone can be asked their opinion. So pollsters collect data that represents the population — but also generally have to figure out what that population looks like. Not everyone votes, of course, and how and when people vote varies by geography, race, income, education, the candidates on the ballot and any number of other things. If pollsters disagree on who constitutes a likely voter, then they will also disagree on who’s winning.

AD

In a Twitter thread related to the new Times poll, Cohn made this point indirectly. He compared the Times results to recent results by the Marquette University Law School from Wisconsin showing Biden with a slightly wider lead over Trump. That poll included responses from people who said that they intended to register to vote in the election. Take them out and the results generally match what the Times found.

AD

Is that a safe assumption? It’s certainly the case that people who aren’t registered to vote now are less likely to vote than people who are, simply because there’s an extra step involved before they can do so. But the important point is that Marquette and the Times are making different assumptions about whom to include. They use different models of the electorate, and therefore, they get different results.

FiveThirtyEight’s Nate Silver pointed out that the difference in the Times poll isn’t really about how well Biden does as compared with Sanders and Warren. They generally do slightly worse in head-to-head polling than Biden does. The difference, instead, was that the Times poll had the Trump-Democratic races closer overall — and therefore the differences between Biden, Sanders and Warren became more important since they put different Democrats on different sides of the victory margin.

AD

The reason the Times’s polls are closer overall is, again, largely a function of how they model the race. In tweets accompanying the article, Cohn articulates how the model the Times is using differs from other pollsters.

AD

“Obviously, white working class voters are a group pollsters have struggled with in recent years, especially in state polls,” Cohn wrote. “We also do a lot that can help us here that others don’t, like strata on party x region, weighting on education, and response rate adjustment on turnout."

That’s an important point. Cohn is saying the Times poll is explicitly meant to accurately capture the sentiment of white working-class voters — generally understood to be the group that was underrepresented in those flawed state polls in 2016 and therefore threw off forecasts. The Times poll also has Trump doing better with those white working-class voters than other polls, which is an important factor in why Trump does better in the Times’s results than he does elsewhere.

AD

That effort to capture those voters doesn’t mean that the Times poll is unfairly weighted to a group that favors Trump. Cohn says the way it considered the electorate and reported results in the new polls is the same way it did during last year’s midterms — including by weighting responses specifically on education. In 11 polls conducted in the same six states in the last three weeks of the 2018 elections, the Times results were about 1.7 point more favorable to Democratic candidates on average than the actual results.

AD

It does mean, however, that the Times is making assumptions about the electorate that may not hold up. Perhaps 2016 was an outlier on turnout from a group that overwhelmingly favored Trump, and those same voters will be more hesitant to come out and vote. Or perhaps other voters who weren’t as energized to cast ballots three years ago will be more motivated to vote this time around. These things can be hard to predict but, as we saw in 2016, are important.

It’s easy to collapse into what became known in 2012 as “unskewing,” reworking poll results from the outside to better reflect who you personally happen to think is going to vote. In 2012, Republicans insisted that support for Barack Obama was being overestimated and that reworking assumptions about who’d vote showed that Mitt Romney was likely to win. (In fact, Obama’s support was undermeasured, and he won by a wider margin than expected.) But it is nonetheless the case that pollsters disagree on basic assumptions about turnout, which can effect what polls show.

AD

That said, it's worth wondering in this case how much of the paper's experience from 2016 drives how the Times is considering 2020. Plenty of political observers have an inclination to overcorrect for where state polls were wrong that year and to make assumptions about what's likely to happen next year as a result. It doesn't seem impossible that the Times's forecast having become a central point of critique, a critique overlapping with criticism of the paper as out of touch with working-class people in the Midwest, might make the paper more focused on those voters than other pollsters.

The problem with questions like that is they’re very hard to answer until people actually vote — at which point predictive polls become unnecessary.

AD
AD