So what does that mean for polls about things other than political races? On Tuesday, the issue-polling-focused Pew Research Center released a report aiming to estimate how much poll results on political issues would be affected by a 2020-size error in presidential vote choice.
Many polls that measure what Americans think about political issues rely on similar methods to sample respondents that election polls do, making them susceptible to error in similar ways as election polls.
Yet while election polling can be directly checked for accuracy — when election votes are tallied — this is rarely possible for measuring opinions on social and political issues, when there’s rarely a clear benchmark for comparison.
Pew reweighted a year’s worth of its own survey questions to represent two scenarios: a “balanced version,” in which samples were weighted to match Biden’s actual electoral advantage of 4.4 points and also a “tilted version” with a 12-point Biden lead, representing the high-end for Biden’s margin in national polling toward the end of the race (poll averages showed Biden leading by 7 to 10 points). Pew then compared those two “balanced” and “tilted” samples across a range of political issues to see how much results shifted.
Overall, the simulation found that overstating Biden’s margin against Trump had a small effect on results for issue questions.
For example, when asked whether newcomers from other countries “strengthen American society” or “threaten traditional customs and values,” there was barely a difference between the two versions. The percentage saying newcomers strengthen American society declined from 62 percent in the “tilted” sample that showed Biden with a double-digit lead to 60 percent in the “balanced” sample that matched Biden’s 4.4-point vote margin. The share saying that newcomers threaten traditional customs and values shifted from 36 percent in the tilted sample to 38 percent in the balanced sample.
Across the 48 opinion questions Pew adjusted, most answer categories changed less than half a percentage point and the average change was less than one point. One of the largest changes — three points — was for the question of Trump approval, a measurement strongly correlated with presidential vote preference. In the balanced version, 39 percent approved of Trump and 36 percent approved in the tilted version.
One factor behind the relatively small impact on issue questions is that in a competitive election, a small swing in vote choice can lead to a dramatically different result. While the swing from a 4.4-point Biden lead to a 12-point Biden one is large in absolute terms, it reflects only about a four-point shift in support for each candidate.
Another factor behind the muted impact: While support for candidates in electoral polling is closely tied to party affiliation and opinions on issues, they are not identical.
For example, 23 percent of Biden supporters said they preferred a smaller government in a Pew poll, usually an opinion associated with Republicans. A majority 74 percent of Biden supporters said they favored a bigger government, but that was not a universally shared opinion among supporters of the Democratic Party’s presidential candidate.
Because issue polling does not need to be as precise as election polling, it might be insulated from shifts in political makeup of poll samples. Yet, issue polls have other unique challenges. In its report, Pew noted that “the measurement of opinions is more subjective and likely to be affected by how questions are framed and interpreted.”
For example, past polling has shown support for offering undocumented immigrants a pathway to citizenship is higher when the survey specifies several requirements that applicants will need to meet. Asking about issues in surveys can be more complex than asking people who they support in an upcoming election.
Even as Pew’s report suggests the downstream impact on election poll accuracy on issue polls may be limited, the 2020 election underscored that polls face serious challenges in obtaining politically representative samples. Addressing those problems remains critical for both the accuracy and credibility of surveys going forward.
Scott Clement contributed to this report.