It’s easy to look at this poll and have an extreme emotional reaction, no matter where you stand on abortion. Pro-life people might feel that the recent visceral fights in New York, Virginia and the U.S. Senate over late-term abortions and whether doctors are legally obligated to care for babies who are born alive after an attempted abortion are finally turning the tide on the greatest moral issue of our time. And pro-choice people might fear that Trump-era Republicans have figured out how to turn public opinion against them and turn back the clock on reproductive rights.
But everyone should stay calm and take some deep breaths before concluding that the landscape of abortion politics in the United States has dramatically shifted in a single month, no matter how crazy that month was for coverage of the issue. Over time, analysts have collectively developed some informal rules for reading polls just like this one. And these rules, when properly applied, suggest that we shouldn’t revise our perception of abortion polling just yet.
Here’s how to think about this survey and many other surprising polls you come across on the Internet):
Don’t overreact to any one individual poll
The First Law of Polling Analysis is never jump to conclusions based on one poll. Polls are great — they’re our most powerful tool (other than elections) for figuring out what the American people actually think and want. But all polls involve randomness, and it’s possible that Marist (which is generally a good pollster) just got a weird sample this time around. We should wait to see if other pollsters see a similar shift or if Marist comes up with this result again in its next poll before concluding that a sea change has happened.
Check the context and the sponsor
It’s also worth looking into what other polls are saying about this general issue. Data from Pew Research Center suggests that there has been basically no change in the public’s baseline views on the issue over time.
The polling gets more complicated when you ask more detailed questions, but there’s a lot of stability around the simple questions. The General Social Survey has asked a battery of questions about abortion regularly for decades (e.g., if a woman should be able to get an abortion for any reason, if the mother’s health is in danger, if she doesn’t want more children), and the trend lines basically don’t bend.
Fox News, which, despite its rightward editorial slant, publishes very accurate polling, has found a little more variation in how many people self-identify as “pro-life” or “pro-choice” over the past two decades, but opinion is basically stable there, too. Americans have collectively moved on other social issues (e.g., marriage equality and marijuana legalization), but they haven’t moved much on abortion. So you should be cautious about polls that show a big shift on abortion; the ups and downs of national politics tend not to move these numbers.
The sponsor of the poll is also part of the context. This poll is sponsored by the Knights of Columbus, which is a Catholic organization. I tend not to use polls in which an interested party is the sponsor in other contexts because it’s not always reliable. For example, I left party-sponsored polling out of my election prediction model last cycle because party-backed polls are often biased and at times seem to be directed at driving a narrative. Not everyone avoids these polls as I do, and not every sponsored poll is bad. But as a general rule, I would advise you to at least mentally (or if you’re building a statistical model, mathematically) discount results such as these somewhat and be more generally skeptical of these polls than you might be of garden-variety surveys.
Think in the medium to long term — especially on policy
Even if there is a bump in pro-life sentiment, it’s not clear to me that it’s going to last. If the events in New York and Virginia really did change people’s minds about abortion, they might change back to their original position when other events push abortion out of the news. This is exactly what happens to Trump’s approval rating: Events (such as the last government shutdown) will push his numbers down, but when that event leaves the news, his numbers start to tick back to their normal levels.
Remain both calm and flexible
The basic takeaway here is that you should remain calm about this poll but not be too rigid in your read of the situation. This poll alone doesn’t prove that America has taken a big leap in the pro-life direction. It might just be an outlier in a mostly stable trend line. But we also shouldn’t be closed off to outlier polls. If America does change its mind and suddenly becomes strongly pro-life or pro-choice, the first poll to show that will look like an outlier. But for now, the Marist poll looks like an outlier, and there are good reasons to doubt that it’s the start of a new trend.