Are women more likely to say ‘I don’t know’ to poll questions? It appears so.

May 16, 2014

A fascinating post at The Upshot written by pollster Allison Kopicki introduces an interesting idea: that women are more likely to choose the "I don't know" option in polling. If our quick analysis is any guide, she's right.

Kopicki offers an example:

For example, we asked, “Would you like to see Marco Rubio run for president of the United States in 2016, or not, or don’t you know enough about Marco Rubio to say?” The results, at first glance, gave Mr. Rubio what looks like an 8-point advantage among Republican and independent men over women. But in reality, his “don’t knows” among Republican and independent women were 17 points higher. So it was actually hard to tell what women really think of him.

She theorizes on the reason. Pollsters are asking people for their opinion on a topic, assuming they've formed one. Research, Kopicki notes, indicates that "women hold themselves to a higher threshold of certainty before they offer an opinion," so given the chance for nuance, they take it.

We wanted to test this hypothesis. We went to Public Policy Polling, a Democratically-aligned polling firm that offers a particularly helpful for our experiment: public cross-tabs on poll data; that is, a breakdown of answers by gender and age and party. (It's also helpful to use data from one firm, because the methodology will be consistent.) We looked at eight recent polls, extracting 98 politics-related questions. (The polls are listed at the bottom of this article.)

Over the course of those 98 questions, a higher percentage of women than men responded "not sure" to the question 70 percent of the time. And on average, they expressed about a third more uncertainty. (In other words, if 6 percent of men said "not sure," on average, 8 percent of women did.)

What was really striking was that one category of questions was noticeably different. Twenty-four of the questions asked poll respondents to choose a 2016 presidential favorite between Hillary Clinton and another candidate. On those questions, the difference evaporated: half of the time, women expressed less uncertainty, half of the time, they expressed more.

When we took those questions out of the broader pool, then, the difference between men and women widened. On non-Hillary horse-race questions, women said "not sure" at a higher rate than men 77 percent of the time. And the level of uncertainty also increased, from about a third to 43 percent higher.


One last note. As the level of uncertainty overall increased, the extent to which women expressed a higher level of uncertainty dropped. The more uncertain men were, the smaller the gap between the genders.

This is far from definitive (and uses a small sample size) but it certainly supports Kopicki's theory and experience. Does it mean polling is inaccurate? No, but as in the Rubio example it suggests that gender differences when asked to choose between candidates may be overstated. Kopicki says that it does influence her as a pollster: Her firm no longer allows an "I don't know" option.


The PPP polls we used: New Mexico (Mar. 27), Michigan (Apr. 9), Texas (Apr. 14), Wisconsin (Apr. 24), Arkansas (May 1), North Carolina (May 5), North Carolina (May 14), Alaska (May 14).

Philip Bump writes about politics for The Fix. He is based in New York City.
Continue reading
Comments
Show Comments