Costs are soaring as cooperation rates remain at or near record lows. In some surveys, less than one in five calls produces a completed interview -- raising doubts whether such polls accurately reflect the views of the public or merely report the opinions of stay-at-home Americans who are too bored, too infirm or too lonely to hang up.
Shrinking Response Rates
Rick Klaastad, 41, of Springfield, Ore., had been working as a telephone interviewer on opinion polls for about a year when he dialed a number and an elderly man answered the phone.
After the Minneapolis Star Tribune published a poll showing John Kerry with a 9-point lead over the president, pro-Bush demonstrators gathered outside its offices demanding that Rob Daves, the newspaper's pollster, be fired.
(Minneapolis Star Tribune)
"He declined to be interviewed," recalls Klaastad, who works for TNS of Horsham, Pa., the research firm that conducts the Washington Post-ABC News surveys. "But apparently he has some electronic device attached to the phone and the next thing I hear was this loud, painful sound. It was about as bad as it gets -- imagine turning an FM stereo up full blast with a really good set of speakers. It bruised my eardrum."
Not all Americans say no so rudely -- or painfully. But far more refuse to participate in surveys today than a few decades ago. In the 1960s, it was common for two-thirds of those contacted to complete a telephone survey. But participation dropped steeply through the 1980s and early 1990s, when it appears to have leveled off.
No surveys are immune. "Phone surveys are suffering, but so are response rates to mail surveys and even mall intercept surveys" in which people are interviewed while shopping, says Tom Smith, director of the General Social Survey at the University of Chicago, the best source of data on social trends in the United States. "All of the dominoes are being knocked down because the whole table is being shaken."
Currently cooperation rates hover at about 38 percent for the big national media surveys conducted over several days, but can dip down into the teens for surveys completed in a single night, says Jon Krosnick, a psychologist at Stanford University who has completed a groundbreaking study of response rates.
Even exit polls are feeling the pinch. In each of the past three presidential elections the proportion of people who agree to be interviewed after leaving the voting booth has dropped -- from 60 percent in 1992 to 55 percent in 1996 to 51 percent in 2000.
For decades, the conventional wisdom has been that high response rates equaled high-quality, more accurate surveys. Generations of pollsters-in-training were told in graduate school that the people who decline to participate in a poll, or cannot be reached, could be different than those who are contacted, in ways that would affect results.
Two converging trends -- the rise of telemarketing and growing time pressures in the home -- have frayed America's nerves and left many people unwilling or downright hostile when it comes to talking to pollsters. But a bigger problem seems to be that people are simply harder to reach. They're working longer, going out more and using call-screening devices when they're home, Krosnick says.
If anemic response rates are polling's dirty little secret, new research suggests this dirty little truth: It may not matter much -- at least not yet.
Last year Krosnick collected data from the five largest media polling operations, including The Washington Post, to determine what impact response rates had on the results.
"As far as I can tell, those who don't participate in surveys are busy people or people who maybe are unhappy with being deluged by telemarketers. But either way, it doesn't seem to be powerfully correlated with the kinds of things people measure in polls. Missing those people doesn't distort our predictions appreciably."
Some research organizations are taking extraordinary steps to increase response rates. Last year, Nielsen Media Research saturated Chicago with slick advertisements designed to raise public awareness of the company and increase the likelihood that people would participate in its media usage surveys.
The ads were deliberately over-the-top. One featured a gaggle of Elvis impersonators. Another starred a pouty-lipped, slightly disheveled hunk in sunglasses above the caption: "What do you see? No matter what you see, tell us what you watch," and identified the sponsor as the Nielsen Ratings: "We listen to TV viewers."