The ads were plastered in bus shelters, on the El trains and billboards, in local newspapers and the Chicago editions of national magazines. Nielsen researchers surveyed about 1,000 randomly selected area residents before and after the three-month campaign to measure how effective it had been.
In a presentation last May at the annual meeting of the American Association for Public Opinion Research, Nielsen's lead research analyst, Ana Melgar, reported the results: "Nothing changed. Not one bit. There were no differences, in terms of name awareness."
After the Minneapolis Star Tribune published a poll showing John Kerry with a 9-point lead over the president, pro-Bush demonstrators gathered outside its offices demanding that Rob Daves, the newspaper's pollster, be fired.
(Minneapolis Star Tribune)
Cell Phones Pose Problems
"Anybody who believes these national political polls are giving you facts is a gullible fool," columnist Jimmy Breslin opined last month. "Any editors of newspapers or television news shows who use poll results as a story are beyond gullible. On behalf of the public they profess to serve, they are indolent salesmen of falsehoods."
These lying liars lie because pollsters have not found an effective way to reach cell phone users, he wrote. "There are 169 million phones that they didn't even try. This makes the poll nothing more than a fake and a fraud, a shill and a sham."
A growing proportion of Americans are giving up their household telephones for cell phones. And pollsters admit that this is a potentially huge problem. It's currently illegal for a pollster to call a cell phone number. And even if cell users could be easily contacted, many would be loath to surrender precious minutes to answer a pollster's questions.
At the same time, pollsters argue that the proportion of people who use only a cell phone is tiny. There may be 169 million cell phones in purses or strapped to the belts of Americans, but the overwhelming majority are owned by people who still have a traditional telephone in their home and are reachable by pollsters.
Early this year, a face-to-face survey of 2,000 randomly selected adults found that only 2.5 percent had cell phone service and no traditional home phone. The figures were slightly higher among 18- to-24-year-olds and renters, reported Peter Tuckel of Hunter College in New York, which conducted the study with veteran pollster Harry O'Neill.
"It appears that, as of now, [cellular-only] households are not a major impediment to reaching respondents," they concluded. "The problem derives more from call screening, primarily through Caller ID, and respondent refusals."
That's cold comfort for some pollsters. Some see the day when they will have to abandon the telephone and are experimenting with new ways to measure public opinion.
"The people who are using telephone surveys are in denial," pollster John Zogby told Breslin. "It is similar to the '30s, when they first started polling by telephones and there were people who laughed at that and said you couldn't trust them because not everybody had a home phone. Now they try not to mention cell phones. They don't look or listen. They go ahead with a method that is old and wrong."
Zogby's critics quickly noted that his own election polls are done by telephone -- a point that Zogby acknowledged in a contrite statement issued a few days after Breslin's column ran. "I still conduct telephone polls," he wrote. "The reality is that polling on the telephone is becoming more difficult. . . . That said, I feel that representative samples can still be achieved on the phone." He also noted that he was "equally confident that my interactive surveys have reached a point where they are valid."
Zogby is not the first pollster to experiment with doing surveys on the Internet, either out of curiosity or out of desperation that the end of telephone surveys is close at hand.
Knowledge Networks, a Palo Alto-based firm, does surveys over the Internet with randomly selected households that have been previously recruited. If families don't have a home computer or access to the Internet, Knowledge Networks gives them a keyboard and a device that connects them to the Internet and allows them to use their television as a monitor -- a red light comes on when there's a survey to complete.
Harris Interactive now does the widely followed Harris Poll entirely online, contacting adults recruited at other Web sites via pop-up ads. They claim that their surveys are at least as accurate as telephone polls, and about 20 to 25 percent less expensive.