By Richard Morin
By Richard Morin
There's new evidence suggesting that Internet polls may be as reliable as telephone surveys someday though certainly not today and perhaps not any time soon. The data comes from Andrew Kohut and the Pew Research Center for the People & the Press in Washington, D.C. Kohut recently directed a test in which he simultaneously conducted a survey of registered voters over the Internet and a companion poll using standard telephone survey methodology.
"The findings ... are remarkably similar on some important issues," Kohut reported. "But conflicting results on other questions reveal significant attitudinal differences between the general public and those who participate in online polls."
Specifically, he found that Web users who agreed to fill out and return an online poll paid "closer attention to election news, place a slightly higher priority on national issues and are more supportive of Clinton's impeachment." These differences, he noted, persisted "even when the online sample is statistically adjusted to account for the under-representation of some demographic groups."
His verdict: Online polling today offers mixed results. That doesn't mean online polls can't or won't get better (or aren't useful for surveying some kinds of elite groups, such as professors or journalists). Indeed, the potential for conducting polls over the Internet has prompted a flurry of interest from professional pollsters.
It's no surprise why survey researchers grow weak-kneed over the thought of reliable online polls. "With the Internet's tremendous growth, an online poll can now compile literally tens of thousands of opinions quickly and at a fraction of the cost of traditional telephone surveys," Kohut noted in his summary of his Pew experiment.
The problem, of course, is that the Net still doesn't look exactly like America, making even random-sample surveys of Internet users unreliable gauges of public attitudes. About 40 percent of the country use the Internet, "a population that is substantially different from the U.S. population at large," Kohut found.
Other researchers have attacked this problem by advertising for e-mail addresses at various Web sites and later contacting those who volunteer to participate in online polls. The results are then adjusted to correct for the differences between Internet users and the population as a whole.
In the recent Pew Center experiment, Kohut and researchers tested another approach. They first collected e-mail addresses last year from individuals who were called as part of the center's regular randomly selected, nationally representative preelection surveys. "For instance, during three telephone surveys in August and September, respondents who use the Internet were asked if they would participate in a future online survey and, if so, to provide their e-mail address," Kohut said. Out of the 4,473 interviewed by phone, 42 percent said they go online and 36 percent of these Internet users (786) provided e-mail address for a future online poll.
Then a random sample of respondents was selected from the pool of e-mail addresses. Respondents were sent an e-mail invitation to participate in an online survey "and given a link and instructions for going to the World Wide Web to complete the poll" at a site that was unavailable to other users.
A total of 167 registered voters completed the survey, or about 35 percent of the 471 people initially contacted to complete the poll. The Pew Center simultaneously conducted a national telephone survey that asked many of the same questions included in the online poll.
The sample in the Internet poll was clearly small, perhaps too small to be anything but suggestive. Still, apparent differences emerged when the demographic characteristics of the online and telephone polls were compared. "The population of registered voters who go online is younger, better educated and more affluent," Kohut wrote.
Specifically, Kohut found that men comprised 58 percent of the online voter sample, compared with 46 percent of all registered voters in the telephone survey. One in five registered voters in the telephone poll was 65 years or older, compared with just 5 percent in the online poll. Nearly two in three of the online respondents were college graduates, compared with 25 percent in the telephone poll.
On a number of key questions, both polls produced similar findings when the results of the online poll were weighted to national census figures. The two surveys produced "nearly identical" results on questions that measured the chances respondents would vote in the then-upcoming November elections.
The online poll also did "a fairly good job" estimating congressional voting preferences; 47 percent in the telephone survey said they planned to vote for the Democratic House candidate, compared with 53 percent in the online poll. It was closer in estimating the Republican share, with 40 percent in the telephone survey saying they planned to vote Republican compared with 42 percent in the online poll.
The online poll dramatically overstated interest in the election and support for impeaching Clinton: 37 percent of the voters who participated in the online poll said Clinton should be impeached, compared with 28 percent in the telephone survey. "But online respondents did not consistently express more conservative opinions than telephone respondents across all questions," Kohut wrote. "For example, substantially more online respondents also disapproved of the way Republican leaders in Congress are handling their jobs."
The results suggest that Pew's two-stage process, using a national telephone poll to produce e-mail addresses, has considerable merit, though online polling still remains fraught with considerable danger. Still, the Pew results suggest a rosy future for online polls & though the future definitely is not now.
© Copyright 1999 The Washington Post Company