|Page 2 of 2 <|
Cellphones' Growth Does a Number on Health Research
Federal law requires that calls to cellphones be hand-dialed; it is illegal to use automatic dialers, which are standard tools for survey and polling firms. Furthermore, a huge fraction of "owners" of cellphone numbers are children ineligible for the health surveys. Once reached, some cellphone users are reluctant to talk at length because they have to pay for incoming calls.
Consequently, it takes roughly nine calls to working cellphone numbers to get one completed survey, compared with five calls to working land-line numbers, said Scott Keeter, a polling expert at the Pew Research Center for the People and the Press, an independent opinion research group. Further, an interview conducted with someone who uses a cellphone costs 2 1/2 times as much as an interview with someone on a conventional phone. In addition to higher labor costs, most surveys now reimburse cellphone users for their minutes, either in cash or through credits to online merchants such as Amazon.com.
People's willingness to answer questions has also been affected by the barrage of phone calls, many unsolicited, they get every day. The response rate in public opinion polls has fallen from about 60 percent two decades ago to 25 percent now, according to Keeter.
Government-sponsored health surveys have fared better. The "cooperation rate" for the BRFSS in 2007 was 72 percent, the same as in 1994. The smaller, in-person National Health Interview Survey had a response rate of 87 percent in 2006. A decade earlier, it was 92 percent.
But the problem goes beyond changing technology and responsiveness. It turns out that people answer the same question differently depending on how you reach them -- a "mode effect."
For example, when a group of people with the same age, race and education are called on a conventional phone, 25 percent say they smoke, but on a cellphone 31 percent say they do. On a land line, 38 percent say they have been tested for HIV, while on a cellphone 54 percent say they have.
The reason for this is unclear. Ali H. Mokdad, an epidemiologist who until recently was at the federal Centers for Disease Control and Prevention and ran the BRFSS, speculates it may have something to do with the fact that people on land lines are usually at home, where they have a role and image to maintain even if they are answering in privacy.
"They are less likely to say something bad about their own behavior. It's like 'This is my house,' " he said.
A study published three years ago showed that when women were interviewed by phone, rather than in person, they tended to underreport their weight, and both men and women (but men more) tended to overreport their height. Both estimates were exaggerated -- women's weight lower and men's height higher -- compared with surveys in which height and weight were measured.
The study showed the importance of using measurements to determine the national prevalence of obesity, as interview data alone substantially underestimate it.
The internal architecture of a survey can also affect the results.
Where a question falls in a series of them can increase the likelihood of a certain answer, a phenomenon known as "differential item functioning." The ethnicity of the respondent may also make a difference. According to a study published in 2006, Hispanics are more likely to give answers at the extremes of numerical scales "because of a cultural value that associates extreme responses with sincerity."
In all, what seems like a fairly simple and straightforward task is in truth hard and messy.
"It's a bit like making sausage," said Christopher J.L. Murray, a physician and epidemiologist who heads the Institute for Health Metrics and Evaluation at the University of Washington.
"As soon as you start to explore how surveys are made," he said, "you begin to see how difficult it is to get consistent information at the population level over time."