Making Sense of Science Reporting

FIAR Director George M. Carter's chief complaint was that stories emphasized a change in "relative risk" -- a 44 percent fall in the number of heart attacks, strokes and surgical procedures among people taking the statin, compared with those in the placebo group. He said the fact that everyone in the study had an extremely low "absolute risk" for heart problems should have been emphasized more. About 1.36 percent of people taking the placebo suffered a heart attack or stroke; that fell to 0.8 percent among those taking the statin. That means that nearly 97 percent of the people using the drug would not see any benefit, he said.

Stein quoted a skeptic in the ninth paragraph and noted near the story's end that "the actual risk reduction for an individual would be very small, given the relatively low risk for most middle-age people, so that the benefits easily could be outweighed by the costs of thousands more people taking tests, drugs and being monitored by doctors."

Stein said, "While I would have liked to have explored many of the nuances of this study more fully, I feel confident we struck a responsible balance. I think it's crucial to provide readers with both the evidence supporting new claims and enough context and interpretation to help them gauge its significance." Independent experts, he said, concluded the study was "a very well done, very convincing piece of research."

One of the issues in science reporting is that most readers aren't schooled in statistics. Harold Varmus, former director of the National Institutes of Health, recommends looking more deeply into the numbers. "The percentages may be high, but what is the risk of an event in the first place? If the risk is low, there's a much smaller benefit." Varmus, a Nobel laureate, is chief executive of Memorial Sloan-Kettering Cancer Center.

Marcia Angell, a physician and former editor of the New England Journal of Medicine who is now a senior lecturer at Harvard Medical School, said journalists can write "overly dramatic" stories for "gullible" readers. "Everyone has an interest in hyping news of medical research -- the researcher, the institution, reporters. Readers should be very skeptical of new findings. Newspapers are in the business of telling you the news, which needs to be startling or counterintuitive or flies in the face of what we knew. By definition these stories are less likely to be accurate."

Don J. Melnick, professor of conservation biology at Columbia University, said that if a story "doesn't sound newsworthy or front page-worthy, it will be buried or not printed at all. That tends to promote people hyping the research. They have to convince their editors to put it in the paper."

Nils Bruzelius, The Post's science editor, said, "I thought the story and Page 1 play were justified because the potential impact was significant, even as I understand the criticisms. There's an inevitable tension between the desire of reporters and editors to get good play for their stories and the need to avoid hype or overstatement, and we feel this very acutely in dealing with scientific or medical stories, because the advances, even those that prove to be part of something very big, usually come in incremental steps. I've long believed that science and medical stories enter this competition at some disadvantage. I certainly don't have data on this but I suspect that most of the top editors who make the front-page decisions tend to be less drawn to these topics than the average reader because, with a few exceptions, they are a naturally self-selected group who got to where they are by dint of their interest and ability in covering such topics as politics, international relations, war and national security -- not science."

· Who sponsored the research and who makes money from its findings? Angell, a critic of drug companies' influence on medical research, said, "The caveats are at the end [of the story]. The pharmaceutical industry is spreading money everywhere and the researchers have their hands out."

That was true of the statin story. In the last six paragraphs, readers learned that the study was financed by AstraZeneca, which makes the statin Crestor, and that the study's author and his hospital will receive royalties on the blood test that was studied. Drugmakers fund many large medical studies. The story said that the company had no influence over the analysis.

Varmus said there is no mechanism for support or motivation to conduct clinical trials without drug industry money. "Obviously, companies have a vested interest in a good outcome and being truthful and getting answers that won't cause them grief later on," he said. Such trials also must follow Food and Drug Administration regulations.

"It's not new that the industry is the primary source of funding clinical research," Angell said. "What is new is the strings attached and the willingness of medical schools and faculty to accept these strings. They have influence over every detail of clinical trials."

Jonathan Weiner, who teaches science writing at the Columbia University Graduate School of Journalism, said, "It's a very messy, complicated problem. With government funding tight, many doctors rely on industry for funding. People in research medicine can't stay current without going to industry-funded conferences that have the quality of junkets." Weiner wrote "The Beak of the Finch," a book about evolutionary biology that won the Pulitzer Prize for general nonfiction in 1995.

For readers, Brown's best advice is this: "In the end, all that counts is evidence."

A longer version of this column appears online. Deborah Howell can be reached at 202-334-7582 or at ombudsman@washpost.com.