New Data On Health: Studies in Confusion
Sunday, February 19, 2006
For women confused by the latest flurry of health advice about low-fat diets, calcium, Vitamin D and hormones, there is a good reason: The findings illustrate how unexpected pitfalls during a major scientific study can yield valuable data but few clear answers.
Research frequently moves in unexpected fits, starts and sometimes puzzling increments. In the case of long, complex projects such as the Women's Health Initiative, the 15-year, $725 million federally funded project that produced the latest results, questionable assumptions, design decisions and unexpected developments can conspire to generate perplexing results.
"We scientists are scratching our heads over some of these results," said Tim E. Byers of the University of Colorado Health Sciences Center in Denver. "So I'm sure the general public is doing so as well. Unfortunately, it's not always easy to wrap things up in a nice bow and say, 'This is the answer.' "
Findings announced over the past two weeks have seemed to overturn long-held medical dictums: Low-fat diets do not clearly reduce the risk for heart disease, breast cancer or colon cancer; hormone therapy is not dangerous for the hearts of younger menopausal women and may actually be protective; and calcium and Vitamin D supplements do not appear to offer the strong protection against broken bones and colon cancer that had been thought. Yet no one is saying fat does not matter, hormones are necessarily safe or supplements are useless.
"I can see how a lot of women might be confused," said Jacques Rossouw, who runs the Women's Health Initiative. "People would like very clear results with a very clear health message, and, unfortunately, these results are not very clear."
Seeking to explain the results, Rossouw and others cited a host of bedeviling factors: Some of the hypotheses used to design the project may have been flawed, or became outdated while the project was underway. It was much harder than anticipated to get participants to take their pills or stick to their diets. Americans started eating differently and taking new medications, perhaps weakening the findings. The project may have been too short, or studied women who were too old or just too healthy.
Because of the weakness of many of the findings, the results have produced conflicting interpretations, with competing camps seizing on subsets of data that support their views. And subtle but important nuances may have been lost when trying to communicate the results quickly to the public.
"Women are being flooded with a lot of information from these studies," said Sherry Marts of the Society for Women's Health Research. "The challenge is trying to put this rush of information in context."
Many researchers emphasized that in many ways the project was very well conceived, designed and executed, and has produced valuable information -- most important, warning older women about the dangers of long-term hormone therapy.
"It's the most important trial of the century," said Adriane-Fugh-Berman of Georgetown University School of Medicine. "It gave us scads of incredibly valuable information. It's important not to lose sight of that."
But many scientists worry that the recent findings will leave the public apathetic about diet and other lifestyle choices, and the government hesitant to fund research.
"If the public and legislators come away saying, 'Oh, we can never get an answer from these scientists. It's useless to spend money testing diet' -- then a lot of damage will have been done," said Meir Stampfer of the Harvard School of Public Health. "Nutrition is important. We've already learned a lot that offers people plenty of sound advice. But there is a lot more we can learn. We just have to approach it in a smarter way."
The confusion, and the wrong turns that have become clear only in hindsight, have left many researchers chastened about attempting such ambitious projects in the future.
"It's very sobering," Byers said. "What it says for the future is that when we do studies, especially studies that are big and expensive and long, we have to make sure we are testing the right interventions and testing them the right way. That can be a lot harder than it seems."
One of the biggest problems in this kind of research is deciding exactly what to test. When the initiative was being planned, for example, studies that looked at what people in different countries ate indicated that the total amount of fat consumed appeared to have a major effect on the risk of certain cancers, especially breast and colon cancers. So the project set out to test that idea by studying more than 48,000 women, working intensively to get half of them to eat less fat and more fruits and vegetables. Because the study was also examining the effects of taking hormones on heart disease, researchers decided to examine whether that diet could reduce that risk as well.
The final results found no overall reduction in the risk for any of those illnesses, which did not surprise Stampfer and others who lobbied against doing the low-fat study. "My view at the time was that this was not a hypothesis that was strong enough to warrant testing at such great expense," Stampfer said. "Other people looked at the same data and interpreted it differently."
Rossouw and others said the decision was justified based on what was known at the time. "There was data coming from a number of international studies and from people born outside the United States who moved here and increased their fat intake," said Elizabeth G. Nabel, director of the Heart, Lung and Blood Institute. "Based on that, it was very reasonable at that time to believe that lowering your total fat would reduce your risk for breast cancer and colon cancer."
And that may still turn out to be correct, several researchers said. The study came close to showing a statistically meaningful reduction in breast cancer among women on the low-fat diets, and it did find a reduced risk for some subgroups of women, such as those who were eating the most fat at the outset and cut their fat intake the most over the next seven years.
Other scientists dispute those findings, saying those kinds of interpretations are often misleading and unreliable. "Even if you do see a small benefit for, say, breast cancer, you couldn't be sure it was due to fat and not due to the fact that the women also increased their fruit and vegetable intake and there was also a small weight loss," said Walter Willett of the Harvard School of Public Health. "I think it's clear this study was a mistake."
Rossouw and others said the problem could have been that too few women in the study cut their total fat sufficiently for long enough for a clear benefit to emerge. By the time the study began signing up volunteers, a lot of women were eating less fat than when the project was conceived. And as the study progressed, those on the low-fat diet managed to cut their fat less than had been hoped, which made the difference between the two groups much smaller than anticipated.
That illustrates another problem: It is often much harder to get people to make significant changes -- and to be honest about it -- than researchers expect. Even though the women in the study were highly motivated, their compliance with the study's requirements fell far short.
The same was true for the calcium and Vitamin D study. Many women took supplements less regularly than expected. And more women in the comparison group started taking supplements on their own, which made the difference between the two groups less than might have been needed to show a clear benefit. Many women also started to take prescription drugs to prevent bone-thinning, further muddying the results.
Although the study found the supplements increased bone density slightly and decreased the risk of hip fractures for older women and those taking their supplements regularly, there was no overall reduction in broken bones. The supplements also increased the risk for kidney stones.
Picking the right group to study can be tricky. In the hormone study, researchers focused primarily on elderly women because they face the greatest risk of heart attacks. The study found that estrogen not only did not protect their hearts, it increased the risk of heart attacks, strokes and breast cancer.
But because so few younger women were included in the study, an intense debate continues about whether the hormone may protect their hearts if they take it before their arteries start to become diseased.
The failure to show a benefit from low-fat diets in preventing heart disease stems from the decision to add that question, almost as an afterthought, to a study that originally focused on cancer. Researchers knew that the key to preventing heart disease was lowering saturated fat. But because the focus of the study was total fat, most of the women did not reduce their saturated fat enough to make a major difference.
"I think we should have lowered expectations of what a lifestyle intervention trial can show over a relatively short period of time," Rossouw said. "Against a climate of less research funding overall, it's going to be harder to do these kinds of trials in the future."