In 2013, one landmark study gave the strongest proof yet in one of the first major clinical trials to measure the diet’s heart benefits. The study, conducted in Spain, showed consuming a Mediterranean diet can lower the risk of a heart attack, stroke or death from heart disease by about 30 percent in those at high risk.
The five-year experiment, published in the esteemed New England Journal of Medicine, made international headlines and was hailed as a triumph.
But on Wednesday, the study’s authors took the rare step of retracting their report. The researchers chose to withdraw their original paper and publish a new one after facing criticism of the way the initial experiment was conducted.
The findings of the revised study arrive at the same conclusions as the original one — that the Mediterranean diet can prevent heart disease. But the language in the new report is a bit more modest.
The original study concluded the diet “resulted in a substantial reduction in the risk” of major heart illness among high-risk people, while the new study said “those assigned” to a Mediterranean diet had a lower risk than those not assigned.
Despite this softened language in the report, the lead author on the study, Miguel A. Martínez-González of the University of Navarra, told The Washington Post that the causal link is just as strong as the original report.
“We are now more convinced than ever of the robustness of the trial and of the conclusions,” Martínez-González said, particularly because “no previous trial has undergone such intense scrutiny.”
Still, the retraction and replacement of such a major study in one of the world’s most reputable journals raised eyebrows in the medical world. If the original study was so problematic that the authors chose to withdraw it entirely, could the new one be trusted?
“Sad day for credibility of major medical journals and nutritional science,” tweeted Sekar Kathiresan, a physician scientist, human geneticist and director of the Center for Genomic Medicine at Massachusetts General Hospital.
The unusual chain of events began with the work of one determined sleuth, a British anesthesiologist named John Carlisle.
Since the early 2000s, Carlisle and a few others had been noticing red flags in studies published by academic anesthesiologist Yoshitaka Fujii. Nothing much came of it until about 2009, when Carlisle wrote to the top editor of the journal Anaesthesia, raising doubt that Fujii’s data was produced by real-life experiments.
The issue had to do with randomized controlled trials, which divide participants by chance into separate groups that compare different treatments. Using chance “means that the groups will be similar and that the effects of the treatments they receive can be compared more fairly,” according to the National Institutes of Health.
“If you’re making up data, the temptation is to make your groups more similar than what would happen under natural circumstances,” Carlisle told The Post.
Carlisle urged the journal to analyze the data in the Fujii studies, to which the editor responded, “Would you mind doing the work?” Carlisle recounted. While he was no statistics expert, Carlisle agreed. “I don’t know what it feels like to be a detective, but I thought it felt like being a detective.”
Using online tutorials and YouTube videos, Carlisle taught himself how to sift out patterns and anomalies. He specifically looked for cases in which participants in studies were not assigned randomly to treatments, even when the trial portrayed itself as random.
Carlisle published a report in the journal Anaesthesia in 2012 explaining how and why Fujii’s data was too good to be true. And the medical world took notice. Investigations would later make clear that Fujii had fabricated most of his data. As of 2015, he held the record for most retractions by a single author — 183, as Nautilus reported.
In the years since, Carlisle applied the same kind of sleuthing to reports beyond anesthesiology. In 2017, he published a report that took a close statistical look at 5,087 randomized, controlled trials over a 10-year period. His analysis pinpointed, among other things, 11 reports in the New England Journal of Medicine with baseline variables that appeared “almost impossible” to happen randomly, Carlisle said.
Within days, the journal began to review the 11 reports. Of those, only one presented legitimate concerns, the journal said — the 2013 Mediterranean diet study. His analysis questioned whether some participants had actually been assigned diets at random.
After Carlisle published his 2017 report, Martínez-González and his team of researchers contacted the journal, which urged them to perform a new analysis.
Martínez-González and his team went about reevaluating their data, finding that while some of Carlisle’s findings were “based on mistaken assumptions,” there were two main problems.
The Mediterranean diet trial, which took place at 11 locations in Spain, involved assigning more than 7,000 people at high risk of heart illness to one of three diets: a Mediterranean diet supplemented with extra-virgin olive oil, a Mediterranean diet supplemented with mixed nuts, or a simple low-fat diet (the control group).
It turned out that among about 10 percent of the participants, the same diets had been assigned to individuals in the same household. If a married couple joined the study together, for example, both participants would follow the same diet. For these participants, assignment was therefore not random. Family members share many environmental variables, making it difficult to identify whether dieting alone led to health outcomes.
Martínez-González said the authors “inadvertently” forgot to account for the issue in the study’s protocol.
In addition, at one of the 11 study locations, a researcher assigned the same diet to an entire village, which Martínez-González described as a “cluster.”
In their new report, researchers made statistical adjustments to correct for these issues. The team “took more care to adjust for potential imbalances,” Martínez-González said.
The revised report, which has replaced the retracted study, did not rely on the assumption that all the participants were randomly assigned.
Despite these changes, the results remained intact, Martínez-González said.
But as Alison McCook of Retraction Watch wrote for NPR, “would the New England Journal of Medicine, one of the world’s most selective and prestigious journals, have published the Mediterranean diet paper from Martínez González if the randomization issues and softened conclusion had been included in the first place?”
“It is difficult to answer hypothetical questions,” New England Journal of Medicine spokeswoman Jennifer Zeis told McCook. “’We believe the evidence [in favor of the diet] is still strong, but not as strong as a randomized study in which the randomization was executed flawlessly.’”
While the conclusions were the same in this case, Carlisle urges readers to always remain skeptical. He argues that all medical journals could do a more thorough job of reviewing papers and data before publication.
For those looking to make decisions based on scientific research — such as whether to pursue a Mediterranean diet — “don’t put all your eggs in one basket,” Carlisle said.
“Whatever you believe, it shouldn’t depend upon just one paper,” Carlisle said. “Try to have not only a balanced diet but also a balanced view.”