Specifically, the so-called P value — a mathematical measure of the strength of the evidence that there’s a true difference between a treatment and placebo — was 0.08. It needs to be 0.05 or smaller to be considered “statistically significant” under the conventions of medical research.
Technically, the study was a bust, although the results leaned toward a benefit from interferon gamma-1b. Was there a group of patients in which the results tipped? Harkonen asked the statisticians to look.
It turns out that people with mild to moderate cases of the disease (as measured by lung function) had a dramatic difference in survival. Only 5 percent of those taking the drug died, compared with 16 percent of those on placebo. The P value was 0.004 — highly significant.
But there was a problem. This mild-to-moderate subgroup wasn’t one the researchers said they would analyze when they set up the study. Subdividing patients after the fact and looking for statistically significant results is a controversial practice. In its most extreme form, it’s scorned as “data dredging.” The term suggests that if you drag a net through a bunch of numbers enough times, you’ll come up with something significant sooner or later.
Exactly what Harkonen was thinking isn’t known, as he didn’t testify at his trial. Nevertheless, the press release’s two headlines focused all the attention on the mild-to-moderate subgroup.
“InterMune Announces Phase III Data Demonstrating Survival Benefit of Actimmune in IPF,” the document said in bold-face letters. Following it in italics was this sentence: “Reduces Mortality by 70% in Patients with Mild to Moderate Disease.”
Those two sentences were Harkonen’s crime.
‘No falsification of data’
In the trial, much was made of P values and the issue of after-the-fact analyses.
Two of the government’s experts testified that if a study misses all its “primary endpoints” (as this one did), then it’s improper to draw conclusions about a drug’s effect in subgroups identified later. The press release acknowledged missing the primary endpoints, but it didn’t indicate that the featured subgroup was identified after the study’s data were collected.
The prosecutors also emphasized that Harkonen had a financial motive for spinning the study in the most positive way. This wasn’t hard to find. The third paragraph of the press release said: “We believe these results will support use of Actimmune and lead to peak sales in the range of $400-$500 million per year, enabling us to achieve profitability in 2004 as planned.”