The Washington PostDemocracy Dies in Darkness

Video: If you don’t like the answer, question the science

Rep. John Fleming (R-La.) criticizes a study on the health impact of mountaintop removal mining in rural Kentucky. (Video: House Committee on Natural Resources)
Placeholder while article actions load

It is routine these days to complain about the politicization of science. It’s rarer to really unpack what that means, or how it happens.

But a hearing Thursday before a subcommittee of the House Committee on Natural Resources captured, in action, a process that has been dubbed motivated skepticism — generating a series of arguments to try to undermine a scientific study with which one disagrees.

In the video above, Louisiana Republican congressman John Fleming, a family practitioner and former Navy doctor, spends five minutes challenging an epidemiological study by Michael Hendryx, a public health researcher at Indiana University, on the health effects of living in the proximity of mountaintop mining.

The study in question, which appeared in the Journal of Rural Health, used student volunteers to collect health information from people in two Appalachian communities, one closer to and one farther from mountaintop mining. And it found “significantly poorer health conditions” in the former community, including “self-rated health status, illness symptoms across multiple organ systems, lifetime and current asthma, chronic obstructive pulmonary disease, and hypertension.”

In the video, Fleming — who may run for the Senate next year if current Louisiana senator David Vitter succeeds in his race for governor — rather combatively disputes the study virtually from start to finish. He asks whether the student volunteers may have been “biased,” questions the “self-reported data” and the sample size, and objects to the fact that the two communities were, unsurprisingly, not exactly the same. For instance, there were differences in the prevalence of smoking across the communities.

Here’s an example from the exchange:

Fleming: It says ‘current smoker’ in the mountaintop mining was 38.2 percent, in non-mining was 29.9 percent. Almost a ten percent difference. More smokers in the mountaintop mining group. Do you see any problem with that at all?
Hendryx: No, because we controlled statistically for smoking in the analysis.
Fleming: Well, no sir, you didn’t control.
Hendryx: Yes, sir, we did.
Fleming: No, because you’re going to have more cancer in the smoking group. There’s no question.

Actually, smoking is something of a classic variable to control for in epidemiological studies. Here’s an example that I found quickly from the web, explaining some basics on how to conduct epidemiological research so as to avoid the problem of confounding variables:

An example of confounding: When examining the relationship between alcohol consumption (E) and heart disease (D), smoking (C) would be an important confounding factor, since smoking is correlated with alcohol consumption and smoking is associated with heart disease [2].

The exchange grew particularly heated after Fleming declared that “I’ve seen fifth grade science projects that were more scientific,” and Hendryx responded, “You know, I came here not to be insulted.” Later during the hearing, Hendryx got a chance to respond more fully to Fleming’s criticisms.

The point here is not that Hendryx’s study is “right” or “wrong” — rather, it’s that this sort of cross examination is not a good way to assess the validity of scientific information. There’s a peer review process for that. Moreover, there is also (almost always) a body of research that extends well beyond any individual study — which has to be assessed in its entirety to grasp the actual state of knowledge.

Reached by phone afterwards, Hendryx said he was appearing as a witness for the Democratic minority on the committee, and that the study in question had been peer reviewed and had not previously been a target of much criticism. He again called the experience “insulting.”