Here are the opening paragraphs from Why Psychologists’ Food Fight Matters: “Important findings” haven’t been replicated, and science may have to change its ways (paragraph break added):
Psychologists are up in arms over, of all things, the editorial process that led to the recent publication of a special issue of the journal Social Psychology. This may seem like a classic case of ivory tower navel gazing, but its impact extends far beyond academia.
The issue attempts to replicate 27 “important findings in social psychology.” Replication — repeating an experiment as closely as possible to see whether you get the same results — is a cornerstone of the scientific method. Replication of experiments is vital not only because it can detect the rare cases of outright fraud, but also because it guards against uncritical acceptance of findings that were actually inadvertent false positives, helps researchers refine experimental techniques, and affirms the existence of new facts that scientific theories must be able to explain.
One of the articles in the special issue reported a failure to replicate a widely publicized 2008 study by Simone Schnall, now tenured at Cambridge University, and her colleagues. In the original study, two experiments measured the effects of people’s thoughts or feelings of cleanliness on the harshness of their moral judgments. In the first experiment, 40 undergraduates were asked to unscramble sentences, with one-half assigned words related to cleanliness (like pure or pristine) and one-half assigned neutral words. In the second experiment, 43 undergraduates watched the truly revolting bathroom scene from the movie Trainspotting, after which one-half were told to wash their hands while the other one-half were not.
All subjects in both experiments were then asked to rate the moral wrongness of six hypothetical scenarios, such as falsifying one’s résumé and keeping money from a lost wallet. The researchers found that priming subjects to think about cleanliness had a “substantial” effect on moral judgment: The hand washers and those who unscrambled sentences related to cleanliness judged the scenarios to be less morally wrong than did the other subjects.
The implication was that people who feel relatively pure themselves are — without realizing it — less troubled by others’ impurities. The paper was covered by ABC News, the Economist, and the Huffington Post, among other outlets, and has been cited nearly 200 times in the scientific literature.
However, the replicators — David Johnson, Felix Cheung, and Brent Donnellan (two graduate students and their adviser) of Michigan State University — found no such difference, despite testing about four times more subjects than the original studies.
I found the article to be very interesting, though I’m not at all knowledgeable about the controversy myself, and thought I’d pass it along. (Someone I know who does know about social psychology agreed.) And I’d of course love to hear what those who are familiar with the matter — not just the Schnall paper, but the broader replication controversy — have to say about it.