Science, writ large, has an ongoing challenge in communicating its ideas and conclusions to the general public. Separately, it has an internal, rearguard problem with irreproducible results. These are distinct battles, and shouldn’t be conflated. The reproducibility issue primarily involves laboratory experiments so laden with secret sauce and customized techniques that no one else can quite get the souffle to rise. The two issues overlap only to the extent that a major retraction or scientific blunder doesn’t help the mainstream scientific institutions when they want to tell the world that they really do know what they’re talking about when they discuss climate change, vaccines, genetic engineering, nuclear power, etc. But the reproducibility problem isn’t why some people reject the scientific consensus on certain issues. These issues have been cultural wedge issues and you’ll never fix that through more robust laboratory protocols.
Dan Vergano at National Geographic reports on the new Pew/AAAS survey showing the huge gap between scientific and public attitudes when it comes to such issues as whether GMO foods are safe to consume. Chris Mooney of our new Energy and Environment blog analyzes the Pew poll results. Aaron Blake at The Fix also provides a report. Here’s the AP story. And here’s the Pew report. You’ll see massive gaps between what scientists think and what the public thinks, in aggregate. There are a couple of exceptions: The International Space Station, for example, has somehow failed to become a wedge issue. Someone surely is working on that right now. [Suggestion: Someone should reveal that three Russians have somehow found a way onto our space station!]
A couple of days ago I published a big story on reproducibility that I had started in August, and it had been just about ready to go into the paper when I had to stop everything to jump on the Ebola crisis.
First, the nut graphs:
Too often, experimental results can’t be reproduced.
That doesn’t mean the results are fraudulent or even wrong. But in science, a result is supposed to be verifiable by a subsequent experiment. An irreproducible result is inherently squishy.
And so there’s a movement afoot, and building momentum rapidly. Roughly four centuries after the invention of the scientific method, the leaders of the scientific community are recalibrating their requirements, pushing for the sharing of data and greater experimental transparency.
So you see that, as a news story, it lacks the urgency of Ebola. But it’s a huge issue. It’s global. I expect this story to be read around the world and translated in other languages and, you know, revered, and studied as closely as the Talmud.
But just to be clear: This is not a new topic. We’re not really breaking any news here. People have been writing about the irreproducibility problem in science for many years. For example, here’s a piece in The Atlantic in 2010. Here’s one from The New Yorker in 2012. Here’s our own Fred Barbash writing about Retraction Watch in 2014. Here’s another 2014 piece, from The Scientist, and another, from The Monkey Cage.
What’s new, it seems to me, is the intensity with which powerful institutions in the scientific world — the NIH, the journals Science and Nature, major foundations — have decided to get together and find solutions. They’ve organized meetings, revised guidelines, funded new efforts like the Center for Open Science in Charlottesville (which I visited as part of my reporting).
See this editorial from Marcia McNutt at the journal Science.
And this article in Nature by Francis Collins and Lawrence Tabak on the plans at NIH to run a tighter ship.
I asked my sources: When and what was the turning point that took this idea and propelled it into the center of the conversation among leading scientific organizations? Several mentioned the Stapel case (which is why I led my story with it). But the truth is simply that science has hit a near gear — there’s so much more of it, and it’s so much more readily spread via the Internet. A major retraction anywhere sends ripples across the planet. In a world increasingly dependent on science and technology, it’s all the more critical that research be robust and avoid the perils of data dredging, publication bias, confirmation bias, motivated cognition, and arm-waving instead of empirical rigor.
Here’s my kicker:
The scientific enterprise is growing at phenomenal speed, spurred by a hunger for knowledge and an awareness that science usually delivers reliable answers about the nature of the world.
Brian Nosek offers up a stunning factoid: More than half of the scientists who have ever lived are alive today.
Though, yeah, someone ought to double-check that.
[More links to come…]
[Here’s Keith Kloor’s take, which includes many links to other articles on the Pew survey. Notable example: Yale’s Dan Kahan, a leading authority on cultural cognition, says there is no significant evidence in this poll for a creeping “anti-science" sensibility in the U.S.]