This story has been updated.

Researchers have designed an inventive test suggesting that the arguments commonly used by climate change contrarians don’t add up, not only according to climate scientists (we know what they think already) but also in the view of unbiased experts from other fields.

The trick? Disguising the data — and its interpretation — as if it was part of an argument about something else entirely.

“What we find is that whatever so called climate skeptics say about [climate] data just doesn’t characterize the data adequately,” said Stephan Lewandowsky, the new study’s lead author and a psychologist at the University of Bristol. “It’s as simple as that. It’s judged to be misleading, false, and just incorrect basically,” by independent experts from fields like economics and statistics, he said.

Lewandowsky, often a thorn in the side of climate skeptics or doubters, has previously published data linking rejection of the science of climate change to beliefs in conspiracy theories, leading to withering criticism from said skeptics. The new study was published in a peer-reviewed journal, Global Environmental Change, by Lewandowsky and colleagues from Australia, Switzerland and Norway. One of its authors, Rasmus Benestad, of the Norwegian Meteorological Institute, is a prominent climate scientist.

However, the research has already received some criticism from scholars — suggesting that there could be a significant debate over the meaning and interpretation of the results.

But let’s back up: How is it actually possible to “disguise” climate-related arguments, so that they can be evaluated by other experts who don’t know much of anything about climate change?

First of all, consider that climate doubters (like scientists) often use objective data to back up their claims. They just tend to represent it in ways that scientists have long found objectionable.

Here’s an example: Data indicate that in the long run — over many decades — global temperatures have been rising. But over shorter periods, temperatures might fluctuate up and down quite a bit. Climate contrarians might exploit this fact by pointing to a small block of data from a short-term period when temperatures were on the downswing, or weren’t rising, and use it to suggest that global warming isn’t actually happening. It’s a tactic known as “cherry-picking” — selecting only data that suit one’s purposes, instead of data that reflect the whole story.  

It’s hard to talk about these problems, though, without the conversation turning into a figurative shouting match between mainstream climate science and climate doubters, who are generally going to disagree with one another no matter what. So Lewandowsky’s group of researchers decided to take another tack.

They found a way to let an unbiased group of expert scientists judge for themselves how sound climate-doubting arguments are by presenting them with real climate data — but labeling these data as something else. For instance, they presented data on trends in Arctic sea ice extent, but relabeled as data on the profits of a fictitious company. And they re-cast numbers on global sea-level rise as stats on world lithium production.

“So instead of saying, there’s a recovery of Arctic ice, we would say, there’s a recovery of our share prices,” Lewandowsky said.

“The crucial step in this is you’re taking out all preconceived notions, political biases, people’s emotions about this,” he added.

To set up the study, the researchers first recruited an expert set of economists and statisticians to serve as test participants. None of these participants knew the study had anything to do with climate change.

The researchers also compiled a list of contrarian statements commonly used by climate doubters to question the existence or extent of anthropogenic climate change — for instance, the idea that Arctic sea ice is growing, not shrinking. To make sure these statements were representative of ideas commonly included in contrarian discourse, the researchers conducted a search of prominent climate skeptic blogs, verifying that each chosen statement was a popular hit.

During the test, the researchers featured such a statement, along with the corresponding climate data — but they changed the labels and wording, making it appear that they were displaying information about entirely unrelated topics, such as agricultural output or business profits. They then asked the expert participants to answer a series of questions about whether they thought the given statement confirmed or contradicted the accompanying data; whether the statement seemed misleading; and whether the statement was appropriate for use by policymakers or industries.

Below is an example taken from the study. The graph and the phrasing of the claim and question have been modified slightly for simplicity, but the overall structure of the experiment has been preserved. The data depicted aren’t actually data on rural county populations — they’re for the mass of glaciers, which in most cases are shrinking.

Just to shake things up a little, the researchers also administered the test to some participants using statements supporting the ideas of mainstream climate science (also disguised, of course). The difference between the responses in each case was striking.

“Across two groups of experts and across six different scenarios, contrarian claims were judged to be misleading, inaccurate, and unsuitable for policy advice,” the researchers wrote in the paper. “Conversely, mainstream scientific interpretations were found to be accurate and suitable for policy advice overall.”

“It’s a huge effect,” Lewandowsky added. “They are about as far apart as anything I’ve seen.”

As an added test, the researchers asked the participants to predict what the masked data should look like in the future — and in general, the experts made predictions in line with what’s been projected by mainstream climate scientists.

“So no one thought the Arctic was going to recover — they thought it was going to continue melting, because that is what the data show,” Lewandowsky said.

But not everyone is convinced. According to Jonathan Jones, a physics professor at the University of Oxford who has critiqued previous research published by Lewandowsky, a problem with the study is the data it chooses to begin with.

“The obvious problem is that because they have control over both the choice of dataset used to assess a contrarian claim and over the corresponding ‘consensus claim,’ it is essentially trivial to construct situations where the data supports the consensus claim and opposes the contrarian claim,” he added. “In reality, many of the real arguments are over precisely which dataset to use (there are several competing datasets for global temperatures) and over which time periods to use (recent trends or longer term trends).”

Max Boykoff, a climate communication researcher at the University of Colorado at Boulder, also had a critical take on the study. While he said the research approach was “laudable” because it tried to “systematically examine” how climate change contrarians often misuse scientific information, he had a number of methodological questions about the approach, including noting that “these disembodied statements extracted from the context within which the claims are made can run the risk of misrepresenting or not fully representing what props up the statements.”

Those remaining skeptical of the science of climate change, then, will likely object to the experimental design. But the researchers say the findings strip away preconceptions to prove a basic point: that across disciplines, experts know good science when they see it.

Chris Mooney contributed to this report.

Read more at Energy & Environment:

For more, you can sign up for our weekly newsletter here, and follow us on Twitter here.