It’s no secret that Americans have trouble with scientific authority. We are, after all, a country nearly half composed of creationists who think humans have been around for only 10,000 years or less.
But just as it was once academically fashionable to dis experts, the worm is now turning, and many are now standing up for them again. And to that trend, we can now add empirical evidence in experts’ favor, thanks to a fascinating new study out by Yale law professor and science communication researcher Dan Kahan and a team of researchers and legal scholars (including one judge).
The researchers studied the reasoning practices of a group of 253 judges, and found that in how they make decisions based on evidence — suppressing ideological tugs in the process — they really are quite special.
The experiment involved a pretty fascinating group of subjects — one that you rarely hear described in research literature. There were, as already mentioned, 253 state-level judges, who were recruited through judicial conferences and online continuing legal education. But there also were 225 practicing lawyers, 250 law students and a representative sample of 800 Americans.
All of these research subjects were asked, in the study, to resolve several questions involving “statutory interpretation” — applying legal precedents or principles to “ambiguous” situations. And the questions were deliberately set up in such a way as to trigger strong ideological biases, of the sort that might undermine fair and impartial deliberation.
At the same time, all the research subjects were asked questions to determine their political ideology and how they view a number of issues, such as climate change and legalizing marijuana, that are known to strongly polarize people along ideological lines.
So how do judges perform, compared with average citizens?
In one of the statutory interpretation problems, the task involved applying a law that banned “littering, disposing, or depositing any form of garbage, refuse, junk, or other debris” in a national wildlife preserve. The participants were then presented with a case in which a group of individuals had left “400 ten-gallon reusable plastic water dispensers” in such a preserve – which, in this case, happened to be along the border between the United States and Mexico.
In some cases, however, the water dispensers were said to have been left behind by immigration workers who were trying to help illegal immigrants cross the desert safely by providing them with water. In others, they were said to have been left there by “construction workers, who intended to drink water from the containers themselves while working on a border fence designed to prevent illegal entry into the U.S.”
The reasoning trap is now set: People with strong views on immigration (on either side of the debate) may be inclined to interpret whether “littering” exists here based on those ideological predispositions. But that is faulty reasoning, and shouldn’t matter to the question of whether the individuals described, in leaving the reusable water dispensers in the desert, had engaged in littering under the language of the law.
Now let’s look at the results for how respondents evaluated the law and whether it had been broken in the “littering” scenario:
As you can see, members of the public split pretty evenly on the guilt of the construction workers (when they read that scenario), but tended to see guilt in the immigration workers. However, with increasing levels of legal expertise — progressing from law students, to lawyers, to judges — study participants became increasingly likely to see “no violation” in either case. (After all, the water dispensers were reusable and intended to be reused.)
Indeed, for the judges, more than 70 percent of participants saw no violation of the law either in the case of the immigration workers or in the case of the construction workers.
Moreover, when the researchers looked at the ideologies of the study participants, they found that for members of the general public, ideological beliefs were strongly related to how people judged the littering scenario. Thus, members of the public who were “hierarchical individualists” in outlook — read, “conservatives” — were prone to punish the aid workers but let off the construction workers. For “egalitarian communitarians” (liberals), it was the opposite.
But for judges, the role of ideology vanished. Judges who were egalitarian communitarians actually were more likely to let off the construction workers — a group that egalitarian communitarian citizens were highly inclined to punish.
Nonetheless, the study also found that judges polarized quite predictably, along ideological lines, in their views on the issues of climate change and marijuana legalization. So it isn’t that judges aren’t political — it’s just that in certain cases where they are applying learned expertise, that expertise supplants any gut political leanings.
“Judges of diverse cultural outlooks—ones polarized on their views of the risks of marijuana legalization, climate change, and other contested issues — converged on results in cases that strongly divided comparably diverse members of the public,” the study concluded. (It wasn’t so kind to law students, though: “Students enjoy an immature form of the professional judgment that fully trained and experienced lawyers possess,” the researchers observed.)
Kahan thinks the findings are generalizable beyond judges to other kinds of domain experts – at least when they are in their realm. “The results give one a reason for thinking that experts making ‘in domain’ judgments in other areas would likewise be resistant to identity-protective cognition,” he commented by e-mail.
In other words, that would include climate scientists, evolution scientists, political scientists and many others. “Our study, although confined to judges and lawyers, furnishes at least some evidence for discounting the likelihood of the hypothesis that climate scientists or other comparable experts are being influenced by identity-protective reasoning,” Kahan and his colleagues write.
Note: This research does not mean that experts have no reasoning flaws. It just means they seem to be able to put their own personal, political identities to the side when engaging in reasoning in realms where they’re highly trained. “Experts can be demonstrated to be vulnerable to foreseeable forms of cognitive biases — including confirmation bias & the like — under one or another set of circumstances even if in general they aren’t vulnerable to identity-protective cognition,” Kahan said by e-mail.
The new study fits nicely alongside a growing trend toward robustly defending and reaffirming the importance of experts. Several recent books have done so, including last year’s “Are We All Scientific Experts Now” by Cardiff University’s Harry Collins, a longtime science studies researcher who himself used to be numbered among the crowd of academics accused of undermining the concept of scientific expertise.
Moreover, in 2005 political psychologist Philip Tetlock rescued experts, at least partly, in his acclaimed book “Expert Political Judgment.” Tetlock classed supposed experts into two groups — “foxes,” who cobble together many different kinds of knowledge and tend to be flexible in their outlooks, and “hedgehogs,” who interpret everything based on a single grand theory — and subsequently found that foxes made much better predictions of future events.
“The foxes’ self-critical, point-counterpoint style of thinking prevented them from building up the sorts of excessive enthusiasm for their predictions that hedgehogs, especially well-informed ones, displayed for theirs,” Tetlock wrote.
The conclusion of all of this research, then, is that there really does seem to be something called “expertise.” Moreover, there are certain habits of mind learned by experts — especially those possessed of the right, nuanced personality disposition — that render them quite good guides to reasoning about topics where for non-experts, political passions get in the way.
So experts really do exist, and they really are different from non-experts. Now, all we have to do is listen to them.