Six weeks ago, I wrote about a case in Pennsylvania in which a prosecutor was attempting to use bite mark evidence in the murder trial of Paul Aaron Ross. This, despite the fact that every scientific panel to review bite mark analysis to date has found no scientific basis for its underlying premises: a) that human dentition is unique, and b) even if (a) were true, that human skin is capable of recording and preserving bite marks in a way that preserves that uniqueness in a usable way. So far, the discipline has been found to be scientifically unreliable by the National Academy of Sciences, the Texas Forensic Science Commission, and the President’s Council of Advisors on Science and Technology. The latter two panels have called for barring bite mark evidence from criminal trials. Experiments by University of Buffalo scientists Mary and Peter Bush have also found no scientific basis for bite mark analysis.

Unfortunately, none of this seems to matter to the courts. Also of apparently little interest to the courts are the more than two dozen people wrongly arrested or convicted due to bite mark testimony. To date, every single court in the country to hear a challenge to bite mark evidence has shot that challenge down. Bite mark analysis is winless in scientific reviews, but it is undefeated in court.

This week, Blair County Judge Jolene G. Kopriva not only rejected Ross’s challenge to the bite mark evidence the state wants to use against him, she refused to even hold an evidentiary hearing to let Ross make his argument. From the Altoona Mirror:

In testimony during Ross’ 2005 trial, an Allentown orthodontist told the jury that Ross’ dental impression was “very highly consistent” with the mark found on Miller’s body.

Defense attorney Thomas Dickey said Wednesday that he was disappointed with Kopriva’s ruling.

“It’s basically saying that because we’ve always let this evidence in, then we’re going to still let it in,” Dickey said.

That is exactly what it means. As I explained in my series on bite mark evidence a couple of years ago, this is the primary reason courts have been allowing bite mark evidence for decades — because previous courts have allowed it. A California appeals court was the first to allow it in 1975. That opinion explicitly noted that bite mark evidence was not scientific, but, oddly, just a matter of “common sense.” Other courts then began citing that case, sometimes by mistakenly noting that the court did find the evidence to be scientifically credible. Since then, most courts uphold bite mark evidence by simply citing other courts, despite the fact that none of them attempted an actual scientific analysis of the practice. At this point, no judge wants to be the first to say all of those other judges are wrong. Never mind the overwhelming evidence that they are. And that includes Judge Kopriva.

“Although the use of bite mark evidence is beginning to face challenges, it would be premature for this court to order that the methodology is no longer generally accepted in the relevant scientific community,” Kopriva wrote.

That magic phrase — “generally accepted in the relevant scientific community” — comes from Frye v. United States, a decision by the U.S. Court of Appeals for the D.C. Circuit in 1923. Pennsylvania is one of just a handful of states that still use that ruling as the controlling authority to determine the admissibility of scientific evidence. (Most states use the Daubert test, which was set for federal cases by the U.S. Supreme Court in a trio of rulings in the early 1990s.) Kopriva has obviously defined the “relevant scientific community” here as other bite mark analysts. As I’ve pointed out here several times now, this is akin to a judge deciding on the scientific reliability of astrology by limiting the “relevant scientific community” to other astrologists.

Bite mark analysis isn’t hard science. Even its defenders concede as much. But it is presented to juries as science. Therefore, the “relevant scientific community” ought to be the community of scientists who have applied the scrutiny of the scientific method to the field. And here, there’s no debate. Not only is bite mark analysis not science, to date every scientific panel to review the discipline has found that the core claims made by its practitioners are either unsupported by science research or directly contradicted by it. For example, bite mark analysts cannot calculate a margin for error for their conclusions the way, say, a DNA analyst can.

Back to the article:

The judge’s order also directs that evidence regarding the bite mark and a suspect’s dental impression must follow guidelines set by the American Board of Forensic Odontologists. Based on that, testimony will be restricted as to whether the suspect’s dental impression can or cannot be excluded from ones that could have made the mark.

This is mostly meaningless. Studies by law professor Brandon Garrett have shown that juries pay little heed to such linguistic distinctions. When they hear not excluded, they think “guilty.” Moreover, prosecutors routinely seize on these sorts of phrases to infer more certain guilt. For example, a prosecutor might tell the jury that only a limited group of people could possibly have committed the crime — a so-called closed population of suspects. If the bite mark analyst can “exclude” everyone within that group but the defendant, the state will argue that this is persuasive evidence of the defendant’s guilt. But there have been numerous cases in which police and prosecutors have defined closed populations of suspects that didn’t include the actual killer. A “closed population” can mean whatever prosecutors want it to mean.

Kopriva also stated in her ruling that the defense, during cross examination, can raise questions about bite mark evidence that will put the jury in a position to accept or reject what’s offered.

That just isn’t sufficient. Our adversarial system of justice may arguably be adequate when we’re talking about weighing objective evidence, but lay jurors simply aren’t qualified to distinguish good science from bad. The mere fact that most bite mark cases involve defense and prosecution experts giving diametrically opposing testimony is in itself another strong argument that bite mark analysis simply isn’t grounded in science. Again, you’ll rarely see two DNA analysts give conflicting testimony over whether a piece of biological evidence fits the DNA profile of a suspect. (They may disagree in more complicated cases involving DNA from more than one source — which is why the PCAST report warned that judges should be very careful about admitting that sort of testimony, too.)

Instead, bite mark analysis is entirely subjective. The jury believes the witness they find most credible and most convincing. And that typically is less a matter of scientific rigor than it is of charisma, persuasiveness, even charm. In fact, in pattern-matching disciplines like bite mark analysis, it’s likely that the more science-based the testimony, the less likely the jury is to find it convincing. Juries like certainty. They like witnesses that give them clear, unambiguous answers. Real scientists will speak in terms of likelihoods, and they’ll give margins for error. To the extent that a real scientist would testify in a bite mark case, he or she would simply state that there’s no way to know if the defendant bit the victim. The state’s expert implicates the defendant, while the defense experts says there’s no way of knowing either way. It’s easy to see how juries would find the former more persuasive.

Most defense attorneys opt instead to put on their own bite mark analyst, who then contradicts the state’s witness. The trial then turns into a farce in which two “experts” bombard the jury with equally bad science. These trials aren’t about getting at the truth. They’re about which side can present the most convincing nonsense.

Kopriva now joins a long list of judges who have gotten this wrong. She’s also the latest example of why we need to stop entrusting judges to be the gatekeepers of science in the courtroom. There’s a reason why they’re judges, and not scientists. Judges are trained in law. They work in chambers, not laboratories. They rely on judicial opinions, not peer-reviewed studies. When it comes to distinguishing good science from bad, they simply aren’t up for the job.

As I’ve argued before, DNA testing was a wake-up call to the problems in the criminal-justice system, not a panacea. It is only dispositive of guilt in a small percentage of cases. But the problems that led to the wrongful convictions in those cases undoubtedly pervade the entire system. Going forward, DNA testing at the onset of an investigation will make the verdicts in that small percentage of cases more reliable. But if we don’t fix the problems we’ve discovered, they’ll persist in the remaining cases, with no DNA testing to correct them.

All of which is to say have a limited window opportunity to get this right. So far, we’re failing.