As noted in Friday’s links roundup, the Justice Department announced this month that it will only use evidence from accredited crime labs around the country. That’s at least an incentive for labs to begin adopting best practices and minimum standards. But as with many of the criminal justice reforms that have emanated from the Obama administration, this one may be more symbolism than substance. Symbolism isn’t unimportant. But it doesn’t do much to correct injustices. From Frontline:

The new policy, announced by Deputy Attorney General Sally Q. Yates at a meeting of the National Commission on Forensic Sciences, is unlikely to precipitate any immediate or universal overhaul of the nation’s crime labs. It will only directly affect labs contracted by the DOJ that aren’t already accredited — and it will only begin doing so in 2020. Furthermore, the new policy seemingly has a built-in loophole: Prosecutors are required to use accredited labs “when practicable,” a phrase Yates told the commission was not meant to be used loosely, but only when using those labs would cause great delay or excessive cost . . .
But other commission members said that the policy doesn’t reach far enough. To prevent lab scandals from happening, accreditation has to do more than simply become universal — it must also become rigorous, said commission member Paul Giannelli, a law professor and evidence and procedure expert at Case Western Reserve University.
Crime labs are accredited by private entities, often a nonprofit associated with a professional forensics association. Each group varies in its standards, but most require labs to maintain tested and calibrated equipment; put in place standard operating procedures, including for the reporting of errors; and establish quality controls. But many of these agencies have been criticized for lacking rigorous standards and showing little muscle when problems occur. For instance, Giannelli said, some labs are able to choose for themselves the cases their reviewers see, rather than having those cases selected at random . .
Nor does accreditation solve many other common problems that crop up in labs, from error-prone or undertrained technicians to unscientific conclusions. The sensitivity and accuracy of DNA science, which began being used in courts in the late 1980s, has thrown into doubt the accuracy of many other forensic disciplines that previously commanded confidence in courts, from arson investigation to hair analysis to bite mark analysis. But in recent years, even DNA forensics have come into question. Such problems cannot be addressed by accreditation alone.

These criticisms get stronger when you consider that most of the crime lab scandals of the past several years (and Frontline has a good map of them at the linked article) have occurred at crime labs that were already accredited.

The two main pitfalls of forensic evidence are (1) the fact that we ask judges, not trained scientists, to determine what is and isn’t credible scientific evidence, and (2) cognitive bias. The former allows bad science into the courtroom, and allows practitioners of even the more credible forensics fields to overstate their findings. The latter is a problem that creeps into all forms of forensic analysis.

The problem with making judges the gatekeepers of scientific evidence is so ingrained in U.S. case law that it’s almost impossible to envision it changing. The best we can hope for is that judges start turning to the scientific community for guidance when determining what is and isn’t credible evidence, and that the scientific community take a more active role in assisting judges in such matters. So far, we’ve seen mixed results at best. With bite mark evidence, for example, the scientific community has pretty resoundingly stated that it’s a specialty supported by no scientific research whatsoever. Yet to date, every court in the country that has heard a challenge to the scientific validity of bite mark evidence has ruled against those challenges.

The other problem — cognitive bias — is the product of having crime labs report to police agencies, prosecutors, or other law-enforcement institutions. Cognitive bias can be minimized with some creative reforms like rivalrous redundancy, but so far, no jurisdiction in the country has attempted such reforms.

Speaking of bite mark evidence, the New York Times recently published a long report on the Texas Forensic Science Commission’s promising investigation of bite mark evidence. As I’ve reported here, the commission seems to be ruffling feathers in the community of bite mark analysts, which is a pretty good sign that they’re on the right track.

But here again, the commission’s findings aren’t binding on the state’s courts. And even if the commission takes a strong stand against bite mark evidence, and even if Texas courts run with those findings, there’s no obligation for other courts around the country to do the same.

I don’t mean to be overly cynical, but there was similar excitement about the National Academy of Sciences Report on forensics that came out in 2009. That report generated a lot of talk about reform, and resulted in the national commission referenced above. But the report also had very little impact on the courts. With just a few exceptions, judges have continued to allow bad science into the courtroom, and appellate judges have continued to reject challenges even to the forensics fields specifically cited by the report as lacking any scientific value.

I’ve made this point before, but the revolutionary technology of DNA testing showed us that many of the forensic “sciences” we thought to be foolproof were seriously — in some cases fatally — flawed. It was a watershed moment in criminal justice. But it was also a one-time opportunity. For decades, bad science was allowed into the courtroom because the experts espousing sounded convincing, and there was no definitive way to prove them wrong. DNA showed they were wrong.

But it’s unlikely that anything as certain, overarching, and game-changing as DNA testing will come along again. Of course, in cases where DNA testing is dispositive of guilt, there will be no need to rely on the less credible forensic fields. But that’s only a small percentage of cases. If we don’t seize on this opportunity and get reform right, not only will bad science continue to plague the criminal justice system, it will do so having survived the “reform era,” giving it additional credibility and making it more difficult than ever to challenge.