When injustices occur in criminal cases, our institutions — prosecutors, police departments, crime laboratories — often treat it as a “bad apple” problem. Institutions blame the individual and fail to look within the institution for the root cause.
We are starting to recognize that the actions of “bad apples” are often rooted in systems that fail to account for cognitive bias, shoddy casework, institutional culture, and perverse incentives for employees and managers. Though popular television shows associate crime labs with science and accuracy, all too often, quality control systems in real-world crime labs are too weak to identify mistakes and fraud, and management is unable or unwilling to address institutional shortcomings. When this happens, innocent people go to prison.
The recent experience of the D.C. Department of Forensic Sciences (DFS) is a case in point, and it has the makings of a mass disaster. The DFS lost its accreditation because it engaged in “misrepresentations and fraudulent behavior” in connection with inconsistent results in a series of firearm examinations. Yet, despite mounting evidence of pervasive mismanagement and structural problems at the lab, the mayor and the D.C. Council have not yet taken serious action to hold all of the DFS leadership accountable and make critical structural changes to reform the lab. This is a mistake. Ignoring deep-seated management, quality-control and scientific defects will cost lives and cost taxpayers.
In Massachusetts, tens of thousands of cases are being reopened because of years of unremedied misconduct at two drug labs in the state. In D.C., there have been several DNA exonerations of innocent men who spent decades in prison because of false and exaggerated FBI testimony concerning hair comparisons. The unreliable lab work in those cases was the tip of the iceberg. When the FBI conducted a landmark audit of all its hair cases, it found 95 percent raised the same scientific defects.
The problems at DFS are chillingly similar. They began with a firearms comparison, a discipline that raises many of the same scientific issues as hair comparison. As currently practiced, firearms examiners look at fired ammunition and opine on whether it was fired from a particular gun. An opinion that items were not fired from a particular gun may be rooted in objectivity, such as the bullets are different calibers or cartridge cases bear the markings of differently shaped firing pins. However, any identification opinion — which law enforcement may use to link fired ammunition to a suspect gun, or to associate multiple shootings to a common suspect — necessarily involves a subjective process of “matching” markings.
Ammunition fired by different guns may bear indistinguishable marks if the guns were manufactured by the same tool or by sheer coincidence. Thus, practitioners look for “sufficient” matching marks. There are no standards for what degree of matching is “sufficient”; it is up to the examiner eyeballing the patterns and making a call. Likewise, there are no standards for when observed differences can be considered insignificant vs. when these differences should lead to the conclusion that two items were not fired from the same gun. This unconstrained subjectivity renders the discipline particularly susceptible to error and bias. The firearms field has failed to implement even basic bias mitigation measures that have been adopted by another pattern matching discipline, fingerprinting, in the wake of an infamous FBI misidentification in which circular reasoning played a role.
Judges across the country have been sounding alarms about current practice of firearms comparison. In United States v. Tibbs, the court found that research on the accuracy of firearms comparisons suffered from serious design flaws such that the court concluded the government could not establish a “known or potential rate of error” for the discipline. The judge ruled claims of source identification lack scientific support and limited the testimony the examiner could provide accordingly.
But the problems at the DFS are much broader than just the scientific defects in one discipline.
Public records indicate that troubling information first surfaced in 2017 about the work of the DFS. Yet it was not until late 2019, when the U.S. attorney’s office’s Fraud and Public Corruption (FPC) section received a lead that a firearms examiner had falsified an examination and management had papered over the misconduct, that the U.S. attorney’s office acted. The FPC investigation did not produce hard evidence of a falsification but did reveal a slew of troubling problems across examiners. And it was clear that these problems were not the result of a few “bad apples” but instead were systemic and came from the top down.
According to the FPC report, investigators learned that the lab’s legal department was pressured by leadership to shape employee discipline to protect the lab’s reputation and sometimes to protect favored employees’ careers. They learned that mandatory quality assurance procedures allegedly were often ignored, and that dry-labbing (rendering conclusions about evidence without examining the evidence) has occurred. The FPC concluded that DFS was plagued by “poor judgment and incompetence” and recommended “a broader review of DFS’s institutional grasp of the importance of integrity issues to its work.”
More recently, a new case, before the same judge who decided the Tibbs case, revealed still more explosive problems. The U.S. attorney’s office asked two private examiners to review the work the DFS had done to connect firearms evidence between two criminal cases. These private examiners concluded that the DFS had gotten the comparison wrong. The lab responded by claiming that, in retrospect, the evidence was “inconclusive” because by refusing to commit to an identification or elimination, the lab could now claim its analysis wasn’t “wrong.”
What followed was shocking, including alleged coercion of examiners to change their answers to inconclusive, and allegedly hiding conflicting answers from the courts, the public and their accreditors. This alleged conduct has led to yet another criminal investigation of the lab — an ongoing criminal investigation.
This still-evolving saga shows just how malleable and problematic firearms evidence can be. People can and do disagree about it. The scandal highlights the dangers of “inconclusive” conclusions being used to bury errors, a prescient alarm raised by the judge in Tibbs.
The problems exposed to date demand action. The DFS needs oversight and new leadership committed to scientific integrity. The recent resignation of the director is a start but accomplishes nothing if others are not held to account and structural changes are not made. And hiring a consulting firm to help the lab reclaim its accreditation also does nothing, given that the lab was accredited when these issues arose. Instead, legislation is needed to require serious quality control, including blind proficiency testing to detect errors and enhanced oversight by independent scientists of the DFS. And a commitment must be secured from the U.S. attorney’s office for a transparent review of all closed firearms cases to ferret out the extent of the problems. If these actions are not taken, cost will undoubtedly rise.
Innocent men lost decades of their lives behind bars, and the city paid more than $43 million in settlements as a result of faulty hair analysis. Only time will tell the extent of the damage from this DFS scandal, but any delay in vigorously responding will surely increase those damages.