In Hollywood, the moment the good guys trace a hair, a bullet fragment or a fingerprint, it’s game over. The bad guy is locked up.
But the glamorized portrait is not so simple in real life.
Far from infallible, expert comparisons of hair, handwriting, marks made by firearms on bullets, and patterns such as bite marks and shoe and tire prints are in some ways unscientific and subject to human bias, a National Academy of Sciences panel chartered by Congress found. Other techniques, such as in bullet-lead analysis and arson investigation, survived for decades despite poorly regulated practices and a lack of scientific method.
Even fingerprint identification is partly a subjective exercise that lacks research into the role of unconscious bias or even its error rate, the panel’s 328-page report said.
“The forensic science system, encompassing both research and practice, has serious problems that can only be addressed by a national commitment to overhaul the current structure,” the panel concluded in 2009.
Now, Congress and the Obama administration are trying to regulate forensic science to help establish standards. Senate Judiciary Committee Chairman Patrick J. Leahy (D-Vt.) and Commerce, Science and Transportation Committee Chairman John D. Rockefeller IV (D-W.Va.) are weighing legislation that could subject techniques to greater scientific scrutiny and help establish their ranges of accuracy.
A Leahy bill would create a new office of forensic science in the Justice Department. Rockefeller is preparing legislation to expand the role of the National Science Foundation and the National Institute of Standards and Technology in setting scientific standards and research goals.
The Obama administration is also looking to “strengthen the linkage between cutting-edge science . . . and the forensic tests used by law enforcement,” said Rick Weiss, spokesman for the White House Office of Science and Technology Policy.
Police and law enforcement agencies have rebuffed recommendations to remove crime labs from their control.
Since 2002, failures have been reported at about 30 federal, state and local crime labs serving the FBI, the Army and eight of the nation’s 20 largest cities.
Advances in DNA testing are exposing errors at unexpected rates. In November, researchers with the Urban Institute reported that new DNA testing appeared to clear convicted defendants in 16 percent of Virginia criminal convictions between 1973 and 1988 in which evidence was available for retesting.
A 2009 study of post-conviction DNA exonerations — now up to 289 nationwide — found invalid testimony in more than half the cases.
“There are just too many related problems for this to be dealt with ad hoc,” said Brandon L. Garrett, a professor at the University of Virginia School of Law.
More DNA testing alone is not the answer, experts say. Biological evidence historically is collected in fewer than 20 percent of criminal cases. Other questioned forensic techniques are used far more often, with mistakes harming defendants and crime victims whose true assailants remain at large.
The National Academy of Sciences report cited the lack of effective standards for examiners, laboratories and court testimony. It also criticized Justice Department agencies for a dearth of research into problems and for being “too wedded” to the status quo to be trusted to lead reforms.
“This is our generation’s sole opportunity” to get arguments out of the adversarial system and resolved through science, said Thomas L. Bohan, who was president of the American Academy of Forensic Sciences in 2010. “It’s a shame they couldn’t have done a good job 10 or 20 years ago.”
Arson investigation is an example of how research has dramatically improved practices.
Since 1990, the number of U.S. structure fires attributed to arson has dropped by half. One reason is that scientific test burnings have disproved the notion that some burn marks could be caused only by liquid accelerants.
Meanwhile, scientific doubts have festered for decades with fingerprint examination. While fingerprint analysis is one of the most valuable and frequently applied investigative tools, its accuracy has not been scientifically defined.
FBI examiners claimed until recently that they can match fingerprints to the exclusion of any other person in the world with 100 percent certainty using a method with an error rate essentially of zero. The academy report found that assertion was “not scientifically plausible” and had chilled research into error rates.
In 1999, a Justice Department official, Richard Rau, told a federal court that the department delayed such a study because of the legal ramifications. As recently as last year, Pennsylvania State University researcher Cedric Neumann was denied a department grant to determine potential fingerprint error rates using closed cases.
Neumann declined to comment for this article.
A person familiar with the episode blamed a polarized climate in the adversarial legal system, saying, “Few agencies in the forensic-science community want to be the first ones associated with an error rate.” The person spoke on the condition of anonymity to discuss sensitive federal research funding decisions.
Meanwhile, errors occur. In 2004, DNA for the first time exonerated a person convicted with a fingerprint match and, separately, the FBI made its first publicly acknowledged fingerprint misidentification. Brandon Mayfield, a Portland, Ore., lawyer, mistakenly was arrested in connection with the terrorist train bombings in Madrid that killed 191 people. The FBI apologized.
Since then, the Justice Department has begun research to try to quantify how complete a fingerprint must be to properly declare a match; how different conditions may affect the reliability of examinations; whether computers can do such work; and how to present forensic testimony about probabilities to judges and juries. The FBI has also required “blind verification” of results by agents unfamiliar with initial examinations.
The bureau said that skilled analysts are extraordinarily accurate, at least when they know they are being tested. An FBI study with Noblis Corp. last year found that when 169 examiners compared thousands of fingerprints and decided there was enough information to declare a match or not, they were correct 99.8 percent of the time.
Still, the Mayfield case highlighted the need for research into real-world conditions. A 2006 study by a London-based scientist, Itiel E. Dror, asked experts to analyze fingerprints that, unbeknownst to them, they had analyzed earlier in their careers. This time, however, examiners were given biasing statements, such as that a suspect had confessed or that a suspect was locked up at the time of the offense. In 16.6 percent of cases, examiners reversed earlier judgments.
Crime lab directors and prosecutors welcome calls for more money for research and to improve examiners and facilities. But with budgets tight at all levels, Washington has few other tools to prompt 350 state and local labs across the country to improve.
Because techniques have not been scientifically proved does not mean they do not work, defenders say, and mistakes can be handled traditionally through case-by-case appeals.
“In the real life of the criminal justice system, we need more resources for those who are on the front lines,” said Scott D. Burns, executive director of the National District Attorneys Association. Noting that prosecutors handle 20 million non-traffic cases a year, Burns said, “The sky isn’t falling, and we usually get it right.”
Pete M. Marone, director of the Virginia Department of Forensic Science and chairman of the Consortium of Forensic Science Organizations, urged Congress not to “reinvent the wheel” by abandoning all existing accreditation standards or groups such as the one he represents.
“Don’t judge forensic science today based on errors from 30 years ago,” Marone said. “What we need is someone setting a research agenda and direction. . . . We need leadership.”