|Page 5 of 5 <|
FBI's Forensic Test Full of Holes
The study, however, raised two concerns.
First, it found that bullets packaged 15 months apart -- a span that assumed separate batches of lead -- had the exact composition, potentially undercutting the theory that each batch was unique.
Second, it found that bullets in a single box often had several different lead compositions. That finding, it cautioned, should have "significant impact on interpretation of results in forensic cases."
Peele, the retired bullet-lead examiner, was the primary author of that study. He said he still felt comfortable having told jurors in the past that bullets from the same box could be expected to match, as long as his remarks were carefully qualified.
In the Hunt case, he testified that his match of the crime-scene bullets to those in the suspects' box was "typical of everything we examined coming from the same box or the next closest possibility would be the same type, same manufacturer, packaged on or about the same day."
Peele said that he always tried to tell jurors that some bullets in the same box might not match. Still, he said it was reasonable for jurors to conclude that matching bullets could have come from the same box. "I don't think it's misleading as long as it's fully explained," he said.
Some of Peele's colleagues went further. FBI examiner John Riley told a Florida jury: "It is my opinion that all of those bullets came from the same box of ammunition." A New Jersey prosecutor suggested that the bullets matched by the FBI were as unique as a "snowflake or fingerprint."
Today, the FBI regards all such testimony as inaccurate. "The science does not and has never supported the testimony that one bullet can be identified as coming from a particular box of bullets," said Adams, the retired FBI lab director.
A Challenge From Within
The FBI's about-face was prompted by a challenge from within its ranks.
William Tobin, an FBI lab metallurgist for a quarter-century, won accolades working on cases such as the crash of TWA Flight 800, in which he helped prove that the plane was downed by an accidental fuel-tank explosion, not terrorism. Shortly before he retired, Tobin was approached by a woman who believed that the bullet-lead science used against her brother, a New Jersey murder defendant, was flawed. Still employed by the bureau, Tobin was not permitted to help.
But when he retired in 1998, he decided to look further. Bullet matching had always been done by the lab's chemists, and as a metallurgist, Tobin wondered about their assumptions. Soon he joined with Erik Randich, a metallurgist at Lawrence Livermore National Laboratory.
By 2001, the two had finished a study that challenged the key assumptions that the FBI had been making about bullet lead. They found that bullets made from the same batch did not always match, because subtle chemical changes occurred throughout the manufacturing process. Tobin bought bullets at several stores in Alaska and found that a large number of bullets with the same composition and manufacturing date were often sold in the same community, suggesting that it was wrong to assume that a bullet match could be narrowed to one suspect.
"It hadn't been based at all on science but, rather, had been based on subjective belief," Tobin said in an interview. "Courts, and even practitioners, had been seduced by the sophistication of the analytical instrumentation for over three decades."
Soon, Tobin began appearing as a witness for defendants challenging FBI bullet-lead matches. Courts began to take notice, too, and the FBI suddenly faced a barrage of questions about a science that had gone unchallenged for three decades.
Adams asked the National Academy of Sciences in 2002 to examine the FBI's work, temporarily halting new bullet-lead matches. Two years later, the academy's findings stunned the bureau.
The panel concluded that although the FBI had been taking accurate bullet-lead measurements in its lab, the statistical methods and its expert testimonies were flawed.
The science "does not . . . have the unique specificity of techniques such as DNA," and "available data does not support any statement that a crime bullet came from a particular box of ammunition," the panel concluded. All the FBI could say going forward was that bullets made from the same batch "are more likely" to match in chemical makeup than those made from different batches. Adams soon declared that such testimony was so general that it had no value to jurors, and he ended the technique.
The FBI Response
The FBI went on the offensive to portray its decision in the best light.
In a news release dated Sept. 1, 2005, the bureau declared that it "still firmly supports the scientific foundation of bullet lead analysis" but that it was ending the technique because of the questions about its "relative probative value," the "costs of maintaining the equipment" and the "resources necessary to do the examinations."
The bureau also sent form letters to the more than 300 police agencies it had assisted with the science and to the umbrella groups representing local prosecutors and local criminal defense lawyers so they could "take whatever steps they deem appropriate."
The letters cited the academy's report but did not call attention to the magnitude of the FBI's internal concerns.
For instance, the letters stated that the impact of the academy's findings "on previously issued examination reports remains unaddressed." In fact, the FBI had conducted its own review to determine how often bad statistics led to mistaken matches.
In March 2005, the chief of the FBI chemistry unit that oversaw the analysis wrote in an e-mail that he applied one of the new statistical methods recommended by the National Academy of Sciences to 436 cases dating to 1996 and found that at least seven would "have a different result today." Marc A. LeBeau estimated that at least 1.4 percent of prior matches would change.
If the FBI employed other statistical methods the number of non-matches would be "a lot more," LeBeau wrote. In fact, when the bureau tested one method recommended by the academy on a sample of 100 bullets, the results changed in the "large majority of the cases," he wrote.
Despite the concerns, the FBI provided affidavits in at least two cases seeking to help prosecutors sustain convictions that were based on bullet-lead matches.
In one such affidavit introduced in Maryland, the FBI cited the academy's report but did not mention it faulted the bureau's statistical methods.
That omission concerns the chairman of the academy panel.
The affidavit "does not discuss the statistical bullet-matching technique, which is key and probably the most significant scientific flaw found by the committee," said Kenneth MacFadden, a private chemistry expert.
MacFadden and Spiegelman said they also believed the affidavit was misleading, because it estimates that the maximum number of .22-caliber bullets in a batch of lead was 1.3 million. The academy said the number could be as high as 35 million.
In a May 12, 2005, e-mail, the deputy lab director told LeBeau, "I don't believe that we can testify about how many bullets may have come from the same melt and our estimate may be totally misleading."
FBI officials said Friday they will stop using the affidavit.
"They said the FBI agents who went after Al Capone were the untouchables, and I say the FBI experts who gave this bullet-lead testimony were the unbelievables," Spiegelman said.
"60 Minutes" correspondent Steve Kroft and producers Ira Rosen and Sumi Aggarwal, Washington Post research editor Alice Crites and staff researcher Madonna Lebling, and freelance researcher Jilly Badanes contributed to this report.