washingtonpost.com
Radiation Detectors' Value Is Questioned
GAO Says Agency Overstated Efficacy

By Robert O'Harrow Jr.
Washington Post Staff Writer
Thursday, October 30, 2008

Officials at the Department of Homeland Security have overstated the performance of costly new radiation detectors designed to prevent the importation of radiological materials that could be used in bombs, according to an unreleased government report.

The department's Domestic Nuclear Detection Office has claimed in a recent report that new tests show the detection machines, known as Advanced Spectroscopic Portal monitors, can more accurately detect and identify radioactive materials than existing equipment in use across the country, the Government Accountability Office said in its report.

The detection office's assessment is part of an effort by the troubled Bush administration program to win congressional approval to deploy the machines.

But auditors who have examined the test results said the office's claims cannot be backed up by statistical evidence. That's because the data collected from what is called the Phase 3 test was too limited, according to the report by the GAO, the investigative arm of Congress.

"Because the limitations of the Phase 3 test results are not appropriately stated in the Phase 3 test report, the report does not accurately depict the results from the tests and could be misleading," the GAO auditors wrote.

The GAO findings trouble lawmakers who have expressed concern that the department is trying to push technology without knowing whether it is worth the expense. The GAO has said the machines could cost $778,000 each to buy and install, far more than detection office estimates two years ago.

"I'm concerned that the testing for the new detectors remains flawed," said Rep. John D. Dingell (D-Mich.), chairman of the Committee on Energy and Commerce, one of the panels that has been examining the program. "We still don't know if this extremely expensive technology works any better than the current equipment. Until there is objective and concrete evidence that the new machines have clear benefits over the existing detectors, I cannot support additional procurements."

In a letter responding to the GAO conclusions, the detection office derided the findings as "misleading and not substantiated." The detection office said the GAO has failed "to acknowledge the depth and breadth" of the program's test campaign.

In a statement, a homeland security spokeswoman said the detection office "is currently undertaking a comprehensive test and evaluation program on ASP systems and will use previous test data as well. The department has been following a prudent path leading to certification of ASP systems."

The dispute is the latest in a series over the last two years that have slowed development of the radiation detector program, one of the Bush administration's top national security initiatives.

The GAO has repeatedly asserted that the detection office has misled Congress about the costs of the machines and their effectiveness, questions that homeland security officials say are misguided.

The project was launched in July 2006 with the award of $1.2 billion in contracts to three companies for the development and deployment of the machines. Those contract awards came after the detection office delivered a report to Congress about the cost and effectiveness of the machines.

Several months later, the GAO found that the detection office's estimates for detection rates were overstated and that the costs of the machines were significantly understated.

As a consequence, lawmakers mandated that Homeland Security Secretary Michael Chertoff personally certify that the machines represent a significant operational improvement over existing equipment before they can be fully deployed.

Last year, the detection office conducted two rounds of tests of the machines, in part to support Chertoff's certification decision. The GAO concluded that the detection office's first round of tests were conducted in a biased way that "enhanced the apparent performance of the ASPs."

After Customs and Border Protection officials questioned the operational effectiveness of the machines in field tests, Chertoff announced a delay in his certification decision.

The GAO's new report focuses on the second round of detection office tests, called Phase 3. Detection office officials told government auditors that Phase 3 tests showed the machines "were as good as or better than" existing equipment "at detecting the presence of radiological source materials at low levels of radiological activity," the GAO report said.

The GAO also answered complaints from the detection office that auditors ignored the extent of the program's testing.

"In our view, regardless of how many tests are performed, the tests must employ sound, unbiased methodologies and [the nuclear detection office] should draw and present conclusions from the test results in ways that accurately and fully reflect the data and disclose their limitations, " the GAO report said.

View all comments that have been posted about this article.

© 2008 The Washington Post Company