Defense officials have a standard procedure for testing a weapons system’s reliability, roughly defined as the ability to continue operating effectively without any major changes or upgrades.
The standard used to require that weapons systems were built with 30 percent reliability, meaning there was, at most, a 70 percent chance that replacements or updated elements would be needed after the systems had entered into operational use. Contractors’ adherence to reliability standards is now voluntary, in part because reliability is seen as so difficult to predict.
According to the Defense Department’s Operational Test and Evaluation unit (DTO&E), only six of 11 Army programs that faced testing by the agency met their reliability thresholds. Among those that “did not do well” were unmanned systems and communications networks, even though the Army had stipulated in engineering and manufacturing development contracts that those programs should meet an early reliability test threshold.
The Navy, which established a high-level director of reliability and maintainability and several other working groups to address reliability issues, had 17 of 27 systems meet their thresholds. The most reliable systems were aircraft and submarines, but “ships and software-intensive systems” did not fare as well.
Among those that failed to meet reliability thresholds: the billion-dollar LPD-17, the new amphibious transport docks that carry helicopters, landing craft and Marines for expeditionary missions.
The Air Force had the worst record for reliability, with only three of 11 systems tested by DOT&E meeting the reliability threshold. The Air Force has produced a guidebook to identify risks and had courses on reliability built into levels of its acquisition and test personnel.
The director of DOT&E, J. Michael Gilmore, said that, of 52 reports on systems’ effectiveness, suitability and reliability sent to Congress over the past five years, 36 were found to be suitable, while only 26 met their reliability threshold. None of the 52 was ever canceled, he said.