Poor training and communication inside the Mars team, and its failure to follow procedures, allowed a tiny metric conversion mistake to destroy the $125 million Mars Climate Orbiter, NASA investigators said yesterday.
The devastating report comes just three weeks before a sister ship, the Mars Polar Lander, is scheduled to land on the frozen wastes near the Red Planet's south pole. The team responsible for both missions, based at the Jet Propulsion Laboratory in Pasadena, Calif., is conducting its own urgent internal reviews while undergoing scrutiny by outside experts--all to ensure that similar problems have not doomed this mission as well.
"We clearly made a serious error," laboratory Director Edward Stone said at a tense NASA headquarters briefing. For now, "the first business of the team is to see that we land safely on Mars. . . . None of us want another mistake to go unchecked."
The immediate cause of the spacecraft's Sept. 23 disappearance as it entered Mars orbit was a failure by a young and inexperienced engineer at contractor Lockheed-Martin Astronautics to make a simple conversion from English units to metric in the coding of ground software used by the spacecraft navigators.
"To be very blunt about it, it was overlooked," said Lockheed-Martin official Noel W. Hinners. The NASA contract specified that the conversion should be made.
Far more serious was the failure by an established system of checks and balances to catch the error, officials said.
Investigation leader Arthur G. Stephenson, director of NASA's Marshall Space Flight Center in Huntsville, Ala., called the conversion error "a little thing" that should have been corrected.
The investigation "has identified other significant factors that allowed this error to be born, and then let it linger and propagate to the point where it resulted in a major error in our understanding of the spacecraft's path as it approached Mars," according to the 45-page report.
The investigators listed eight contributing causes of the failure and made 16 recommendations for immediate changes before the upcoming landing.
The failure is a blemish on the laboratory's record as world leader in the difficult business of interplanetary exploration. But NASA officials rejected suggestions that the problems stemmed from the agency's "smaller, faster, cheaper," philosophy, which has replaced complex billion-dollar projects with lower-budget missions flying much more frequently.
That new approach "includes following the rules and processes," said Edward Weiler, NASA's top space science official. "The rules and processes were not followed this time."
The navigators, handling several missions at once, were overworked, isolated from other parts of the project, "unfamiliar with the spacecraft," inadequately staffed and trained. They never raised alarms about the problem, investigators said. "When conflicts in the data were uncovered, the team relied on e-mail . . . instead of formal problem resolution processes." The investigators also criticized the lack of complete end-to-end verification of navigation software and related computer models.
Over the months of its journey, the spacecraft gradually veered off on the wrong course. "All the [course] corrections went in the direction of driving the spacecraft down toward the planet," Stephenson said. By the time the mistake was uncovered, just before a critical final rocket firing to insert the craft into orbit, there was no time to recover. The spacecraft must have broken apart or burned when it plowed into the Martian atmosphere, engineers say.
Among the investigators' recommendations, which Stone said are all being acted upon, were that the laboratory clarify responsibilities. In visits to the lab since the orbiter loss, the report said, a recurring theme was "who's in charge?"
The investigators, noting that the lab's deep space navigation techniques have worked well for 30 years, found at the laboratory "a widespread perception that 'orbiting Mars is routine.' This perception resulted in inadequate attention to navigation risk mitigation."
Said Weiler, "We need more people asking what can go wrong."