Hurricane Katrina wrecked New Orleans and other parts of the Gulf Coast. It also whacked Washington, reminding us that there's a world of difference between having good management plans and having good leadership.

In the Katrina catastrophe, the political leadership at the Federal Emergency Management Agency fell embarrassingly short, cooperation among federal, state and local authorities crumbled, and lessons learned in past emergencies were ignored or lost when the levees failed New Orleans.

One of the great challenges for any administration is getting a handle on what works and what does not work -- at FEMA and other critical agencies. The Bush administration has invested substantial time and dollars in management scorecards and other efforts aimed at improving program performance, but trying to predict any program's chance of success is difficult at best.

In the year before Katrina hit, the Office of Management and Budget completed a review of FEMA's response and recovery programs, using data from the 2003 hurricane season.

The FEMA programs received the lowest possible passing grade -- "adequate."

The evaluation, by OMB's Program Assessment Rating Tool, found that FEMA needed improvement in several areas. PART was started by the Bush administration to help the White House prepare annual budget requests to Congress and to help OMB use a consistent approach when reviewing federal programs.

The PART review turned up flaws in how FEMA ensures that resources and assistance will reach disaster victims. It also found that FEMA may face legal barriers that would limit program effectiveness if it does not have a prompt disaster declaration authorizing recovery and relief activities.

According to the PART review, FEMA could show only to a "small extent" that it was making progress on long-term performance goals or that its recovery program was effective and achieving results.

As the Department of Homeland Security inspector general noted in the PART review, "FEMA's public image can be directly attributed to the success of FEMA's disaster response and recovery system."

Robert J. Shea, a senior OMB official who helps oversee PART, acknowledged that FEMA "did not have data that said they were, in all cases, achieving their goals."

But he said the PART review of FEMA also showed that the agency "had a good process for identifying weaknesses."

In the case of Katrina, Shea said, "we had a disaster the magnitude for which these programs before had not been tested . . . a soup of factors that prevented peak performance in this situation."

To be sure, FEMA was overwhelmed by Katrina. It has had to help 400,000 people find housing -- a record. As a comparison, four hurricanes in Florida last year left about 20,000 people in need of housing, said Nicol Andrews, a FEMA spokeswoman.

Shea said Katrina will lead to more after-action reviews and stepped-up efforts in the Department of Homeland Security to identify program weaknesses at FEMA. Retooling FEMA is one of the top three priorities of Homeland Security Secretary Michael Chertoff, Andrews said. (FEMA officials responsible for PART were not available for an interview.)

OMB will not be changing the questions it asks FEMA in the next round of PART, Shea said, but he suggested that FEMA's answers will be different next time because of Katrina.

"It is not that PART failed to cure this problem," Shea said. "The PART is a great tool to go about fixing them in the future."

But capable leadership may be more important to FEMA than program reviews and management scorecards.

The key to successful programs, Shea said, begins with White House officials and Cabinet heads committed to improvements and to holding agency heads accountable for progress.

"A process alone gets you nothing," he said. With post-Katrina FEMA, Shea said, "what you have now is information that can help you devise a better plan to achieve your goal, since you've fallen short."