When the Data Don't Really Measure Up

Wednesday, April 9, 2008

As part of his presentation to Congress yesterday, Army Gen. David H. Petraeus displayed 11 multicolored charts to demonstrate progress in Iraq. A close look at the facts indicates that the data often lacked context or were misleading. Here is a guide:

WHAT IT SHOWS: This chart, showing steadily rising Iraqi security expenditures and lower U.S. expenditures, suggests that the Iraqis are beginning to pick up much of the tab for their own security.

ANALYSIS: The lines on this chart through 2008 closely track the Iraqi budget and previous testimony by U.S. officials. The figures for 2009 appear to be based on guesswork, and Petraeus's office declined to provide supporting information. But all the data are so oddly defined that the comparison is not meaningful.

The Iraqi expenditures reflect the budget for running the Interior and Defense ministries, meaning that at least half of the amount is salaries for soldiers and police. The numbers have increased because the personnel in those ministries has skyrocketed in recent years.

The U.S. expenditures are just for the Iraq Security Forces Fund, which provides for the training and equipping of Iraqi forces; the line on the chart does not include the billions of dollars spent on the salaries of U.S. troops assisting the Iraqis and the cost of the extensive logistics that the U.S. military provides to Iraqi forces.

When the Government Accountability Office last year asked for the total-cost figure, the Defense Department said it could not provide an estimate.

Army Lt. Gen. James Dubik told lawmakers in January that Iraq might be able to provide for its own internal security by 2012 but could not defend itself for another 10 years.

"The truth is that right now they simply cannot fix, supply, arm or fuel themselves completely enough at this point," he said. The Defense Department inspector general recently faulted the handling of the management of the Iraq Security Forces Fund, saying there was no assurance that intended results were achieved or that resources were protected from waste and mismanagement.


WHAT IT SHOWS: The number of Iraqi combat battalions, particularly those "capable of taking the lead," has steadily grown.

ANALYSIS: This chart uses imprecise language to suggest an improvement in the capability of Iraqi forces when there may well be little or no improvement.

Only a small sliver of the battalions, perhaps 10 or 12, are rated at the top category of operational readiness (green in the chart). But the diagram reaches part of the way into the group of third-level readiness battalions (orange in the chart) to achieve its total of "112 battalions in the lead."

By definition, none of those battalions should be capable -- and in his testimony, Petraeus acknowledged that even the best of these troops still need logistical assistance from the United States.

A draft Government Accountability Office report in September said that the number of Iraq army units capable of operating independently declined from 10 in March 2007 to six in August. The numbers were removed from the final report and are now classified.

In a November report, the GAO faulted the Pentagon for making "confusing and misleading" claims that any particular Iraqi unit is "independent." The GAO said there is not sufficient evidence that some units are more capable than others, given that the ability of the Interior and Defense ministries to "maintain and sustain their forces, provide effective command and control of their forces, and provide their forces with intelligence is undermined and cannot be accomplished without Coalition support."


WHAT IT SHOWS: This chart combines maps of Baghdad's ethnic neighborhoods with density plots of ethno-sectarian killings to show that violence has declined significantly from December 2006 to last month.

ANALYSIS: Hidden beneath many of the density plots are colors that show a major reshaping of Baghdad, from an ethnically mixed city to a patchwork of rival ethnic and religious enclaves whose residents rarely intersect outside their gated communities.

Many analysts, including in the U.S. government, believe that this de facto division of Baghdad -- as opposed to brilliant U.S. counterinsurgency work -- is largely responsible for the decline in violence.

"The polar ization of communities is most evident in Baghdad, where the Shia are a clear majority in more than half of all neighborhoods and Sunni areas have become surrounded by predominately Shia districts," said the National Intelligence Estimate on Iraq last year. "Where population displacements have led to significant sectarian separation, conflict levels have diminished to some extent."

Many experts say that a more reliable indicator of the country's safety is the flow of refugees and internally displaced people, because if people feel secure, they return home. On that score, the situation in Iraq remains deeply troubled.

The U.N. High Commissioner of Refugees estimates that beyond the more than 2 million refugees who have fled to other countries, there are another 2.8 million Iraqis who are internally displaced, out of a population of 26 million. More than 1 million have no regular income; 300,000 have no access to clean water.

U.N. and U.S. officials have urged refugees and other displaced people not to try to return home precisely because Baghdad has become so balkanized. Officials fear that tensions and violence will erupt again if people try to reclaim their homes in once mixed neighborhoods.

-- Glenn Kessler

View all comments that have been posted about this article.

© 2008 The Washington Post Company