By Deborah Howell
Sunday, December 25, 2005
Numbers aren't just facts. They can be interpreted in many ways, even if they come from the same or similar sources.
Ann Scott Tyson, a respected military reporter just back from Iraq, wrote in a front-page story Nov. 4 that "newly released Pentagon demographic data show that the military is leaning heavily for recruits on economically depressed rural areas where youths' need for jobs may outweigh the risks of going to war."
The story said that more than 44 percent of military recruits come from rural areas, most from the South and West. "Many . . . are financially strapped, with nearly half coming from lower-middle-class to poor households, according to new Pentagon data based on Zip codes and census estimates of mean household income."
The story, which was largely based on Pentagon data, included some analysis done by the National Priorities Project (NPP), a liberal-leaning think tank that questions the war in Iraq. The NPP also used Pentagon, census and Zip code data. A different analysis, released by the conservative Heritage Foundation a few days later, was reported by other media outlets.
In looking at the story, I talked to Curt Gilroy, who, as director of accession policy for the secretary of defense, has oversight of all active-duty recruiting; Tim Kane, a Heritage researcher; Betty Maxfield, demographer of the Army; Bruce Orvis, director of the Manpower and Training Program at the Rand Corp.'s Arroyo Center, and Robert Brandewei, director of the Defense Manpower Data Center in Monterey, Calif.
All said the story and NPP analysis lacked context because they did not report trends over the past several years and did not look at "nationally representative data" or the entire recruit population. A statement from Gilroy and Maxfield said that "incomes and socioeconomic status of recruits' families closely mirror the U.S. population. These findings are contrary to those" in Tyson's article.
Kane agreed that a higher proportion of recruits come from rural areas and the South, which is historically true. "But the key word is proportion. The data in the Post article are accurate, but the inferences are not," he said. "The percentage of recruits coming from poorer areas has declined every year since 2001 and the percentage coming from richer areas has increased."
The Pentagon statement said the Post article used "anecdotal evidence and . . . data compiled by the NPP that were incomplete. The Department of Defense does not know whether the data are right or wrong, but the data are clearly not representative or even appropriate for part of its analysis."
In particular, the Pentagon said the NPP considered "the Army's top 20 counties for recruiting." Though those counties produced an above-average number of recruits, the counties "account for a minuscule number and proportion of total recruits . . . 275 out of 180,000 recruits (less than 0.2 percent). One would be hard-pressed to conclude much about income levels from such a small sample size."
Rand Corp.'s Orvis said, "You just can't look at the top 20 or the bottom 20. You have to look at the entire distribution." Data for 1999-2004, he said, show that the income of recruits' families is close to the national average for homes of youths 17 to 21, and family income among recruits has increased every year since 1999. He also said that half or more of recruits come from urban or suburban areas and that there has been little change in where recruits come from since 1999.
Post National Editor Michael Abramowitz said, "Ann set out to tell the story of what kind of young people are joining today's military. Obviously the armed services draw from a range of demographic, income and ethnic groups. The Pentagon's own numbers indicate that that the military is drawing disproportionally from rural and southern communities, and from families with slightly lower incomes than the population in general.
"The numbers also show a close correlation between the unemployment rate and recruiting. These are the phenomena that Ann accurately described in her story. While we did note some trends, such as the growth in wealthier recruits, we probably could have done a better job highlighting some of the nuances in recruiting patterns and providing more context. But the overall thrust of the story still seems accurate and sound to us."
Brandewei took the NPP analysis off the group's Web site and tried to match it with the same data from his agency. He said he could duplicate only two counties, and in the other 18, his numbers and the NPP's diverged by as much as 20 percent.
Anita Dancs, research director for the NPP, said that the survey was the first the organization has done on military recruiting and that it included Army Reserve figures but not Marine Corps data. "The armed services gave us reasonably raw data and it varied from branch to branch. I can't comment on what the Army provided; it may be different from what we were working with."
She said that while the NPP did not have access to family income data, "we did base our analysis on Zip code data of all of the recruits. We didn't do a comparison over time. We just wanted to put the latest data on our Web site (at http://www.nationalpriorities.org/ ) so that people could retrieve the number of recruits by Zip code, county or state and decide for themselves whether the proportion is right. We continue to believe that the lower- and middle-class is overrepresented" in recruiting.
My bottom line on polls and surveys, no matter what kind: Look for the widest context. Ask as many experts as possible what the numbers mean. Numbers can be right but not tell the full story, and that's the case with the article on recruiting.
Deborah Howell can be reached by phone at 202-334-7582 or by e-mail firstname.lastname@example.org.