Earlier this year, North Carolina trumpeted a spectacular high school graduation rate of 97 percent at a time when 1 in 3 students in the state, and nearly 1 in 2 African Americans, routinely fails to receive a high school diploma.
Other states were not far behind. According to data compiled by the Department of Education, states with higher-than-average graduation rates include Wisconsin (92 percent), Indiana (91 percent), California (87 percent) and Maryland (85 percent).
If some of the claims sound too good to be true, it is because they are based on "extremely unreliable" data and "ludicrous definitions" of high school dropout rates that vary widely from state to state, according to a report released yesterday by the Education Trust, an education think tank.
The Education Trust found that many states provided "misleading" data, while three states and the District of Columbia failed to supply any information. All 50 states and the District of Columbia are required to report high school graduation rates to the Education Department under the accountability provisions of the 2002 No Child Left Behind Act.
"We've got to end this rampant dishonesty about graduation rates," said Education Trust Director Kati Haycock.
Educational researchers have long complained about the unreliability of state high school graduation and dropout data. But the issue has assumed new importance with the passage of No Child Left Behind and the Bush administration's embrace of more transparent reporting mechanisms to monitor the achievement gap between white and minority students.
Unofficial estimates of graduation rates, based on analysis of school enrollment figures, suggest that 30 percent of incoming ninth-graders never complete their high school education. For minorities, the proportion of dropouts rises to about 50 percent. But many of these dropouts are classified as "transfers" to other school systems in official data.
States use a wide variety of different methods to calculate graduation rates, making state-by-state comparisons virtually meaningless. In most states, there is no statewide system for tracking individual students, which means that their education departments depend on reports by school principals, which may or may not be reliable.
"As a nation, we spend 40 times as much money checking data on test scores as we do on whether students complete school," said Gary Oldfield, a Harvard University professor who recently completed a study of graduation rates in southern states. "Some of the data doesn't even pass the laugh test."
The Education Trust report praised some states, such as Idaho, Washington and Alaska, whose graduation rate statistics are very close to unofficial estimates. Washington, for example, reported a graduation rate of 66 percent.
But it criticized other states for using "irrational" graduation rate definitions. It cited the example of North Carolina, which excludes dropouts from its data, and looks only at the percentage of graduates who get their diplomas in four years or fewer.
"We know there's a problem of apples and oranges," said Janice Davis, North Carolina's acting superintendent of education. She said the state was changing its reporting system to try to get more meaningful data.
The Education Trust also criticized states for "laughingly" low targets on improving graduation rates. Most states have declared that any progress at all -- even one-tenth of 1 percent -- is sufficient to meet their federal obligations under No Child Left Behind. North Carolina is one of several states, including Virginia, that is developing a statewide system of tracking students through a unique student identification number. In theory, this will make it more difficult for principals to classify dropouts as "transfers" to other schools. Virginia hopes to have the new system in place by 2008.
Virginia's official high school graduation rate is 82 percent, eight percentage points higher than the most widely used independent estimate by the Urban Institute, a Washington, D.C., think tank. The Maryland rate of 85 percent is 10 percentage points higher than the Urban Institute estimate.
Gary Heath, assistant Maryland superintendent for accountability, said he was "real comfortable" that the state was reporting accurate numbers, based on audits of 25 to 30 of 250 high schools. He said the unofficial estimate failed to pay sufficient attention to special education students, who often take more than four years to graduate, and population movements, including military base closures.
District officials have acknowledged flaws in their reporting system and say they are working to produce a better system. The official District dropout rate is 6.9 percent, considerably lower than most unofficial estimates.