As they enter the fourth year of what they see as the oppressive No Child Left Behind regime, our state governments are fighting back. Some are complaining of federal interference. Some are filing lawsuits. Some are suggesting they might stop taking the federal money that chains them to the law.
But among their acts of rebellion is one that, for some reason, I have yet to hear them brag about. Many states are finding creative ways to misinterpret the rules for reporting their statistics so that their school children seem to be doing wonderfully even though that often is not the case.
This is the latest version of a game that has been popular since Alexander Hamilton and James Madison created the federal system as a playground for generations of political mischief makers like themselves. I did an earlier column ("Our 26 Most Dangerous Schools and Other Fables," Jan. 4) about how this has distorted reporting on "persistently dangerous" schools under No Child Left Behind. Now there is a new report on how states are hiding their feeble high school graduation rates under thick glops of statistical nonsense. It is "Getting Honest About Grad Rates: How States Play the Numbers and Students Lose," by Daria Hall of the Education Trust, a non-profit organization based in Washington, D.C., that works for higher academic achievement, particularly for low-income and minority children. The report | www.2.edtrust.org/edtrust. is available on the Education Trust website.
No Child Left Behind tries to encourage high schools to improve their graduation rates, but unlike its test score improvement provisions, it does not threaten much action if they don't. It turns out this is like telling all the thieves in the neighborhood that you have turned off your burglar alarm. No Child Left Behind was passed in 2001 because many states ignored similar rules in the 1990s that had no muscle behind them. That is happening again with graduation rates, Hall said.
"This year, states were required to report statewide graduation-rate data to the U.S. Department of Education," she wrote. "But in far too many cases, the information they provided is of little value to school-improvement efforts. In fact, three states reported no graduation-rate data at all. Another seven did not report data broken down by students' race, ethnicity, or socioeconomic status.
"Of the states that did provide graduation-rate information, most reported rates that look dubiously high when compared to the results of multiple independent analyses of state graduation rates. These studies estimate that nationally, almost one-third of all high school students don't graduate on time, with significantly worse rates for students of color. But in many of the state reports, these alarming numbers are nowhere to be found."
Hall found 34 states that chose graduation-rate goals that were lower than their reported graduation rates for 2002-2003, the most recent year they are required to disclose. She also found 31 states, including Virginia, that decided any progress, even the slightest upward bump in graduation rates, would be sufficient to meet the adequate yearly progress provisions of No Child Left Behind as far as they were concerned. There were four other states, including Maryland, that set their required improvement rate at only one tenth of one percent a year. And two states, New Mexico and South Carolina, appeared to have tired of the whole thing and declared they would not require themselves to make any progress at all.
And yet, by some magic, the state-reported graduation rates were almost all higher, and in some cases much higher, than those estimated by an independent expert, Christopher Swanson of the Urban Institute. Hall found this unexpected success particularly intriguing in North Carolina, which told the U.S. Education Department that 97 percent of its high school students got diplomas in 2003.
North Carolina in the past has appeared to take school improvement seriously, but Hall says the state's education officials have been handling their graduation rate figures with the situational ethics I use when calling close, hard serves to my backhand.
The North Carolina graduation rate, Hall said, is "based not on the percentage of students who entered in the ninth grade and received a diploma four years later, but on the percentage of graduates who got their diplomas in four years or less. In other words, students who dropped out of high school were excluded from North Carolina's calculations altogether."
This produced, not surprisingly, a 2002 graduation rate of 92 percent and a 2003 graduation rate of 97 percent, highest in the nation. It also produced the largest gap---33 percentage points---between the state-reported rate and the Urban Institute estimate of reality.
According to Hall, the North Carolinians have been arguing without blushing that their formula is consistent with the wording in No Child Left Behind that says a graduation rate is "the percentage of students who graduate from secondary school with a regular diploma in the standard number of years." Hall says "no administrator, policymaker, or educator could, in good faith," make such an interpretation. She chides North Carolina, and does the same to the federal government for accepting this fairy tale about North Carolina's near perfect graduation season.
The U.S. Education Department announced an expert panel to study graduation rate calculations after the Education Trust and other groups complained about them in 2003, Hall said, but nothing much has happened. "The states that have taken responsibility and steps to improve their graduation-rate calculations and reporting have done so in spite of, not because of, the Department's actions," she said. "And the states that have continued to calculate and report inaccurate data without consequence have lost yet another year they could have used to build public support for the hard work of improving results for students."
Hall noted that many of the states she criticizes plan to put electronic tags on the records of each student and make it easier to determine if they are dropping out or just transferring when they leave a high school before graduating. Many states responding to the Education Trust report cited these efforts in their defense, and said in the meantime they were doing the best they could with inadequate tools. Janice Davis, North Carolina's acting superintendent of education, told my colleague, Post Staff writer Michael Dobbs, that "we know there's a problem of apples and oranges" and that they were trying to get more meaningful data.
Gary Heath, Maryland's assistant state school superintendent for accountability and assessment, said he thought his state's reported 85 percent graduation rate was pretty accurate, and better than the Urban Institute estimate. Charles Pyle, spokesman for the Virginia state education department, said it was unfair to cast doubt on his state's 82 percent graduation rate when it had taken unprecedented action to help students pass the battery of state tests they need to graduate, and it had installed a new computerized record system that will provide a clearer graduation picture by 2008.
Susan Aspey, a spokesman for the U.S. Department of Education, said "we are very concerned that the graduation data doesn't accurately reflect what's truly happening in the states." She said Education Secretary Margaret Spellings was emphasizing the issue, and supporting a project to help states provide better data until their computerized systems are ready.
Advocates of No Child Left Behind note that even when the law has few teeth, as seems to be the case with graduation rates, it still forces the states to provide some data which independent analysts like Hall can chew on. And those states that don't give out the required data at least have to explain why.
States are not accustomed to having to defend their self-congratulatory statistical tricks. But the federal law has forced some of these maneuvers into the open, where curious outsiders can see what is going on. Maybe that will make them less likely to define dropouts completely out of their equations when the next reporting deadline arrives.