The Los Angeles Board of Education shocked the city, and much of the education world, last week by ordering six charter schools shut down after a charter official was found to have orchestrated cheating on state tests. It is rare for a school board to close that many charters at once. Even the local teachers union, often hostile to charters, advised against it.
But more surprising, and perhaps a sign of a significant shift in the national debate over testing, is the fact that the jump in scores at the Crescendo charter system was investigated at all. USA Today, in a series of stories launched this week, has compiled nationwide evidence of inexplicable test score gains, followed by equally puzzling collapses, that experts say suggest cheating but are ignored by the officials responsible for those schools.
Looking at test results in Arizona, California, Colorado, Florida, Michigan, Ohio and the District, the newspaper found 1,610 examples of grades at schools that increased three standard deviations or more over the average statewide gain on the same test. That means the students in that school and that grade “showed greater improvement than 99.9 percent of their classmates statewide,” the story by reporters Greg Toppo, Denise Amos, Jack Gillum and Jodi Upton said.
In 317 instances, USA Today found similarly large year-to-year declines in entire grades at schools. What could cause that? One possibility was that school officials found ways to inflate scores one year, then lost the ability or the will to distort the numbers the next year. It would be unfair to say educators broke the rules without examining the evidence, but that is the point. The school districts could not be bothered to take a look.
If we are going to use standardized tests, and I don’t see any alternative until we find a better way to measure student achievement, then we ought to take seriously the possibility that some schools are making their gains look much better than they are. USA Today cites several examples of schools that failed to investigate glaring cases of possible score inflation.
I think this is important, but I have a bias. My wife Linda Mathews, USA Today’s senior projects editor, conceived and edited the new series, “Testing the System.” Her 13 reporters, it seems to me, have gone deeper into this issue than anyone before, but readers should check out the stories and decide for themselves.
The initial installment acknowledges that sharp gains in achievement can be achieved honestly. It quotes Mike Feinberg, co-founder of the KIPP charter school network and a key subject of my last book, saying that “remarkable growth” is possible with “great teaching and more of it.” USA Today doesn’t say so, but I know that test score gains at one of Feinberg’s schools in Houston and another KIPP school in the District have been investigated and found to have no irregularities. Such is not the case in several other schools reported by USA Today.
At Portsmouth West Elementary School in Ohio, third grade math scores ranked in the 21st percentile in 2007, but in 2008, when those same students completed fourth grade, they soared to the 94th percentile, then declined the following year when they were in fifth grade. The school superintendent attributed the big jump to regrouping students by ability and adding an extra teacher to help with low-performing students.
At Stanton Elementary School in the District, the newspaper said, fourth grade math scores took a big jump, but there was something odd about the test papers. Erasure rates in that grade on that test were ten times the district average. “On a math test administered to 20 students, 345 answers were changed—97 percent of them to the correct answer,” the newspaper said. USA Today obtained D.C. school documents through a Freedom of Information Act request that revealed those scores were investigated but no violations were discovered. There was no explanation for the remarkable number of changes from the wrong to the right answer. The only indication of any action taken was a note saying that one teacher, not identified, was barred from proctoring tests in the future.
At Charles Duval Elementary School in Gainesville, Fla., fourth graders were in the 5th percentile in 2005 in math but in 2006 as fifth graders they reached the 79th percentile. The next two fifth grades also soared, the group in 2008 reaching the 91st percentile. But in 2009 fifth graders were back down in the 1st percentile.
State officials stepped in, USA Today said, but not to investigate if the high scores were falsely achieved. Instead, they said they were just there to get the school back on a upward track. The deputy superintendent for the school district attributed the drop in scores to “the sudden death of a beloved math teacher and the district’s failure to screen adequately for kids who needed extra help in reading,” USA Today said.
That is not, to say the least, a satisfactory explanation. Feinberg at KIPP has never seen such a downturn at any of his schools. When big gains are followed by big losses, he told USA Today, you have to ask “what were the adults doing that might not have been . . . ethical?”
It’s a good question. The many instances of hard-to-explain test score jumps in that many states suggest that school districts should spend some time and money finding out why. Once they have read the USA Today series, the many people who oppose using test scores to measure achievement are likely to adopt this issue as one of their principal arguments. I don’t agree with them on tests, but I share their concern about results that don’t make sense.
Up to now nobody wanted to get involved. School officials say to themselves, why buy trouble? How does it help us to investigate scores that make us look good? The answer is: If someone is putting out dishonest numbers, you and your students, as well as parents and taxpayers, are going to pay for it eventually. USA Today has more stories coming. Hopefully they will inspire a new constituency demanding a hard look at very weird numbers.