By Jay Mathews
Washington Post Staff Writer
Tuesday, February 28, 2006 10:54 AM
Bob Somerby, the energetic and insightful proprietor of The Daily Howler Web site , has accused me of leaving an important fact out of my Feb. 2 story in The Post on rising test scores at Alexandria's Maury Elementary School. I think he is right to complain, and I hope in my explanation I will shed more light on why the new system for rating schools is frustrating-- and yet better than the alternatives.
My story said Maury had the lowest passing rates in the city in 2004 on the state reading test, but managed to improve enough in last spring's tests to make AYP-- federalese for "adequate yearly progress"--the benchmark for schools under the No Child Left Behind law. I wrote the story to show how complex and confusing the No Child Left Behind assessment system was, with officials at the Virginia state education department having to make very subjective judgments about which schools met the standard and which did not.
Somerby is a careful reporter. I knew that the minute I saw that he spelled both my name, and the differently-spelled name of NBC talk show host Chris Matthews, correctly in the same issue of the Howler, something that many other publications have failed to do. Somerby thought the No Child Left Behind standards were absurd. Poking around the Alexandria schools' Web site, he found a fact about Maury that he felt proved it, and that I had not mentioned in my story.
The Web site shows that the passing percentage for third-graders in English at Maury the first time they took the test last spring was 27 percent, but the overall English passing rate for third- and fifth-grade English, after factoring in retests, was 92 percent. Somerby said Maury appeared to be two schools, the one in my story, with the headline "A Study in Pride, Progress," and a very different one on the Web site.
"Inside that low-income school only 27 percent of third-grade students passed that Reading/Language Arts test last year," Somerby said. "And no one could really be 'proud' of that score -- unless the kids at Maury don't count. Across the state of Virginia last spring, 77 percent of third-graders passed that very same test.
"One Maury School is a study in progress. The other Maury seems to be floundering. It seems to be a low-income school whose children need tons of help, not a front-page free ride from The Post, with a photo of a gorgeous child smiling.
"But which school is the real Maury School? The 'study in progress' described in the Post? Or the school whose scores are a study in failure? At this point, we simply can't tell you -- although we'd be likely to bet the ranch that the low score is more on the mark."
Somerby dug further, and soon got an explanation from Alexandria schools testing and assessment director Monte Dawson, my prime source on the story. The big jump, Dawson explained, came mostly from the fact that 12 of those third-graders retook the test and passed it, upping the passing rate considerably, but only because of a odd mathematical rule approved by the state school board in 2000. Somerby quoted Dawson's explanation:
"Remediation Recovery, which has been around since 2001, means that fourth grade students who failed the third grade test in 2004, got to retake the third grade test in 2005. Up until this year (2005), if they passed the third grade test, then they were included in the numerator only of the calculation to determine the third grade passing score. As illustration, if 4 out of 5 third grade students passed and 1 out of 5 fourth grade Remediation Recovery students passed, the passing percentage would be 100 percent."
In writing my story, I looked at the dozens of pages submitted in Alexandria's request for Maury to make AYP and did not see the 27 percent figure Somerby found on the school Web site. I am sorry I missed it because it illuminates an interesting part of the process, and fortifies the main point of my story -- that this is way too complicated for most of us to understand.
I reconstructed what happened with the help of Virginia state officials. When Maury's 19 third-graders took the English test the first time last spring, five passed and 14 did not. Of the 24 fifth-graders who took the English test, 22 passed. The school worked with the third-graders who did not pass it and gave them a retest, and 12 passed on that second try.
Counting third- and fifth-graders together, 62 percent passed the English test the first time. So how did we get to 92 percent passing rate for those two grades in the final tally?
Charles Pyle, spokesman for the Virginia Department of Education, explained that in 2000, the state school board changed the counting procedure to encourage more schools to do what Maury did -- give the students who failed some extra help and let them try again. Often the second-test passing rates of students who flunk a test initially are lower than their class's overall passing rate, since they are the class's weakest students. So if those second-test results were combined with the first test results in the usual way, it would likely lower the overall percentage and make the school look worse than otherwise. School districts in Virginia figured this out and resisted the urge to work with their lowest-performing students and test them again.
To give schools an incentive to make that effort, the school board ordered an unorthodox change in the way the school percentage would be calculated after the retesting. If a school had 100 students, with 30 failing the test the first time and 10 of those passing the test the second time, they could add 10 to the 70 who passed the first time, divide those 80 passing students by 100, and get a nice boost from 70 to 80 percent in their passing rate. Done the conventional way, they would have had to add 30 to the denominator as they added 10 to the numerator, and gotten a passing rate of only 62 percent, lower than the 70 percent rate they had before.
Wasn't that fun? This is another good example why I am glad I paid attention in fifth-grade arithmetic.
This explanation did not make Somerby happy, however. To him, it dramatized the idiotic complexity of the assessment process, and I agree with him. As I have said many times in this column, much of the No Child Left Behind law makes no sense in the real world. The notion that almost all students will reach proficiency in math and reading by 2014 is a fantasy, unless we push our definition of proficiency down so low that it becomes meaningless. Important members of Congress from both parties put that in there because they did not want to be accused of condemning some children to non-proficiency, and in an era where politicians are labeled anti-war for simply questioning the decision-making that led to war, I understand their concerns.
So No Child Left Behind is a mess, as Somerby says, but what is the alternative? Somerby is a smart guy. If he were given dictatorial power to set up a national school assessment process, I imagine his plan would make great sense and not require all this bizarre fiddling with passing rates.
Unfortunately, in the real world, under the Constitution, he would have to share the power to create his plan with the states, the courts and, most importantly, the Congress, the architect of the least sane parts of No Child Left Behind.
So should we just scrap school assessments based on testing altogether? I would say no. Despite all the confusion and frustration, Education Week's Quality Counts reports show that students are learning more than they were 10 years ago when many states were resisting testing and higher standards.
The measure of a good education program is whether it is helping kids. Despite Somerby's understandable distress at that 27 percent passing rate for last year's third grade, Maury kids got more help and the school as a whole is doing better.
Because of No Child Left Behind, the Alexandria superintendent transferred Lucretia Jackson, one of the best principals I have ever seen in action, to Maury and gave her a carefully selected staff of teachers to work with. Raising the achievement of low-income students is not easy, but it can be done, and Maury is one more example of that.
Somerby derided the federal requirement of annual improvement because "especially in a small school like Maury, one group of third-graders may not be as capable as the group from the previous year." That is a good point. Both the states and the federal government are moving toward a cohort assessment system where progress will be measured by how much this year's fourth grade improved over the performance of those same children the year before.
But I object to one implication of Somerby's remark -- that children are stuck with whatever capability they have demonstrated by third grade and cannot be expected to get much better. Since I wandered into Garfield High School 23 years ago and found that that inner city Los Angeles school was outperforming all but four high schools in the country in Advanced Placement test participation, and beating the national passing rate on the tests, I have been convinced that the majority of Americans are wrong to think kids from low-income backgrounds cannot be expected to achieve at high levels. And this column, as regular readers know, has been full of other examples of that, because it is No 1 on my list of obsessions.
Improving schools is a messy process and hard to measure, as the Maury example shows. I wish there was a way to clean up the most annoying parts of No Child Left Behind so that Somerby and I would not have to work so hard to figure out what is going on.
But that is the way progress occurs in a democracy, by fits and starts. Despite that embarrassingly low 27 percent passing rate in third grade, those Maury students are getting better, and that is the most important thing.