Kaya Henderson (Marvin Joseph - WASHINGTON POST) Kaya Henderson (Marvin Joseph/The Washington Post)

Public schools in D.C. just saw larger gains on 2013 math and reading tests on the National Assessment of Educational Progress — which is sometimes called “the nation’s report card” — than any other major urban school system in the country. Impressive, right? Well, maybe not so much.

One of the central educational goals of score reformers for more than a decade now has been to boost standardized test scores, this based on the assumption that such jumps indicate kids are learning more. There are a lot of caveats to that assumption.

For one thing, a new study by neuroscientists at MIT, and Harvard and Brown universities shows that students who achieved the highest gains on standardized tests did not show the same gains in the ability to analyze material and think logically. In fact, in many cases no cognitive gains were realized at all, no matter how high the test scores went up. The likely takeaway: Teachers have gotten very good at teaching students strategies to do better on tests.

There are other important points that my colleague Emma Brown points out in this story that make it hard to really know why the rise in D.C. test scores occurred. The test score gains were impressive on the face of it:  five points in fourth-grade reading, eight points in eighth-grade reading, seven points in fourth-grade math and five points in eighth-grade math.

But consider this:

* NAEP scores for D.C. public schools have been going up for a decade, well before Michelle Rhee became chancellor in 2007 and began the test-based school reforms that her successor, Kaya Henderson, has continued.

* DCPS still has the nation’s widest achievement gaps between white and black students and white and Hispanic students than any other big urban school district. Here are the percentages of fourth-grade students who scored high enough to be labeled “proficient” in reading, which in NAEP terms is high:  — 78 percent for whites, 26 percent for Hispanics and 13 percent for blacks.

* Poor black D.C. students score lower, on average, than their counterparts in other cities.

* White D.C. students are generally more affluent than black and Hispanic students. And the city has seen some demographic changes in the last decade in which the proportion of white fourth-graders, Brown noted, has tripled, and the proportion of black students fell from 87 percent to 67 percent.

D.C. Schools Chancellor Kaya Henderson understandably wants the credit for the test score rise to go to her administration’s reform policies, which include evaluating teachers by student test scores. But she can’t really know that that is the reason. In fact, data Brown received shows that black students in the city’s public charter schools overall did better than black students in the traditional public school system.

NAEP is often called the nation’s report card because it is the only measure of student achievement given periodically to a sampling of students around the country. It is seen as a high-quality test, but it still has critics, some of whom point out that the test cannot measure many qualities students must develop to be successful, and others who say NAEP’s definition of “proficiency” is too high.

One of the big dangers of relying on standardized test scores as proof of school reform success is that the link can’t really be made. And when the reforms are credited with scores going up, it stands to reason that they should be blamed when scores go down, which, over time, they pretty much always do before they go up again.

So what does the rise in test scores in D.C. schools really means? Who knows?

(The Washington Post/Source: National Center for Education Studies) – D.C. Public School fourth-graders improved in both math and reading since 2011, but average scores lag behind other large urban school systems.