Are KIPP achievement gains inflated?
By Jay Mathews,
New research on the nation’s largest and best-performing charter-school network has a dull title — “Student Selection, Attrition, and Replacement in KIPP Middle Schools” — but it adds fuel to a fierce national debate over why KIPP looks so good and whether schools should follow its example.
No charter school network has been researched as much as KIPP, which has 125 schools and 39,000 students in 20 states and the District. Most of the studies say its schools have had large and positive impacts on student achievement when compared with regular public schools. But some smart critics, including scholars Richard Kahlenberg of the Century Foundation and Gary Miron of Western Michigan University, have found a potential glitch in the analysis.
Most KIPP campuses are fifth-through-eighth-grade middle schools. Students arrive far below grade level but flourish because of KIPP’s longer school days and years and careful teacher selection, training and support.
Nonetheless, some KIPP parents move away or decide a school is not right for their children. Kahlenberg and Miron say that inflates the average scores of students who stay, compared with regular schools: At KIPP schools, they argue, lower-performing students who leave early are not replaced by incoming low scorers as they are in regular schools.
Mathematica Policy Research, the education research equivalent of MIT, has been analyzing years of data in preparation for the biggest KIPP study yet, due in a few months. They have found solid KIPP gains even when they include post-KIPP test results from students who leave early.
They decided to examine the attrition issue anyway, and have released an analysis that confirms KIPP’s success while saying its critics may have a point — though not quite the point they initially made.
Mathematica researchers Ira Nichols-Barrer, Brian P. Gill, Philip Gleason and Christina Clark Tuttle say the critics are right to note that students who leave KIPP early have lower fourth-grade test scores than those who stay. They also discovered that the few new seventh- and eighth-graders who enroll in KIPP have better fourth-grade scores than those already there and better than the new seventh- and eighth-graders at regular schools. Those KIPP newcomers could have what is called a peer effect, their example improving the work of other KIPP students.
“Our findings confirm suggestions by Kahlenberg and Miron that selective replacement of departing KIPP students produces a gradual improvement over the course of middle school in the pre-KIPP achievement levels of the KIPP student population,” the Mathematica researchers say.
KIPP Foundation chief executive officer Richard Barth said his schools have reduced attrition to just 11 percent a year and will aggressively recruit more students to fill empty spots in seventh and eighth grade.
How much advantage does the peer effect give KIPP when its achievement gains are compared to regular schools? Not much, the researchers say.
“The best available evidence suggests that KIPP produces its largest impacts on students in their first year at KIPP — before selective replacement could possibly have any effect,” they say. Even students who leave early show positive impacts in later years, they say.
What the peer effect adds to KIPP achievement rates is unclear, the researchers say. Critics did not mention it because they did not have that data.Some studies suggest the peer effect in schools such as KIPP would be as low as zero. Others say the effect could be significant but would produce no more than 29 percent of overall reading gains or 21 percent in math.
Several schools, both regular and charter, have shown gains in low-income neighborhoods by taking the KIPP approach of expanding learning time and improving teacher recruitment, training and support. But they number less than 1 percent of all schools, and it will take much time — and many more arguments — to figure out if they have something all disadvantaged schools can use.
For previous columns, go to washingtonpost.com/blogs/class-struggle.