By Jay Mathews
Washington Post Staff Writer
Friday, September 19, 2008 7:30 AM
One of the benefits of finding public schools that work is the chance to study them and discover exactly what they are doing that other schools are not doing. Sadly, this rarely seems a blessing to the educators at those schools, who have to fill out surveys, sit for long interviews and have strangers recording their every move. Often they feel like Michael Phelps might have felt, told to take a drug test every time he won an Olympic gold medal.
I sense these often intrusive assessments have been particularly galling for many teachers at KIPP (Knowledge Is Power Program). It has become the most studied school network in the country, one more indication that it is probably also the best. KIPP serves children from mostly low-income minority families at 66 schools in 19 states and the District, a network way too big for most researchers to handle. But since KIPP began to expand in 2001 from the two successful charter middle schools created by co-founders Dave Levin and Mike Feinberg, scholars have been examining pieces of the growing enterprise.
KIPP has cooperated with the research; one of its "Five Pillars" -- its philosophy of success -- is "Focus on Results." Five independent studies of KIPP have been done so far. A sixth has just been released, available at http://policyweb.sri.com/cep/publications/SRI_ReportBayAreaKIPPSchools_Final.pdf.
It is an investigation of five KIPP middle schools in the San Francisco Bay Area by Menlo Park, Calif.,-based SRI International. I think it is among the best studies ever done of KIPP.
The researchers say they are impressed with the schools, but that is not news. What is intriguing is their collection of data from neighboring regular schools and their interviews with scores of KIPP staffers. As a result, they have both undermined critics of KIPP who say it takes only the best students in poor urban neighborhoods and revealed how some KIPP schools are struggling with student attrition and their own standards of discipline and leadership.
The 119-page report said students at three Bay Area KIPP schools that could be compared to neighboring schools made more progress on average than non-KIPP students and more progress than is normal for American students their age. It said in 10 separate data comparisons that KIPP fifth-graders at those three schools scored better on state reading and math tests by 5.6 to 33 percentile ranks.
Some scholars have suggested KIPP schools have an advantage because, although their incoming fifth-graders are impoverished, they have better records in regular schools than classmates who do not go to KIPP. Researchers Richard Rothstein and Rebecca Jacobsen, using data from KIPP and school districts, said fifth-graders, all of them black, at the KIPP Ujima Village Academy in Baltimore in 2002 ranked on average in the 42nd percentile in reading and the 48th in math, compared with graduating black fourth-graders for all of Baltimore, who were in the 36th percentile in reading and the 34th in math. Of the students entering Levin's KIPP Academy in the Bronx in 2002, 42 percent had passed New York's fourth-grade reading test, compared with only 28 percent of the fourth-graders in the 31 regular public schools in that area.
KIPP has since provided data from the District and Houston showing no such advantage, to which they are likely to add the Bay Area results. The SRI study demonstrates that the KIPP students, before they entered KIPP, were not only below the average for their local school systems but also below average for fourth-graders in neighboring schools with similar demographics. "Students with lower prior ELA [English Language Arts] and mathematics achievement on the CST [California Standards Test] were more likely to choose KIPP than higher-performing students from the same neighborhood," said the research team, led by Katrina R. Woodworth. Using the three KIPP schools for which they had comparative data, the report showed non-KIPP fourth-graders in those districts scoring 344, 344 and 323 on the state reading test, compared with fourth-graders destined for KIPP scoring 307, 311 and 316, respectively.
The SRI report revealed, however, that attrition rates for Bay Area KIPP schools were high, a problem KIPP confirmed with its own report in May of attrition rates at 45 KIPP schools nationwide. The SRI report said 60 percent of Bay Area fifth-graders entering KIPP in 2003 left before completing eighth grade, and they were usually low achievers. Schools in low-income neighborhoods usually have high attrition rates, because families move frequently. But the KIPP rate was higher, the researchers said, than two local Bay Area regular school systems, which reported 22 percent and 50 percent attrition during the same period.
Caroline Grannan, the most active and knowledgeable blogger by far on Bay Area KIPP issues, applauded the report. She said it confirmed her findings on KIPP attrition and her speculation that the least successful students were the ones leaving. She said researchers should focus on how much of KIPP's record for raising achievement is due not to its longer hours and intense teaching but to low achievers dropping out.
KIPP officials said they are working on the attrition problem. They said Bay Area attrition has been declining recently. The May report said the Bay Area KIPP attrition rate for 2006-7 was 14 percent, below the network's national rate of 16 percent but above the 10 percent average attrition rate for KIPP regions that have established several schools and coordinate their fundraising and other activities. They said students are not being forced out; in the most recent year studied, only three students in the five Bay Area schools were expelled for disciplinary reasons. KIPP officials said it is sometimes difficult to persuade students to stay in a program that demands hard work, but they are finding better ways to win that battle because the benefits to those who do stay are so obvious. Since 2001, middle school students who completed four years at KIPP increased their average math achievement level on average from the 40th to the 82nd percentile and their reading level from the 32nd to the 60th percentile -- gains not seen anywhere else.
The SRI researchers endorsed the KIPP view that California's basic funding formula for charter schools -- which are public schools independent of school system rules -- is an obstacle to success. Bay Area KIPP and other charter schools receive less than $6,000 per pupil, about half what KIPP charter schools in New Jersey, New York and the District get. KIPP has not opened any new middle schools in California since 2004. The only ones now planned are those with guaranteed support from private donors, as has been promised by billionaire Eli Broad for KIPP schools in Los Angeles.
Most of the Bay Area KIPP teachers interviewed said they had regular support and observations by their principals, considered a key to the organization's success. KIPP school leaders are selected for their track records as teachers. They are given a year's training before starting schools in which they have the power to hire, as well as fire, teachers as they see fit. The SRI interviews, however, indicated that one Bay Area KIPP school leader was not meeting the organization's standards, with fewer than half of the school's teachers saying their principal carefully tracked student progress, monitored teacher quality and knew what was going on in their classroom. Only 8 percent said that principal regularly observed their classes. The study did not identify the school or the principal.
The study also took a close look at how hard the schools' mostly young teachers were working. KIPP critics and supporters have wondered whether their faculties can sustain the necessary effort over the long term. "The typical Bay Area KIPP teacher reported working 65 hours a week in 2006-07," the report said. Annual teacher turnover rates ranged from 18 to 49 percent.
The report indicated that one advantage for KIPP teachers is that they are spared the compliance monitoring and latest fads that fill the days of regular public school educators with red tape. "It is not the external accountability that drives day-to-day operations," the report said. "It is internal or professional accountability. . . . Lower than intended test scores lead to closer inspection of problems and discussion of individual students' needs rather than a search for a new program that promises a quick fix."
This will not be the last time KIPP teachers are asked to produce their data and explain what they are doing. Mathematica Policy Research Inc. has begun multiyear study of KIPP schools nationally that probably will be the most detailed look at the achievement results of a charter school organization ever attempted. That study will probably take an entire book to explain, and that is a book I plan to write.