Jason Kamras, a previous National Teacher of the Year, designed the District’s teacher evaluation system and is the school system’s chief of human capital. (Ben Tankersley)
Education experts have long viewed teacher turnover as a negative factor that erodes student achievement and contributes to an unstable school environment. But a new study of IMPACT — the controversial D.C. Public Schools teacher evaluation system that has been accused of contributing to the city’s higher-than-average turnover — suggests that not all turnover is created equal.
The departure of teachers who score poorly on IMPACT is actually a good thing because student scores on math and reading tests tend to improve substantially after such teachers depart, according to a working paper to be published Monday by the National Bureau of Economic Research.
In contrast, student scores tend to drop slightly when high-performing teachers leave their assignment for another school or district, presumably because it is difficult to find replacements who are as effective.
On the whole, because of the strong positive effect of exiting low-performing teachers, turnover under IMPACT led to an improvement in average student achievement, the study found.
Jason Kamras, chief of human capital for D.C. Public Schools, said the findings are exciting because they are the first independent evidence “that our districtwide effort to increase teacher quality has led to big increases in student learning” when weak teachers leave.
“To see that effect — particularly to see it play out in high-poverty schools, where it’s needed the most — I think it’s really powerful,” Kamras said.
Still, the results do not fit neatly into any of education reform’s warring camps.
The study lends credence to the idea that using student test scores to identify and remove poor teachers, as IMPACT does, is an effective strategy for improving the quality of the teaching force. But the results also bolster critic complaints that — to the extent that IMPACT creates a stressful working environment that drives good teachers away — it could undermine the goal of improving teacher quality.
“The high stakes associated with IMPACT have been controversial, both within the District of Columbia, as well as in broader discussions of education policy,” the four authors, researchers at the University of Virginia and Stanford, wrote. “There are elements of both sides of this debate in our estimates.”
[IMPACT is shaping D.C. schools’ workforce, study finds]
The study examined the departures of teachers after the 2009-2010 through 2011-2012 school years, the first three years that IMPACT was in use.
Introduced in 2009 by then-Chancellor Michelle Rhee, IMPACT was among the first evaluation systems in the nation to judge teachers in part according to their students’ progress on tests. It also was one of the first, and is still one of the few, to reward teachers who do well (with substantial bonuses) and punish those who do poorly (with termination or the threat of it).
D.C. Public Schools officials, along with some education observers nationally, credit that approach with helping city schools make relatively rapid progress on national math and reading exams, as compared with other big cities.
But other education experts, and some D.C. teachers, say IMPACT unfairly penalizes teachers who work in the most challenging high-poverty schools. And they say hooking a teacher’s pay and livelihood to student test scores and surprise classroom observations can make the job so stressful and unpleasant that excellent teachers decide to leave.
“We must be careful that IMPACT is not forcing good teachers out of our lowest-performing schools,” Michelle Lee, a math teacher at Cardozo Education Campus, told D.C. Council members in December 2013, not long after the time period examined by the new study. “The stress and paranoia I feel on a daily basis . . . is frankly too much.”
[D.C. teachers offer wide range of views on city policies]
DCPS surveys of exiting high performers show that they most often leave for personal reasons, Kamras said, such as a spouse getting a new job or a family member falling ill. IMPACT isn’t even among the top 10 reasons they depart, he said.
The departure of teachers who were rated “ineffective” or “minimally effective” on IMPACT had a significant positive effect on student achievement, according to the study, equivalent to an additional one-third to two-thirds of a year of student learning in math and somewhat less in reading.
“That’s four months of learning in reading and math,” Kamras said. “There are few educational interventions that have had that kind of impact on student learning.”
The effect of losing “effective” and “highly effective” teachers was negative but far less dramatic. Researchers said it was not statistically significant. But these high performers comprised nearly two-thirds of all D.C. teachers who departed in any given year: 13 percent of them left each year, on average.
Low-performing teachers in the District were three times more likely to leave: 46 percent left in any given year.
“Losing 13 percent of the best teachers each year places strong demands on teacher recruitment to prevent a reduction in achievement in those classrooms,” the study said. “However, exiting 46 percent of low-performing teachers creates substantial opportunity to improve achievement in the classrooms of low-performing teachers.”
The District’s overall teacher turnover during the period of study was 18 percent, higher than the average of 13 percent in large urban school districts, according to another recent study.
Since the period of the study, the D.C. school system has tried to reduce the stress that IMPACT causes, giving less weight to test scores and cutting back on classroom observations of highly rated teachers.