Plans for Harvard economist Roland Fryer to conduct an independent study of the IMPACT evaluation system are off, scuttled by DCPS’ reluctance to employ what he called “the gold standard in empirical research.”
DCPS said it will work with the Washington Teachers’ Union to find a replacement for Fryer, founder and director of Harvard’s Education Innovation Laboratory (EdLabs). The selection of a researcher to study IMPACT’s effectiveness, and to make recommendations for improvements, was included as a side letter to the 2010 collective bargaining agreement. Many teachers have said that IMPACT’s criteria disadvantage instructors who work in high-poverty schools with challenging student populations. Very few teachers judged “highly effective” under IMPACT work in schools in Wards 7 or 8.
The project was a sticky issue from the beginning. The selection of an outside researcher was supposed to be mutually acceptable to the union and the District. But former WTU president George Parker said he never signed off on Fryer. His successor, Nathan Saunders, also balked. There was no question about Fryer’s credentials (MacArthur Fellow, one of Harvard’s youngest tenured professors), but Saunders said he was troubled by his connections.
Saunders cited Fryer’s prior experience as a contractor hired by former Chancellor Michelle A. Rhee to assess the now-defunct “Capital Gains” program that paid cash to middle schoolers for good grades and behavior. There was also the issue of EdLab’s funders, which included private foundations underwriting performance pay bonuses that are part of the IMPACT system.
Saunders said Fryer “might be predisposed to a certain point of view.”
What ultimately sank the venture was Fryer’s interest in randomly assigning some teachers into “treatment” and “control” groups. Random assignment is considered one of the strongest research designs because it ensures that the sample selected represents the characteristics of the entire group under study.
But Jason Kamras, DCPS chief of human capital, said in an e-mail that Fryer’s design “was not practical from our perspective for the efficient running of schools.”
“We worked together for quite some time to try to make it work, but ultimately couldn’t come up with an approach that wouldn’t disrupt instruction for students,” he said.
Kamras explained that Fryer’s plan would have required suspending the contract provision for “mutual consent,” under which a principal and a teacher must both agree on the educator’s assignment to a given school.
“Given this, we came to a mutually agreeable and amicable conclusion that the study was not going to work,” he said. “I’ve informed President Saunders of this, and we will be meeting shortly to identify another research partner.”
Fryer said he retains “tremendous respect” for Kamras and Chancellor Kaya Henderson, but that he had no choice but to move on.
“I am a pointy-headed economist who refuses not to aim for the gold standard in empirical research (i.e. randomization) whenever possible,” he said in an e-mail Wednesday morning. “Anything less is not fair to kids, their parents, or the policy makers who use the research.”