The Aspen Institute’s study of IMPACT’s first year (2009-10) , draws no strong conclusions about the closely-watched D.C. teacher evaluation system, adopting a cautious “promising-start-much-work-to-be-done” formulation. But two things make the report, released Tuesday by the nonprofit public policy and research organization, worth reading. DCPS’ cooperation with the author, former Boston assistant schools superintendent Rachel Curtis, produced some revealing data. The other is that Aspen didn’t rely on DCPS or the Washington Teachers’ Union to put together its focus groups of teachers and principals for a discussion of issues surrounding IMPACT.

A couple of the nuggets:

Teachers with high “value-added” scores--meaning that their students met or exceeded predicted growth targets on the DC CAS--didn’t necessarily do well on the Teaching and Learning Framework (TLF), the nine-part test of classroom skill that is at the heart of IMPACT. “At this early stage in the use of value-added analysis nationally the hope is that there is a strong correlation between a teachers’ score on an instructional rubric and his or her value-added score,” Curtis wrote. “This would validate the instructional rubric by showing that doing well in instruction produces better student outcomes.”

But that isn’t quite the case. A DCPS analysis showed only a “modest correlation” between the two ratings (3.4).

Curtis also reported on the distribution of “highly effective” teachers based on years in DCPS. Those with the least experience (zero to six years) had the lowest representation among the 636 teachers who received “highly effective” IMPACT scores (12 percent). Educators in the six-to-10 year range had the highest (22 percent).

Aspen’s teacher focus group generally agreed that a major virtue of IMPACT was that the Teacher Learning Framework provided a provided the first clear set of expectations for their performance. But teachers also said the initial attempts to roll out the classroom skill test were poorly executed.

“Teachers described their initial introduction to the Teaching and Learning Framework as less than a day of professional development with trainers who used PowerPoint slides and a script. . .Teachers described trainers who were unable to answer probing questions, such as what was meant by the word ‘rigor,’” Curtis wrote.

Others had concerns about how IMPACT is affecting teaching. One teacher about to be observed by a master educator “told the story of being padlocked out of the school auditorium and told that staging a production of ‘Romeo and Juliet’ was not an ‘IMPACT-ready lesson.’ She was advised by a school administrator to send kids who represent behavioral challenges to the library when she’s observed,” Curtis wrote.

Principals, rarely heard in public on the subject of IMPACT, also weighed in, and Curtis compared the torrent of strong opinions to “opening a valve on a fire hydrant.” Some said that the lack of a systemwide curriculum linked to the Teaching and Learning Framework lessened its value. “It’s fine to talk about pedagogy but we’re not talking about content,” one said. And, like the teachers, a number of principals agreed that 2009-10 should have been a pilot year for IMPACT, so that teachers could become more comfortable with the Teaching and Learning Framework without facing high-stakes consequences such as dismissal.

Looming over the IMPACT debate are the more than 700 teachers who were found “minimally effective” in 2010 and who are subject to termination if their scores do not improve. Some teachers and union leaders say the District is doing an inadequate job of supporting the faltering teachers by helping them to improve their practice. They contend that it proves their point that IMPACT is a rating and winnowing tool, rather than one designed to foster growth.

Jason Kamras, the District’s chief of human capital, pushed back at that argument during a Tuesday panel discussion at Aspen. He said master educators and instructional coaches were available to help struggling teachers. There was also a listing of professional development sessions that educators could consult. But Kamras added that there was a limit to what the District could do:

“At the end of the day, we believe it is the responsibility of the individual teacher to seek out [resources and opportunities] to improve their practice,” he said.