The convoluted statistical model that the state uses to evaluate how much a teacher “contributed” to students’ test scores awarded her only one out of 20 possible points. These ratings affect a teacher’s reputation and at some point are supposed to be used to determine a teacher’s pay and even job status.
The evaluation method, known as value-added modeling, or VAM, purports to be able to predict through a complicated computer model how students with similar characteristics are supposed to perform on the exams — and how much growth they are supposed to show over time — and then rate teachers on how much their students compare to the theoretical students. New York is just one of the many states where VAM is one of the chief components used to evaluate teachers.
If it sounds as if it doesn’t make a lot of sense, that’s because it doesn’t. Testing experts have for years now been warning school reformers that efforts to evaluate teachers using VAM are not reliable or valid. But reformers, including Education Secretary Arne Duncan, have embraced the method as a “data-driven” evaluation solution championed by some economists. Earlier this year, the American Statistical Association issued a report slamming the use of VAM for teacher evaluation, saying in part:
*VAMs are generally based on standardized test scores and do not directly measure potential teacher contributions toward other student outcomes.*VAMs typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model.
Lederman filed her lawsuit against New York State Education Commissioner John King Jr., Assistant Commissioner Candace H. Shyer and the Office of State Assessment of the New York State Education Department, challenging the rationality of the VAM model being used to evaluate her and, by extension, other teachers. The suit alleges that the New York State Growth Measures “actually punishes excellence in education through a statistical black box which no rational educator or fact finder could see as fair, accurate or reliable.”
The lawsuit shows that Lederman’s students traditionally perform much higher on math and English Language Arts standardized tests than average fourth-grade classes in the state. In 2012-13, 68.75 percent of her students met or exceeded state standards in both English and math. She was labeled “effective” that year. In 2013-14, her students’ test results were very similar but she was rated “ineffective.” The lawsuit says:
This simply makes no sense, both as a matter of statistics and as a matter of rating teachers based upon slight changes in student performance from year to year.
Superintendent Dolan supported Lederman, saying in an affidavit:
As superintendent of the GNPS, I have personally known Dr. Lederman for approximately 4 years. I have had the opportunity to meet with her personally. I have also reviewed her record of teaching, particularly the performance of her students on New York State assessment tests. I can personally attest that she is highly regarded as an educator by the administration of GNPS. Her classroom observations have consistently identified her as an exceptional educator. She is widely regarded in the GNPS as someone who brings out the best in her students. She has taught for seventeen (17) years in the GNPS and her record is flawless.
Sharon Fougner, the principal at Elizabeth M. Baker Elementary School, where Lederman teaches, signed an affidavit saying that she believes the awarding of 1 out of 20 possible points to Lederman under VAM is”arbitrary and capricious” and agreed with Dolan that Lederman is an excellent teacher.
Still, the state of New York says she is “ineffective,” and offers a teacher no way to appeal the result.
The lawsuit will be worth watching because it is taking on the entire notion of VAM. If VAM were to fall in New York, more legal challenges would be likely in other states.
(Correction: Earlier version said in one place that superintendent called Lederman an administrator. He didn’t. He called her an educator.)