The Washington PostDemocracy Dies in Darkness

The irony in new study that bashes popular teacher evaluation method

Bill Gates (Rick Wilking/Reuters)
Placeholder while article actions load

They just keep on coming. Last month, a report was released by the American Statistical Association, the largest organization in the United States representing statisticians and related professionals, that smacked the “value-added method” (VAM) of evaluating teachers that has been embraced by school “reformers” in most states. And now, there’s new research that does the same thing.

These reports support the findings of other experts who have long warned against using for high-stakes purposes VAM, which purports to be able to take student standardized test scores and measure the “value” a teacher adds to student learning through complicated formulas that factor out other influences on student achievement (such as being hungry or tired or sick).

The newest one, as my colleague Lyndsey Layton reported, which was published Tuesday in Educational Evaluation and Policy Analysis, a peer-reviewed journal of the American Educational Research Association, looked at data collected through a project funded by the Bill & Melinda Gates Foundation, called the Measures of Effective Teaching. Gates has poured hundreds of millions of dollars into projects that evaluate teachers in part by student scores on standardized tests as a way that supposedly measures a teacher’s role in “student growth.”

But in something of an irony, the new study,  Instructional Alignment as a Measure of Teaching Quality, which found little or no correlation between quality teaching and the appraisals teachers received using VAM, was also funded by the foundation. The authors, Morgan S. Polikoff at the University of Southern California and Andrew C. Porter at the University of Pennsylvania, question whether VAM data will be useful in evaluating teacher performance and shaping classroom instruction.

“While value-added measures do provide some useful information, our findings show that they are not picking up the things we think of as being good teaching,” said Polikoff. “Given the growing extent to which states are using these measures for a wide array of decisions, our findings are troubling.”

While  there are economists who argue that VAM can measure teacher effectiveness adequately, testing experts, academics, and other economists say that more than abundant evidence shows that it doesn’t, and that reformers should stop trying to evaluate teachers and principals with unreliable and invalid measurement tools. If you want to look other articles and reports on problems with VAM, here’s a list of  70, compiled by Audrey Amrein-Beardsley, associate professor at Mary Lou Fulton Teachers College at Arizona State University, and published on her blog, VAMboozled!

This method of assessment has become increasingly adopted during the Obama administration because Education Secretary Arne Duncan made these teacher evaluation systems mandatory for states that wanted to compete for Race to the Top grant money or receive a waiver from the most onerous demands of No Child Left Behind. In fact, the Education Department  just revoked Washington state’s NCLB waiver because it had failed to do this.

It all makes you wonder if there is a point at which Gates and Duncan will look at the  evidence that keeps piling up about VAM and question their own embrace of the assessment method. There’s been plenty of opportunity already for them to do this, but better late than never would apply here.