(Correction: Changing attribution of paragraph about VAM, which was incorrectly attributed to the American Statistical Association report in an earlier version. )
There are three possibilities why New York Gov. Andrew Cuomo would ignore the position of the American Statistical Association on how not to use statistics to evaluate teachers.
One is that he doesn’t know the organization’s position. Two is that he knows but doesn’t understand it. Three is that he knows and understands but thinks he knows better.
Any of those would explain why the governor (and the Obama administration and other school “reformers”) would ignore warnings from the association and other assessment experts who say that using student test scores to evaluate teachers is not a valid assessment tool, and proceed to ratchet up the percentage by which student standardized test scores would be used to judge individual teachers.
In his recent State of the State Address, Cuomo advanced some major reforms to the state’s school reforms that were at odds with researchers, teachers and others over how to evaluate educators. (His embrace of school vouchers was also at odds with the position of the Obama administration and most other Democrats.) Cuomo said he wants to:
*Boost overall school funding by nearly 5 percent — but only provide the full increase if state legislators do what he wants on school reforms.
*Require that student standardized test scores account for a full 50 percent of a teacher’s evaluation rather than the current 20 percent. Because the current evaluation system, which was itself poorly implemented a few years ago, determined that very few teachers are “ineffective,” Cuomo apparently assumes that the problem with the system is that test scores aren’t a big enough factor.
–It is worth noting that other states have linked test scores to educator evaluation in recent years, a step many took to win federal Race to the Top funding as well a waiver from the Obama administration from the most onerous parts of No Child Left Behind. But many have been attempting to reduce the impact of test scores on evaluation rather than increase it, like Cuomo.
*Require that the other 50 percent of a teacher’s evaluation to come from observations made by school officials and outside educators brought in for the task.
— Assessment experts say that the best evaluation systems use multiple measures, and, in fact, some top-rated school systems in the country have had success with such systems that put no weight at all on student standardized test scores.
*Deny an “effective” rating to any teacher who is rated “ineffective” based on test scores, and award bonuses of up to $20,000 to teachers who are “highly effective” (thanks to test scores of their students).
Here’s what the American Statistical Association said last year in a statement and report (which you can read below) about the high-stakes “value-added method” (VAM) of evaluating teachers. It uses student standardized test scores to determine the “value” of a teacher through complicated mathematical formulas that purport to be able to isolate every factor that goes into student academic performance, including hunger and sickness:
Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.
Education historian and advocate Diane Ravitch wrote about those unintended consequences:
“There will surely be unintended consequences, such as a diminishment in the number of people willing to become teachers in an environment where ‘quality’ is so crudely measured. There will assuredly be more teaching to the test.”
Furthermore, a joint statement by the American Educational Research Association and the National Academy of Education on “Getting Teacher Evaluation Right” says:
“With respect to value-added measures of student achievement tied to individual teachers, current research suggests that high-stakes, individual-level decisions, or comparisons across highly dissimilar schools or student populations, should be avoided. Valid interpretations require aggregate-level data and should ensure that background factors – including overall classroom composition – are as similar as possible across groups being compared. In general, such measures should be used only in a low-stakes fashion when they are part of an integrated analysis of what the teacher is doing and who is being taught.”
Teachers are pushing back on Cuomo’s evaluation proposals. A group called the Concerned Teachers of New York State has written an open letter that teachers are signing in droves. On its Web site, the group says it consists of teachers “who are deeply concerned that the policies that are being pursued by Governor Andrew Cuomo will be actively destructive to the public school system of New York State” and that they “do not represent the positions of any formal organization, anymore than we represent the positions of signatory teachers on any issues other than the ones explicitly mentioned in the Teacher’s Letter.”
Here’s the text of the open letter from the Concerned Teachers of New York State.
In your recent “State of the State” Address, you indicated that you want to pay an annual, $20,000 merit-pay-style bonus to teachers that are rated “Highly Effective” under your revised annual professional performance review (APPR) plan. You also proposed increasing the emphasis of state-derived exam-score-based metrics in a teacher’s APPR rating to 50%, and additionally developing scoring guidelines wherein no teacher can receive a rating greater than “Developing” if the exam-score-based rating is “Developing” or lower. We feel that these two proposals, if enacted, will be extremely damaging to the long-term interests of the public school system and students of New York State.
The increased emphasis your proposed plan puts on state exam scores is problematic for several reasons:
These are the major concerns that we have with your proposed evaluation system/merit-pay scheme. While we are sure that you think you are doing us a favor, we are writing to let you know that we are not interested in your “bonus”. We suggest that you take the money that you have encumbered for this proposal and put it in to programs that will improve the quality of education throughout the state, without regard to teacher-effectiveness ratings. In the spirit of offering constructive alternatives, here are several that we urge you to consder:
Most importantly, talk with us not at us. At current, we do not feel that you hear us in our concerns, or that you listen to our needs. We are very interested in having a Governor that we can talk to about how best to educate the students of New York. But we are not interested in being paid extra to watch as you degrade the quality of public education for the students of New York State. Please support us in our profession, and abandon these destructive proposals.
Very truly yours,
The Concerned Teachers of New York State
Here’s the American Statistical Association’s statement on using VAM to evaluate teachers: