If ever there were a meaningless exercise in the annals of evaluation, it would be this one.
The Florida Times-Union newspaper sued the state Education Department to get access to what are called “value-added” scores of teachers that are used to make high-stakes decisions about their jobs. These scores come from student standardized test scores, which are then plugged into a complicated formula that supposedly can calculate the “value” a teacher adds to a student’s achievement. In Florida, half of a teacher’s evaluation comes from these scores and the other half from administrative observation; the ratios are different in different states.
The First District Court of Appeals granted the newspaper’s request, forcing the department to turn over the scores.
Here’s the thing: These formulas can’t determine a teacher’s value with any constant validity or reliability, and testing experts have urged policy makers not to use it for any high-stakes decisions about students, teachers, principals or anybody else. Unfortunately, Florida and many other states, encouraged by the Obama administration, have ignored this advice and now use this “value-added method” (VAM) of evaluation.
There are numerous problems with using VAM scores for high-stakes decisions, but in this particular release of data, the most obvious and perhaps the most egregious one is this: Some 70 percent of the Florida teachers received VAM scores based on test results from students they didn’t teach and/or in subjects they don’t teach.
Yes, you read that right: Teachers are being evaluated on students they didn’t teach and/or subjects they don’t teach. How can that be?
In subjects for which there are no standardized test — which is most of them — teachers are evaluated on school-wide averages. Andy Ford, president of the Florida Education Association, said that only about 30 percent of Florida public school teachers teach both students and subjects for which there are Florida Comprehensive Assessment Test exams.
Last April, seven teachers, along with the National Education Association and the Florida Education Association, filed a lawsuit challenging that evaluation system, arguing that it was unfair and violated the Equal Protection and Due Process Clause of the Constitution. One of the seven was Kim Cook of Alachua, Fla., who, as this post explained, was evaluated at Irby Elementary, a K-2 school where she works and was named Teacher of the Year last December. Forty percent of her evaluation was based on test scores of students at another elementary school whom she never taught.
Then in the summer, the state legislature passed a bill making it illegal to evaluate teachers on standardized test scores of students they never taught. But, Ford said, the bill still allows teachers to be evaluated on students they may have in one class, but in a different subject. That means a social studies teacher can be graded on the reading test scores of his/her students. If you are trying to find the sense in that, quit trying.
That, Ford said in an interview, makes all of the scores “meaningless.” He’s got that right.
The real problem here is not the release of the scores — which unfortunately will be viewed by many as reflective of a teacher’s effectiveness when they really aren’t — but rather that the Florida Education Department actually calculates these scores and uses them in evaluations under the mistaken notion that they are useful assessment tools.
If this kind of meaningless exercise doesn’t prove the meaning of meaningless, tell me what does.