You can read this 2014 post about the scary way the cut scores were set for New York’s Common Core tests. And you can read the brief post below, which succinctly explains the problem with cut scores in general. It was written by Peter Greene, a veteran teacher of English in a small town in Pennsylvania, who initially published this on his Curmudgucation blog.
[Teacher: Why the ‘rich student data’ we get from testing is actually worthless]
By Peter Greene
The afternoon is only half over, but the sun is riding a little low in the sky. Within a few hours, it’s going to get dark outside. That’s because we just switched the clocks back an hour, and it’s no longer daylight saving time.
Here’s the thing– it will get precisely as dark as it got yesterday, just as the sun rose just as high today as it rises every single day. The distribution of light and dark through the day, the distribution of the sun’s high points and low points– it will be pretty much the same today as it was yesterday.
What changed is not the distribution of light and dark, but the labels that we put on it. Saturday we labeled this position of the sun 4:00. Today we are labeling it 3:00.
This is a fine way to explain cut scores. The distribution of student scores, the lights and darks, the highs and lows– that stays pretty much the same. What changes is how people choose to label them. We can take the highest point of the curve and we can call it “on level” or “above expectations” or “below expectations.” And the labels we use are our reality. It’s not true to say that right now it is “really” 4:00. It’s 3:00 today.
The position of the sun, just like the number of students who got a certain number of questions correct on a test, a piece of raw data. But what we label it, whether we label it 12:30 a.m. or “exceeds expectations,” is just a label, even an arbitrary label, that we have slapped on the raw data to give it meaning. And we can give it any meaning we want.
Many folks make fun of daylight saving time because it doesn’t really change a thing. Sun is still up for the same number of hours, and we stumble around in the dark for the same number of hours. Nothing really changes except the label we install. If I have the authority, I can make this moment 3:00 or 4:00 or 9:00 or 13:00. It won’t change the reality of the moment– just what we call it. Standardized test results, predictably draped across the bell curve, are the same. If I have the authority, I can label the parts of the curve anything I like. But it won’t change reality a bit.