Bob Schaeffer of FairTest, the National Center for Fair and Open Testing, asked to respond to my column last week, “Easing test pressure won’t save kids.” I said eliminating high-stakes tests was not a realistic response to the outbreak of cheating by educators. Pressure is a necessary part of school, or any human endeavor, I said. If we eliminated the standardized tests that upset many people, we would have to use something else, like inspections or portfolios, that would create new pressures and temptations to take shortcuts.
Schaeffer is smart and erudite. I wanted more from him than a response. So I talked him into a debate on the blog. Here it is, the result of e-mails exchanged the past several days:
Schaeffer: Your opinion about the root cause of the Atlanta cheating scandal is contradicted by the Georgia Bureau of Investigation. In addition to citing “unreasonable” score gain targets and “unreasonable pressure on teachers and principals” to achieve results, the “Why Cheating Occurred” portion of their report found that “meeting ‘targets’ by whatever means necessary became more important than true academic progress.”
That’s clearly a systemic problem, not one that can be blamed solely on front-line educators who crossed the ethical line. No question their behavior was wrong. But character flaws were not the primary reason so much cheating occurred.
The problem was accurately predicted by social scientist Donald Campbell 35 years ago. In what is called “Campbell’s Law,” he wrote, “the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures.” He continued: “When test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”
That’s precisely what happened in Atlanta and elsewhere. Overhauling assessment, not more security, is the solution.
Mathews: You are absolutely right. It is the way the assessment was used, the illogical targets and cruel harassment, that was bad. We have some school districts doing better at that, raising few if any of those red flags. I am thinking of Arlington County, Va., where former superintendent Rob Smith made it his priority to reduce the achievement gap between whites and Asians on one side and blacks and Hispanics on the other. He got good results. Andres Alonso seems to be doing the same thing in Baltimore, and handling his own cheating scandal well. Can’t we keep ambitious goals for improvement, but urge school boards to appoint smart leaders like Smith and Alonso to manage them?
Schaeffer: Hoping that “smart leaders” will compensate for the improper use of politically mandated high-stakes tests is unrealistic. Remember that Atlanta’s Beverly Hall, who was in charge during that city’s cheating scandal, was named national “Superintendent of the Year” by her peers. In Baltimore, when Andres Alonso spent $400,000 on test security, scores dropped significantly for the first time in six years.
The problem remains systemic — putting so much weight on standardized exams virtually guarantees that some school employees with try to boost scores “by hook or by crook.” This problem is not confined to education: When bankers were judged primarily by their short-term profits, many made fraudulent loans to boost their compensation. That scandal nearly wrecked the U.S. and global economy.
It’s time to admit that test-and-punish school “reform” strategies are failures, as the recent National Academy of Sciences report Incentives and Test Based Accountability in Education documented. “No Child Left Behind” has been law for 9 1/2 years, but academic performance has improved more slowly than in the pre-NCLB era. And progress toward narrowing achievement gaps is stalling.
The U.S. needs to overhaul assessment in the direction outlined by the Forum on Educational Accountability and others.
Mathews: I am delighted you provided the link to the Forum on Educational Accountability statements. I urge everyone to get a look at them. You are a strong writer who likes specifics, so I am sure you had the same reaction I did to the vague and jargonistic nature of the forum’s prose. But any effort to summarize the views of many educational professionals shares those flaws. Let’s look at the one statement that seems to address our issue. This is from their most recent statement on what should be done:
“13. Rate of improvement: Evidence of a school’s learning outcomes shall be evaluated in light of expected statewide rates of improvement. The expected rates of improvement, to be specified by a formula in ESEA, must be rooted in actual levels and rates of improvement that are attained by the more successful Title I schools and for which evidence indicates that these rates can be sustained over time. Aspects of improvement can be combined into a comprehensive indicator system with a composite expected rate of improvement.”
ESEA, the Elementary and Secondary Education Act, has to be reauthorized by Congress. Congress tends to go for ambitious targets, as they did with No Child Left Behind (which was also part of ESEA), because they want to protect themselves from election opponents who might say they are crippling our nation by accepting learning targets that are too low. Once again democracy will intrude on the dreams of sweetness and light from reformers like these. That will lead to more test pressure and more work for you.
Beyond the failure of this group to say anything specific about how they want to measure progress and what tests, if any, they would trust, doesn’t this take us back to square one?
Schaeffer: You cherry-picked through a number of online documents to criticize a summary of the Forum on Educational Accountability’s detailed proposals for replacing the test-and-punish regime of No Child Left Behind. In fact, the current NCLB law takes more than 1,000 pages to lay out its mandates and formulas. Suggesting that assessment reformers should be able to describe their alternative system in one simple paragraph is as ludicrous as NCLB’s requirement that all children score proficient on standardized exams before 2014.
The failure of NCLB and its state-level clones cannot be reversed by “staying the course,” “raising the bar” or any of the other faith-based notions frequently invoked by high-stakes testing true-believers. As I noted above, Donald Campbell foresaw the problems more than 35 years ago. It is impossible for a nation to test its way to academic success.
The elements of better assessment systems — including tools for evaluating classroom portfolios, verification by school inspectorates, and periodic low-stakes external testing — already exist. Before we can debate the details, however, the Obama Administration, leading Members of Congress, think tanks, and large dollar campaign contributors who have driven the current round of counter-productive education “reforms” have to recognize that their strategy is not working. More importantly, they must understand that high-stakes testing cannot work no matter how much they try to refine it.
Mathews: Oh dear. That’s not going to work, Bob. I am not asking that a reasonable alternative be summarized in one paragraph, but I think a writer as skilled as you could do it in three or four paragraphs. There will be no incentive, and indeed no reason, for the many influential groups you name to “recognize that their strategy is not working” if there is no alternative that makes sense to the voters, taxpayers and parents who decide if we are going to drop the current system.
Schaeffer: Are you seriously arguing that the U.S. should continue a high-stakes testing regime that barely one fifth of Americans believe is helping improve public schools (see annual Phi Delta Kappan/Gallup poll) until there is widespread agreement about the details of a better alternative? Did such a public consensus exist when “No Child Left Behind” was written behind closed doors on Capitol Hill in the fall of 2001?
In fact, true assessment reformers including FairTest, the Forum on Educational Accountability and the Broader, Bolder Approach to Education have repeatedly offered outlines of a better system, one that is more consistent with what schools in the world’s best-performing nations do. It would include:
* Classroom assessment by better trained teachers who evaluate what their students know and can do using portfolios, projects and exhibitions as well as their own quizzes, mid-terms, and final exams.
* External checks by some combination of state-level “inspectorates,” panels of educators from other districts, or community leaders. This “Trust but verify” approach would assure that teachers are grading fairly, accurately and to high standards.
* Periodic low-stakes external testing (a la NAEP) as a further barometer of academic learning.
* A process for reviewing data on classroom, school and district performance, perhaps similar to college accreditation, with mechanisms for intervention that focus on identifying the source(s) of problems and providing the assistance needed to overcome them.
A supposed lack of better models is not the reason why politicians have not yet overhauled “No Child Left Behind.” Rather, their faith in high-stakes exams blinds them to the evidence that their strategy is a failure. Parents, teachers and other public school stakeholders need to let their elected officials know that continuing the current testing status quo is unacceptable for our children and our nation’s future.
Mathews: Thanks Bob. You get the last word. We will have to do this again.