The Washington PostDemocracy Dies in Darkness

How much ‘value’ can teachers really add to student achievement?

By Chris Gilbert

As a high school English teacher, it’s heartening to think I can “add value” to all of my students. Unfortunately, this is completely unrealistic. I can prepare engaging lessons that appeal to different learning styles, maturity levels, and backgrounds. I can also repeat, reteach, and communicate my concerns to students and parents; my influence, though, is ultimately limited. I cannot force students to engage with curriculum, nor can I control study habits outside of my classroom. I cannot negate the effects of poverty, and I am challenged by a system increasingly subjected to misguided reforms, slashed funding, and excessive testing. Teaching is a messy art involving numerous, unpredictable variables and participants, and the learning process spans multiple venues and points in time. A fair evaluation instrument would address this multifaceted reality. The value-added instrument currently utilized in North Carolina and many other states, though, is unfair and inaccurate.

I first heard about this evaluative model during a faculty meeting. As I sat there, overwhelmed by the enormous, multicolored chart before me, I felt like a character in a dystopian novel. We were learning about the new part of our teacher accountability instrument: Standard VI. My rating on this standard would come from “student growth data” (test scores) and be measured by EVAAS, the “Education Value-Added Assessment System.” The chart was tri-colored, with each shade indicating a specific teaching effectiveness rating: red-“does not meet expected growth”; blue-“exceeds expected growth”; green-“meets expected growth.” In this dystopian story, teachers are evaluated by standardized test scores and branded with color-coded levels of effectiveness, students are abstracted into inhuman measures of data, and educational value is assessed by how well forecasted “growth” levels are met. Surely, this must be a fiction.

Regrettably, this is reality. North Carolina is one of four states to utilize EVAAS statewide, and various districts in 13 other states use it as well. EVAAS’ official website flaunts its ability to “Assess and predict student performance with precision and reliability…[it] provides valuable diagnostic information about past practices and reports on students’ predicted success probabilities at numerous academic milestones.” EVAAS is a complex analytical program that utilizes “multivariate, longitudinal modeling,” and “furnishes the statistical rigor necessary” to predict how students will perform on future tests. Teachers are then evaluated by whether students fail to meet, meet, or exceed predicted performance levels.

Behind this absurd jargon lies an educational reality that operates outside the confines of complex equations and algorithms: students here are unpredictable, evolving, and continually jostled by a myriad of internal and external influences. Yet, obscure software allegedly foretells their testing results, and the data becomes part of teacher evaluations, thus potentially affecting salary and job security. This is increasingly commonplace, as multiple states have hastily adopted value-added models (VAMs) such as EVAAS to assess teachers and receive Race to the Top funding. My attitude about this is one of shock and disgust, as a deep divide exists between policy and scholarship. Researchers have been exceedingly critical of the use of VAMs to evaluate teachers.

Audrey Amrein-Beardsley and Clarin Collins, in their critique of EVAAS, conclude that, “there are consistent problems with inconsistencies with the SAS® EVAAS® data…Perhaps the methodologists pushing, and in this case selling the SAS® EVAAS® model for profit, are promising more than their model can and ever will deliver.” Organizations such as FairTest, The Economic Policy Institute, and The National Research Council (NRC) have also cautioned against using VAMs for evaluative purposes. The NRC states, “VAM estimates of teacher effectiveness … should not be used to make operational decisions because such estimates are far too unstable to be considered fair or reliable.”

Despite their illegitimacy, I believe VAMs are adopted for two primary reasons. First, they provide an expedient way to measure “effective teaching” by producing data, and this reductive evidence perpetuates the illusion that education is an orderly, quantifiable process. The larger reason, though, is that this data lends itself to manipulation by those who shape public policy. VAMs are being paired with Common Core State Standards and forthcoming standard-aligned assessments, and the results of these high-stakes tests are predicted by many to be shockingly low (some states have already revealed such data); this “failure,” coupled with abysmal educator VAM data, would provide further ammunition for those who support the privatization of public education.

Historically, such manufactured crises have presented technocrats with opportunities to implement unpalatable policies. While discussing this in her book,  “The Shock Doctrine: The Rise of Disaster Capitalism,” Naomi Klein quotes Milton Friedman:

He observed that ‘only a crisis–actual or perceived–produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes politically inevitable.’

For those desiring an educational crisis, dismal VAM results could be used to further bolster the narrative that public education is broken; such a crisis would pave the way for more corporate programs and curriculum, more testing, and more profit-driven schools. I hope we will eventually put this dystopian novel down. Most likely, though, we will keep turning the pages.