A new study on standardized testing shows how random events during an exam can affect not only test results, but college and career options and income for the rest of a student's life.
The authors of the paper studied test results for the Bagrut, a series of tests similar to the SAT that Israelis take when they finish high school. They compared those results to data on the students' future earnings and to air pollution measurements on the day they took the test. They found that even a moderate increase in the level of air pollution on the days of the test reduced students' incomes by about 2 percent by the time they became adults.
That's right: Air pollution had a small effect on test scores, but the ramifications of those small differences throughout students' lives affected their careers and their incomes.
Israeli universities rely more heavily on Bagrut scores than do American schools on the SAT, so it is not clear whether random factors like air pollution during an SAT administration could have such a dramatic effect on American students' future careers.
That said, the Obama administration effectively required that teachers be evaluated on the basis of students' test scores until recently. In districts where those evaluations have been used to make decisions about teachers' pay, standardized tests can have real consequences for educators, if not for their students.
The authors of the study are three economists, Victor Lavy and Avraham Epstein of the Hebrew University of Jerusalem and Sefi Roth of the University of London. Israel's frequent sandstorms mean that levels of pollution there can fluctuate widely, they write.
Examining data for 55,873 adults at least 28 years old, the authors concluded that while air pollution had only a negligible effect on their test scores when they were students, those small differences in their results had major consequences for their careers.
Many factors in and out of school affect how students perform on tests. One recent study found that students performed worse on reading tests in the week after a violent crime was committed near the school.
By way of comparison, the effects of pollution on student performance identified in this study are about as strong as those identified in research on teachers whose students do well on standardized tests. That is, taking a test over several days when pollution is somewhat worse than normal has about the same effect on income as spending a year in a classroom with a moderately, rather than an exceptionally, effective teacher.
The researchers also speculated that asthma was part of the explanation for how students performed on the tests. The effects of pollution were more pronounced among boys, low income students and Ashkenazi Jews, all of whom are more likely to suffer from asthma.
Relying on test results to make decisions about college admissions is not only unfair, Lavy and his colleagues argue, but potentially damaging to the economy, since random events like Israeli sandstorms could conceivably exclude gifted and intelligent applicants from important careers.
Advocates for standardized testing have endured several months of discouraging news as teachers and parents have become increasingly frustrated with efforts to give tests more importance in the school system.
In June, the Gates Foundation announced that it did not support making important decisions about teachers' careers based on the results of tests aligned to the Common Core standards, which the foundation helped to propagate. Secretary of Education Arne Duncan said in August that schools would not be penalized for waiting to include test data in evaluations of teachers.
Still, some argue that data from standardized tests forces the public to confront disparities in academic achievement between racial and socioeconomic groups, and that tests give parents the information they need to make an informed decision about educating their children.