In our recent report, "The Reign of ETS: The Corporation That Makes Up Minds," we argued that the claims Educational Testing Service (ETS) makes about its admissions tests are unsubstantiated and represent a specialized kind of fraud against the test-taker. In her comment on the report, Jessica Tuchman Mathews hurried to endorse ETS's contention that the tests significantly predict a student's college performance, as measured by freshman-year grade point average. She questioned the statistcal approach that led us to conclude, based on data from 827 of ETS's own validity studies, that the Scholastic Aptitude Test improves the prediction of first-year grades minimally, by an average of only 3 to 5 percent. We drived these figures, confirmed by a recent Harvard Medical School study, by using an ETS method (found in Prof. Cameron Fincher's paper "Is the SAT Worth Its Salt?," distributed by ETS and the College Entrance Examination Board), a point Mathews ignored.
The point to remember is that for the vast majority of applicants, ETS test scores predict first-year grades no better than a roll of the dice.
ETS itself admits that the tests are even less helpful in predicting upper-level grades, extracurricular accomplishments or career achievements.
Yet predictions based on ETS scores have assumed enormous proportions in admissions decisions. Mathews asserts that our report "exaggerates" this influence. She fails to mention a 1976 College Board survey, which our report cites, where 80 percent of the schools surveyed identified test scores as the single most important factor in admissions decisions.
Instead, Mathews quotes a 1979 College Board survey wherein "fewer than 2 percent" of colleges rank test scores as "the most important factor" in admissions. If taken at face value, the 2 percent suggests a truly remarkable turnabout from the 80 percent that ranked tests as most important in 1976. The explanation can be found in the design of the 1979 survey question: presented with 13 different factors and asked to rank them on a scale of 1 to 4, fully half of the four-year colleges failed to list anything as their "single most important factor," thus rendering the answer an unreliable indicator of their actual admissions priorities. Despite this unwieldy format, over 50 percent of the four-year colleges indicated that "test scores were a very important factor"; and 30 percent of all colleges reported using cutoff scores. i
Mathews claims our "unsettling" assertion that test scores are more related to the family income of test-takers than to their potential for future performance "misrepresents the evidence." She ignores the ETS manual for colleges, quoted in our report, which states: "Every time you move from one income group to the next higher, the mean SAT-verbal and mathematical scores increase." Moreover, studies have shown that the lower the income, the stronger the correlation of scores and income. And beyond the first year, scores and income correlate far more than scores and performance. These correlations suggest a class bias in the test itself.
Still, ETS and Mathews are persuaded that the test scores reveal that working-class and low-income students in fact have less "aptitude" generally than their upper-class counterparts. This is simply erroneous. An American Council on Education study of 36,581 students in 55 colleges concluded flatly what parental income did not correlate with students' freshman grades. In addition, an ETS study of 15,535 students found, "Students from families with different incomes did not significantly differ in the number or level of accomplishments [outside the classroom]."
Despite this evidence, Mathews claims that, without ETS tests, schools would be forced to use measures more vulnerable to social and racial bias. Experience and studies, such as those of Temple Law School and the University of California, however, have shown that an admissions policy based on grades and other criteria will admit substantially more working-class and minority students than one based on standardized tests.
Still, Mathews argues that ETS tests ought to continue shaping people's futures because they remind policymakers of the "bad news" that class inequities permeate this society. But these class-bias tests perpetuate the inequality that ETS claims they merely illuminate.
This country needs better tests and other broader methods of student evaluation to surface systematically the different talents and qualities of its people. There is much to be lost from dependence on the false verdicts of ETS tests.