If you are among the growing number of students who have taken or plan to take the ACT college entrance test, pay attention. An independent analysis of results in Ohio has concluded that two of the ACT’s four subject tests — Reading and Science — are lousy predictors of college success and should be ignored by admissions officers.
The Stanford and University of Chicago scholars who wrote the report, recently published by the National Bureau of Economic Research (NBER), say scores from the other two ACT subject tests, Mathematics and English, “are much more tightly correlated with college success than are Reading and Science scores. In fact, after controlling for Mathematics and English scores, Reading and Science provide essentially no predictive power regarding college outcomes.”
ACT Inc., which has pulled even with the SAT in total number of test takers, challenged the study’s conclusions. It said the composite ACT score reflecting all four subject tests was still “a good predictor of future college academic success.” An ACT statement said each of the subject tests helps identify strengths and weakness so they can be addressed by their schools.
One of the authors who examined the ACT told me they did not reach any conclusions about the predictive value of the SAT because they lack the data to do so. Both the ACT and the SAT have been pounded by critics, but they maintain a strong hold on the college admissions process. Hundreds of colleges allow at least some students to apply without submitting ACT or SAT scores, since they find high school grades just as good a predictor of college success. But the most selective colleges, including every school in the Ivy League, usually require that applicants take the SAT or the ACT.
The scholars who analyzed the ACT, Eric Bettinger of the Stanford education school and Devin G. Pope of the University of Chicago business school, plus Stanford doctoral student Brent Evans, say that if the Reading and Science subject test results were ignored, the process of matching students with the colleges best for them could be significantly improved.
“By introducing noise that obscures the predictive validity of the ACT exam, the Reading and Science tests cause students to be inefficiently matched to schools, admitted to schools that may be too demanding — or too easy — for their levels of ability,” said the paper, “Improving College Performance and Retention the Easy Way: Unpacking the ACT Exam.”
The statement by the Iowa City-based ACT Inc. did not challenge the analysis of Ohio scores, but said the authors’ conclusion was wrong. “We have 50 years of research showing the efficacy of each of the four subject tests in predicting college outcomes,” the ACT statement said. “Our research indicates that each of the individual subject tests is a strong predictor of a student’s aligned courses.”
The ACT statement noted that students usually score about the same on each of the subject tests, but “that is not always the case. It is these differential patterns that are of special relevance to both college advisors and admissions officers. For example, a student with strong English, math and reading scores, but a weaker science score, provides college advisors with information that can be used for guidance. It enables the college to best prepare this student for coursework in science, perhaps with targeted interventions.”
Bettinger, Evans and Pope said they looked at Ohio Board of Regents data “on all students who matriculated to a four-year public college in Ohio” in 1999. The ACT is scored on a 36-point scale for both the subject tests and the composite average. In Midwestern states such as Ohio, it is much more popular than the SAT.
“Our model predicts that a student who gets an ACT composite score of 24 by getting a 26 on the Reading and Science tests and a 22 each on the Mathematics and English tests is 59 percent more likely to be a first-year dropout and 43 percent more likely to drop out by the third year of college, relative to a student who gets the same ACT composite of 24, but with a 26 each on the Mathematics and English tests and a 22 each on the Reading and Science tests,” the report said.
Admirers of the ACT have long pointed to the Science subject test as a beneficial feature, since the main SAT test has only verbal, math and writing sections.
The report authors said they thought colleges might be reluctant to drop the ACT composite score because that could lower the average ACT of their incoming freshman classes, and that in turn could lower their ranks on the U.S. News &World Report college list.
I don’t think this report is going to have any more effect on use of the ACT than the many critical examinations of the SAT have had on its use. But it is always possible that such research may someday reach a critical mass, and the two tests will lose influence over the admissions process. Some admissions officers have suggested substituting results on college-level exams such as Advanced Placement or International Baccalaureate, but that also is unlikely to happen soon.