Anthony P. Carnevale is the director of Georgetown University’s Center on Education and the Workforce. He was vice president of the Educational Testing Service from 1996 to 2003.
The changes to the SAT announced by the College Board this past week will hopefully make it a more relevant, more useful and fairer test. The essay section that can be so easily gamed will be reworked and made optional. The math and reading-and-writing sections will seek to assess skills students may actually need in college. But will the redesigned SAT come closer to realizing the test’s legendary status? Here are some of the myths it is up against.
1. The SAT is the best measure we have for assessing if a student is ready for — and can succeed in — college.
The College Board maintains that the SAT “has a proven track record as a fair and valid predictor of first-year college success.”
But the most reliable studies don’t bear that out. Jesse Rothstein at Stanford University calculates that on its own, the SAT explains a mere 24 percent of the variation in college freshman grade-point averages. By contrast, high school GPA alone explains 34 percent.
Only when combined with other factors does the SAT start to become a useful predictor. When considered along with high school GPA, for example, it explains 41 percent of freshman GPA variation. Yet even then, it may not be as useful as looking at high school GPA plus scores on AP exams, SAT Subject Tests or International Baccalaureate examinations, according to the National Association for College Admission Counseling. And when you look beyond first-year GPA and try to predict later grades and graduation rates, the SAT fares even worse.
Certainly, the version of the test now offered isn’t an indispensable predictor of college success. And it doesn’t look like the redesigned test, to be offered starting in 2016, will justify its outsize role in selective college admissions, either. “The predictive validity is going to come out the same,” Cyndie Schmeiser, the College Board’s chief of assessments, told the New York Times.
2. The SAT has helped establish a national meritocracy.
The SAT was originally adopted by Harvard and other colleges in an effort to gauge natural ability and to make access to their schools less dependent on wealth, family connections and elite prep-school educations. Today, National Merit Scholarships are awarded annually based on Preliminary SAT exam scores.
In fact, the SAT does not recognize merit, and it has failed to move many talented but disadvantaged young people into our top schools. Caroline Hoxby’s research finds between 25,000 and 35,000 students each year whose grades and SAT scores are in the top 5 percent of their high school class but who do not attend the nation’s best colleges due to lack of information.
Who will do well on the SAT and go to the best colleges is mostly settled before students sit down to take the test. The great sorting of young Americans is linked to having the right parents who have the right income and, more important, the right education level. My research with Jeff Strohl has found that highly disadvantaged students (who tend to be black, attend public schools with high poverty rates and come from low-income families with high-school-dropout parents) can be expected to score 510 points lower than average students and 784 points lower than highly advantaged students (who tend to be white, attend private schools and come from wealthy families with highly educated parents).
The SAT is not necessarily doomed to replicate privilege, however. It could be scored to reward striving, providing additional credit to students who outperform expectations based on their social and economic status. After all, the point that students reach says less about merit than it does how far they had to come.
3. Test-prep courses substantially improve scores.
Organizations that provide test-preparation courses are happy to perpetuate this myth. Kaplan, for example, proclaims: “Test prep is the most effective and efficient way to take your score higher.” Anxious parents and students have bought into the myth, making test prep an $840 million industry.
But test-prep courses are not the best use of parents’ money or students’ time. Independent studies show that the effect of test preparation on SAT performance is marginal, boosting scores by 30 points, on average.
The College Board didn’t mention that marginal increase this past week when it announced a partnership with the nonprofit Khan Academy to provide free test prep. “The College Board cannot stand by while some test-prep providers intimidate parents at all levels of income into the belief that the only way they can secure their child’s success is to pay for costly test preparation and coaching,” President David Coleman said. Free test prep may calm some anxious parents. But it’s not likely to have much effect on scores.
4. The SAT can predict career success.
Consulting firms, software companies and investment banks are among the employers who ask job candidates to dig up their SAT scores, according to a report last month in the Wall Street Journal.
It’s a poor recruiting strategy. The SAT assesses a narrow band of academic reasoning ability. The test does not do a good job of measuring “whether someone has the raw brainpower required for the job,” as the Journal’s article put it. It does not assess the skills and personality traits, such as conscientiousness, that drive professional success.
Far more important for predicting a successful career, at least in terms of earnings, are how much education you get and what you major in. A teacher who scored high on the SAT still typically earns less than a manager with a low score. A worker with an associate’s degree can expect to earn $1.7 million over a lifetime, compared with $2.3 million for a worker with a bachelor’s degree, and $2.7 million for a worker with a master’s degree. And college graduates who majored in engineering and information technology earn $70,000 annually, while humanities, arts, education, psychology and social work majors earn less than $50,000.
5. The changes planned for 2016 will bring a 20th-century test into the 21st century.
The College Board hopes that redesigning the test will make it more relevant — at a time when some colleges have stopped requiring the SAT and in the face of competition from the ACT and new Common Core assessments.
But aligning the SAT with Common Core standards, reducing the obscure vocabulary and narrowing the focus of the math section, while positive changes, won’t reverse the declining importance of the SAT in the American education system.
An assessment truly fit for the 21st century would test a broader array of knowledge, skills and abilities. Given the increasing importance of science, technology and engineering, for instance, a single science passage in the reading section, as is planned for the new SAT, is hardly enough. A full science section, as the ACT has, would be more appropriate.
Why should we rely on an imperfect proxy of “aptitude” when other metrics, such as the Common Core subject tests, can tell us the real gaps in students’ understanding?