The Washington PostDemocracy Dies in Darkness

Can coaching truly boost SAT scores? For years, the College Board said no. Now it says yes.

Katerina Maylock, with Capital Educators, teaches a college-test-preparation class at Holton-Arms School in Bethesda, Md., in 2016. (Alex Brandon/AP)

For many years, the College Board said that test prep couldn’t significantly boost scores on the SAT exam — even though a lucrative test-prep industry blossomed around the premise that it could. Now, suddenly, the organization is saying that, in fact, coaching can help improve scores — especially for those kids who use a free coaching program with which it has partnered, the online Khan Academy.

The College Board just announced in a statement on its website:

New data shows studying for the SAT for 20 hours on free Official SAT Practice on Khan Academy is associated with an average score gain of 115 points, nearly double the average score gain compared to students who don’t use Khan Academy. Out of nearly 250,000 test-takers studied, more than 16,000 gained 200 points or more between the PSAT/NMSQT and SAT …
In addition to the 115-point average score increase associated with 20 hours of practice, shorter practice periods also correlate with meaningful score gains. For example, 6 to 8 hours of practice on Official SAT Practice is associated with an average 90-point increase.
“The SAT is a strong measure of college readiness.  It is heartening to see this positive association between personalized practice on Khan Academy and growth in college readiness,” said Khan Academy founder and chief executive Sal Khan. “This was only possible because of the hard work of many people, especially incredible teachers, counselors and school districts who have leveraged these practice tools for their students.”

Note that the statement from the nonprofit College Board says that kids who don’t use Khan Academy for SAT prep get some benefit, too, but not as much as if they use the favored Khan Academy. Khan Academy is a popular nonprofit educational organization created in 2006 by Salman Khan to provide online lessons in math, science and other subjects to help people learn free.

Sal Kahn on his famous online academy

A College Board spokesman said there is a change in the effect of some test prep on student SAT performance in part because the newly redesigned SAT — first administered in March 2016 — is now more aligned with material students are learning in school, and because the Khan Academy knows how to coach to get maximum results, which suggests that other companies don’t.  Zachary Goldberg, senior director of media relations for the College Board, said in an email response to a question about why the College Board now thinks coaching, at least by Khan, could help:

The new SAT is a different test. It is an achievement test that measures what students are already learning in high school and what they need to know to succeed in college and career. With the new SAT there is no penalty for guessing. Students no longer lose points for wrong answers. Gone are “SAT words”— words no one has seen before or will likely see again. Only relevant math concepts are tested. The SAT makes it easier for students to show their best work.
Too much of commercial test prep teaches to the test — looking for shortcuts and tricks to “beat” the test. The SAT in its old format lent itself to this approach. The College Board and Khan Academy firmly believe in practice, and particularly practice that is personalized to pinpoint areas where learners need additional help. Preparing for the new SAT is the same as preparing for college.
The results we released this week show that these free resources advanced students regardless of gender, race, income, and high school GPA. Our partnership with Khan Academy is leveling the playing field and is delivering opportunities for all students. We would encourage you and your readers to listen to the stories of students like D’Andre and Diana from Oak Ridge, Fla. We would hope even our most strident critics would celebrate what these and other students have accomplished after devoting themselves to productive practice on Khan Academy.

The past insistence of the College Board that the SAT could not be coached to any significant effect was repeated and firm. For example, the College Board’s annual report in 1955 said, “If the board’s test can be regularly beaten through coaching, then the board is itself discredited.” In the mid-1970s, William Turnbull, the president of the nonprofit Educational Testing Service, which administered the SAT for the College Board, was quoted in a Stephen Brill New York magazine article as saying that “the existence of the coaching schools is nothing more than the triumph of hope over reality.”

The news seems to contradict a document on the College Board’s website, titled: “Effects of Coaching on SAT Scores,” which says:

Coaching companies’ current estimates of the benefits of coaching for the SAT are much too high. Coached students are only slightly more likely to have large score gains than uncoached students. In addition, about 1/3 of students experience no score gain or score loss following coaching.

and this:

The typical gain associated with coaching is 8 points for verbal and 18 points for math. Coaching seems to result in about one more verbal question correct for 25 to 30 hours of effort and one more math question correct for 8 or more hours of effort.

Bob Schaeffer, education director for the nonprofit group known as FairTest, which works to end the misuse and abuse of standardized tests, is a frequent critic of the College Board and said in an email:

The College Board’s admission that SAT coaching can boost scores significantly once again demonstrates the hypocrisy of the testing industry. After six decades of aggressively claiming that SAT prep courses do not have a  major impact, the College Board has suddenly reversed its position. Of course, the program they now assert can make a big difference is the only one the College Board partners with. Not surprisingly, they did not study the offerings from any test-prep firm, many of which advertise even larger score gains.
Given the latest news, how should a conscientious admissions office value any particular test result — was that 650 Math score, for example, posted by an applicant who took the SAT “cold,” or after using Khan Academy (or one of the other free websites that also offers test-prep courses)? Or was it the result of a $1,000 Kaplan/Princeton Review/Compass Prep/etc. intensive course, a $2,500 personalized tutoring program or intervention by the self-styled New York City SAT “guru” who charges $1,500/hour for his services?

The study presented by the College Board and Khan Academy did not, as Schaeffer noted, evaluate other test-prep companies, and there isn’t a lot of recent data on their effectiveness. A study of commercial test-prep courses released by the National Association for College Admission Counseling in 2009 concluded that students who were coached by these firms gained on average approximately 30 points, but it noted that that was enough to make a difference in college acceptance decisions — though it said there wasn’t concrete data to be certain. Some test-prep providers say they often get increases in scores of more than 30 points after repeated test administrations.

It was the College Board that approached the Khan Academy a few years ago with a proposal to offer SAT coaching online, according to this 2014 New York Times article  about the SAT overhaul. The idea was intended to give more opportunities for low-income students who couldn’t afford expensive test prep to have the same or a similar opportunity to excel on the test. The author, Todd Balf, wrote that in interviews, College Board President David Coleman repeatedly “referred to some test-prep providers as predators who prey on the anxieties of parents and children and provide no real educational benefit.” He called the College Board-Khan Academy partnership on SAT coaching “a bad day for them.”