THE BIG TEST
The Secret History of the American Meritocracy
By Nicholas Lemann
Farrar Straus Giroux. 406 pp. $27
Reviewed by Gerald W. Bracey
What is it about the Educational Testing Service and the Scholastic Aptitude Test that generates such awe, hyperbole and hatred? First came Allan Nairn and Ralph Nader in 1980, with The Reign of ETS: The Corporation That Makes Up Minds. Then came David Owen in 1985, with None of the Above: How the Educational Testing Service Controls the Gates to Higher Education and Success in American Society. Nairn and Nader made ETS sound like the CIA, and Owen's picture looked like a cross between Big Brother and the Wizard of Oz. The University of Delaware's James Crouse and Dale Trusheim, in The Case Against the SAT (1988), clenched their teeth and tried to let cool statistical formulas rather than overheated rhetoric prove that the SAT doesn't help colleges select applicants, doesn't help applicants select colleges, and hurts minorities and poor kids.
Now Nicholas Lemann, a national correspondent for the Atlantic Monthly, takes up the cudgels with The Big Test: The Secret History of the American Meritocracy. His treatment is the grandest in scope, the most literate and most readable. In the end, though, it is too glib and wrong -- dishonest even.
It is wrong because Lemann falls into the same errors as his predecessors. While he makes fewer moral judgments about ETS, he paints the SAT as destiny itself, calling it "the all-powerful bringer of individual destiny in the United States." In his view, selective colleges established SAT-uber-alles admissions policies that led to the creation of a new class he calls "Mandarins."
Lemann presents a history in three parts. Part one describes the events leading up to the SAT, the later founding of ETS and the early years of ETS's successes. The second begins weaving in related events and following the lives of some of the newly ordained meritocrats, the Mandarins. The third focuses on these lives and their interplay, and on the roles they played in the events leading up to the defeat of affirmative action in California. It is a swaggering good tale peopled with colorful characters, from the testmakers who created the SAT in the 1920s to the students who used it 40 years later to launch themselves as Lemann's Mandarins.
The account by Lemann (Harvard '76, he refuses to divulge his SAT scores) identifies the SAT as part of a grand scheme by Harvard president James Bryant Conant and ETS's first president, Henry Chauncey, a former Harvard assistant dean, to establish a "natural aristocracy." The aristoi would be chosen on worth. IQ initially defined worth until the SAT replaced IQ tests for determining it (this replacement is more cosmetic than conceptual: The SAT correlates very highly with IQ tests). The phrase "natural aristocracy" had appeared in Thomas Jefferson's 1782 plan for public education for Virginia, and again in 1813 in a letter to John Adams, where Conant had first seen and been inspired by it. Jefferson contrasted his natural aristocracy, determined by selecting and educating people "of worth and genius," with the degenerate blood-based aristocracy of blood in Europe.
If this sounds too ambitious, conspiratorial even, in the 1930s it was not. The early psychometricians, as testing people call themselves, knew some things. They knew that science was as deserving of their worship as any religion could ever be. They knew they were doing for psychology what Newton had done for physics: laying down the fundamental laws of the mental universe and leaving it to others to work out the details (they apparently were unaware that Newton's universe was collapsing under Einstein's assault). They knew they were right. They knew intelligence was controlled by a single gene and affected virtually everything a person tried to do in life. They knew that the "Nordic" race had more intelligence than Jews, Catholics, Greeks, Hungarians, Italians, Russians, Turks and, especially, blacks. They knew this last because Carl Campbell Brigham, the principal developer of the SAT in 1926, had reached these conclusions in his 1923 book A Study of American Intelligence. Brigham later recanted this view and also called the SAT a "mere supplement" to the rest of the high school record. (According to Lemann, he also opposed the establishment of ETS on the grounds that a single organization could not simultaneously conduct disinterested research in testing and promote its own products. He was right.)
Given everything that these men "knew," it is easy to appreciate their enthusiasm for grand social engineering projects, including those involving eugenics.
The book's central thesis is that Conant and Chauncey succeeded but that the whole scheme backfired. The SAT became the all-powerful determinant of admissions to selective colleges, but it tests too narrow a range of talents. Moreover, those chosen few, blessed by the test, did not enter public service as Plato's Guardians would have but followed their own interests, particularly in the professions.
Does the case hold up? Ultimately, no, although popular culture certainly believes so. David Owen observed that "a low grade in a course is just one teacher's judgment, but a low SAT score is a brand for the ages. People who forget their shoe size don't forget what they got on the SAT" (I remember mine from 42 years ago). Some friendships dissolve on the discovery of disparate scores. And a recent Cincinnati Post cartoon depicted a mother reading to her child in bed: "And the little pig with the higher math and verbal lived happily ever after. The other two were swallowed by the wolf."
The thesis of the super-powerful SAT fails, though, just where it is held to be omnipotent: in the admissions offices of selective colleges. Consider admissions at Brown University, one of the most selective in the country. In 1998, Brown could have admitted all of its freshman class from applicants with SAT verbal scores above 750 (the 99th percentile) and still have had enough applicants left over to fill another whole class. The range of verbal scores that Brown actually admitted, though, was 450 points, as was the range of those who actually showed up, from 350 to 800 (for math the range was 450-800).
Brown admissions officers were engaged in a round of what Peter Moll called Playing the Private College Admissions Game. Dean of admissions at Vassar when he wrote his book, Moll contended that private schools admit by category. Yes, they want brains. But they want the well-rounded all-American kid, the special talent, the legacies (children of alumni) and kids with a social conscience (perhaps soon to be an endangered species). They also want, I would add, the paying guest -- students who can attend without financial aid.
Colleges play the admissions game because of the vagaries of the SAT itself, something Lemann does not consider. Students' scores vary if they take the test more than once. ETS would not reveal the current score difference at which its computer suspects cheating, but David Owen put the number at 250 points (for the total). Moreover, the SAT succeeds only modestly even at its stated goal, predicting freshman grades. Correlations between the SAT and those grades generally run around .45, meaning only 20 percent (the square of the correlation) of the grades is accounted for by the test. Fully 80 percent of grades are determined by other facts.
By making the SAT the "all-powerful bringer of individual destiny," Lemann makes life itself an afterthought: "Their [the meritocracy makers'] goal was to construct a competitive race that would begin in elementary school and be substantially over by the time one graduated from college or professional school. . . . Those who like to think of American life as a great race should think of the race as beginning, not ending, when school has been completed."
As if winning cases didn't count in law firms. As if research and publications didn't count in academia. As if the most ham-fisted person could become a neurosurgeon. As if landing the big account didn't matter in corporations. In an Atlantic article on the SAT, Lemann wrote, "Today the academically selected elite does not control America to nearly the extent that is commonly assumed. Business, either corporate or entrepreneurial, isn't really its territory, and neither is career government service or elective politics." To write this and then give the SAT destiny-determining powers in his book is dishonest.
But if the Mandarins are missing from large segments of American society (and the presence of poor-boy-to-president Bill Clinton suggests that in fact they're not), how can Lemann claim that the SAT determines "the structure of success in America"? And surely some of those in corporate headquarters had high SAT scores. And if that is true, then personal characteristics, not test scores, determine where people end up. Finally, in this line of thought, Alan Bakke, whose reverse discrimination suit against the University of California plays a pivotal role in the book, was fully 38 years old when he decided to stop being an engineer and go to medical school. If the race had been over, Bakke couldn't have gotten back on the track. Bakke is exceptional only because of his suit. Europeans are stunned by what they call the "second chance" quality of American education.
In fact, if we limit ourselves to the crass criterion of money, most research finds that going to a selective college matters some, but not a lot. The most recent, best controlled research indicates it doesn't matter at all. Researchers examined the incomes of people who were admitted to selective colleges and went, and people who were admitted but attended less selective colleges. For large samples who entered college in 1951, 1976 and 1989, there were no differences in income for the two groups.
Having said that the SAT has not created the dire problems Lemann thinks it has, let me say that it has created some problems. It creates problems when SAT-prep classes pass as high school English courses. It creates problems because poor students and minorities do not do as well as middle-class and affluent students, something ETS is apparently trying to remedy in part with its new, sure to be controversial "Strivers" program. This program is designed to reward disadvantaged students who do better than "predicted" on the SAT.
What few seem to realize is the SAT is not needed by the highly selective colleges that are supposed to make most use of it (the overwhelming majority of colleges in this country are not very selective). Some years ago, Bates and Bowdoin Colleges made the SAT an optional part of the application package but required it for students after admission -- for placement and guidance purposes. Students who did not submit SAT scores with their applications scored about 150 total points lower than those who did, but they made just as good grades and did not drop out more because of academic problems. In addition, the geographic and ethnic makeup of the schools increased as did the variety of intended majors. The faculty was happier with the class composition. There is a message in this, I think.
It is clear that Lemann has not given a lot of thought to the questions raised by his proposals or to alternatives to the SAT. Nor has he actually looked at a lot of test-related data. The Big Test presents a fascinating history of the development of the SAT and ETS. But its claims for the consequences of this history do not hold up when measured against the facts.
Gerald W. Bracey is a former director of research, evaluation and testing for the state of Virginia. His most recent book is "Put to the Test: An Educator's and Consumer's Guide to Standardized Testing."