The Answer Sheet: What Does the SAT Test?

Monday, September 14, 2009

Here are recent excerpts from The Answer Sheet by Valerie Strauss, The Post's new education blog.

Edward Carroll takes tests for a living. To be precise, he takes the SAT, the dreaded college admissions test. Over and over.

The 38-year-old has taken every SAT made available over the past decade by its owner, the nonprofit College Board. Usually he takes it alongside high-schoolers on Saturday mornings.

He has also analyzed another decade's worth of tests.

"People think I'm crazy," he said, but his job depends on his diligence: He is a standardized test expert and tutor at the Princeton Review, an education and test-prep company.

That means he is also an expert on the ACT, the other major college entrance exam, owned by the nonprofit education and workforce-development organization ACT. Most schools use these exams to inform their admissions decisions, and just about all of them that do accept the results of the SAT or the ACT.

For those high school students now starting the grueling admissions process, Carroll offers the following from conversations with The Answer Sheet on e-mail and on the phone:

Q. What does the SAT actually test? What does it say about someone who takes it?

The SAT, more than anything else, shows how well you take the SAT. It is NOT a measure of a student's raw math or verbal ability. The College Board itself does not claim that the SAT predicts subject skills, but rather that it is a predictor of performance in college (along with the rest of a student's application).

Personally, I think it also filters out students who can't perform quickly. The test is rigidly and tightly timed. It is very, very difficult to finish each section and the [College Board] knows it. They design it that way so that they can assure a nice range of scores to the colleges for comparison.

So it doesn't tell much of anything important?

It is a very flawed test if you expect it to reveal much about student content skill or personal study and performance ability.

Who does well on it?

The SAT puts students in a pressurized environment, and students who perform well in testing situations will excel. Everyone knows a story of a slacker student who doesn't work hard in school and does well on the SAT. . . . On the SAT, if you're brilliant and slow, you'll get a very average score.

Why is it so difficult?

It's not, really. The questions are not straightforward.


Each test is equally tricky. The simplest example of that is that it often asks "What is x + 1?" when students have to do lots of calculations to solve for x. Then, as they do throughout math class in school, they choose the value of x that they found. To put it another way, they do everything correctly in solving, then circle the wrong answer (because the trap answers will always be there)! Thus, they get the question wrong, but NOT because they didn't know what they were doing.

Is the ACT more indicative of a student's ability than the SAT?

In short, yes. It tests what students learn better than the SAT. It has its own flaws, but what it purports to do it does better than the SAT.

What are the flaws?

It is hard to finish. . . . It has to have students fall into a predictable range. It is standardized. . .

And you always have to be suspicious of the easy answer no matter what test you are taking. These tests are multiple-choice. They have to have the answer on the page, so they make the other answers as attractive as possible but specifically wrong. Students have to eliminate an obviously wrong answer and get it down to 3 or 2.

Any truth to the notion that one SAT given each year is harder than the rest?

The SAT does not change from test to test. It is a myth that there is a better day or month in which to take the test. This is a standardized test. It does not change, and the scores from one test are equivalent to scores on another test. Despite our criticisms of the SAT over the years, the College Board is very good at one thing -- making its tests the same every time. It's nonsense to think otherwise, but this myth persists.

The addition of a written essay in the SAT test in 2005 didn't change things?

The "old" one changed in March 2005. In a nutshell, it didn't change very much. They took their old SAT Writing test and tacked it onto the SAT -- that's how it got an essay. The rest of it (slight math changes and elimination of Analogies) had very little effect on the overall student experience test, despite what you may have heard. The biggest effect on students is that it is now longer -- 3 hours 45 minutes -- much more than that if you include [administrative] and break times.

Why would someone take the tricky SAT if the ACT is more straightforward?

The SAT does work for some students. . . . Some of the material on the ACT is at a higher level than the SAT. . . . I can tell SAT kids, you will never see a question with really advanced geometry.

Let's talk about test prep.

In general, our approach to the SAT is that we analyze it for unintended patterns, then tell students what they are. We don't pretend to teach more than that. This is one of the reasons that [the Princeton Review] has received criticism in the past. But we do raise scores.

Some people say test prep can only raise scores a little. Others say it can help a lot. What do you think?

It can help a lot. It depends on the point at which you start.

You've taken scores of tests. How do you do on them?

I do pretty well, but I'm human, so I can still make a mistake. . . . I can tell you this. I have gotten a perfect score in each section but not on the same test.

The kids must wonder about you when you take the SAT.

The high school kids think . . . I'm a little slow and I'm just getting to college.

Do you finish the SAT in the allotted time?

I can, but I have a much better vocabulary than your typical 16-year-old. . . . Your average adult who has gone through high school will do pretty well on the verbal section and will do less than they think they will in math, because the Pythagorean theorem doesn't come up much in their daily lives.

Why do kids always say, "Nothing," when parents ask them what happened in school?

Not all of them do. Some are chatty and tell their parents every last detail about who did what to whom at school.

But it is true most would rather not. A survey by a British government agency on this very issue was released this year. The conclusions, as reported by the BBC:

-- 82 percent of parents wished they had more information about their children's school life.

-- 16 percent of children volunteered information about their day at school.

-- Nearly 25 percent of children felt like their parents "were hassling" them by asking about school.

An informal survey conducted by The Answer Sheet -- in which several dozen children from elementary to high school were questioned -- showed that most of the kids don't want to talk about school for the same reason:

After eight hours in school and more time doing homework, they are sick of school and don't want to talk about it. You are probably asking your kids about school at dinner or even later. They are tired, and their focus is away from schoolwork. They don't want to go back there until they have to.

My friend Liz's son, when he was 4, said to her after being asked every day for more than a year about his day in nursery school: "Why do you keep asking me about school? I'm never going to tell you." He eventually did, as a teenager, after she stopped asking him insistently.

It is true that there are times when my own children don't want to talk to me about anything -- choosing to spend their free time talking to their friends or listening to music.

But I do find out something about their day by asking extremely specific questions -- "Tell me one thing that happened in English class," for example -- and insisting that I get answers.

It works. Sometimes.

© 2009 The Washington Post Company