washingtonpost.com  > Opinion > Editorials
Editorial

Making the Grade

Tuesday, August 10, 2004; Page A18

"BE A CONFORMIST." So advises one leading test prep company on how to beat the analytical writing assessment for the graduate business school entrance exam, or GMAT. Lacking originality might land any high school or college student a C-plus, but in the new world of the computer-graded essay, conformity will win the top prize. As The Post's Jay Mathews reported, the GMAT has pioneered the use of computer programs to grade essays in high-stakes standardized testing, and it is being closely watched by other makers of standardized tests. The computer program works by comparing a submitted essay to a database of other already-scored essays on the same topic. The more similar it is to a high-scored essay on the same topic, the better the score.

But are computers capable of valuing a cogent argument, a particularly riveting example or an exhilarating buildup to a conclusion? Hardly. The computer can't understand or appreciate an essay the way any human reader can, so it dissects the essay down to elements that supposedly are predictive of human scoring. When analyzing an essay, a computer might give high value to the number of subjunctive modal auxiliary verbs, and regard the transition words "therefore," "in contrast," "in conclusion" as signs of organized thinking. Pity, therefore, the test-taker who did not use more infinitive and subordinate clauses that would have proved the superior structure of this essay, according to the computer. The failure of the submitted essay to look like any of the essays downloaded into this system just might earn a failing grade.

_____Today's Post Editorials_____

_____What's Your Opinion?_____
Message Boards Share Your Views About Editorials and Opinion Pieces on Our Message Boards
About Message Boards
_____Free E-mail Newsletters_____
• News Headlines
• News Alert

Not so, say the computer program's developers at Educational Testing Service, who patiently point out that every essay graded by a computer is also read at least once by a real, live human being, and twice if there's a need to arbitrate any arguments between the human and the computer. But don't worry about that happening much, croons ETS, because the GMAT's human readers agree with the computer's scoring 87 to 94 percent of the time.

But using computers to help grade the test merely underscores the idea that creativity and content are irrelevant, as shown when craftily written nonsense essays earned top marks as part of a study. Scoring high, then, becomes more about hewing to some statistically generated model essay whose cookie-cutter structure can be easily analyzed by a computer. And with some colleges already using the program to help make placement decisions for writing classes, and some schools using it to give students feedback on their essays, a question arises: Does an essay's value come from hewing strictly to a formula for writing it?

The computer program recommends that a conclusion contain at least three sentences. Why, if two will do?


© 2004 The Washington Post Company