The panel overlooks a few grammatical errors and misspelled words. "F. Scott Fitzgerald once handed in a manuscript with seven consecutive misspelled words," Bremen says. "If you can write like F. Scott Fitzgerald, you will be okay."
"We rewarded [the squirrel paper] because it was unique, and the student came up with it in 25 minutes," says Noreen Duncan, who teaches African American literature at Mercer County College in New Jersey. Some mistakes are permissible, she says, because anything that can be written in that time is, by definition, "a first draft."
The team uses a technique known as "holistic scoring," a euphemism for reading an essay very quickly (a minute or so per paper) and making a snap judgment. This is not like grading a school essay, in which points may be deducted for uncapitalized letters or an insufficient number of paragraphs. The scoring technique puts a premium on a student's ability to develop a logical chain of reasoning over the mechanics of writing.
Reading quickly boosts the productivity of the scorers, who are paid as much as $22 an hour. But that is not the main point, Vickers says. "If they labor over the essay too long, they become too analytical," she says. In the world of holistic scoring, "analytical" is bad. An overall first "impression" is good.
Most scorers end up within a point of each other on most essays. The discussions at the range-finding sessions are designed to establish guidelines for dealing with the difficult-to-categorize essays, many of which will probably be kicked up to a supervisor.
The big question is whether scorers trained to speed-read hundreds of essays will recognize exceptional writing when it comes across their computer screens. A mock scoring session conducted by the Princeton Review, a company that prepares students for the SAT and other high-stakes admissions tests, suggested that outstanding writers such as William Shakespeare would do poorly on the test because they would refuse to write according to the formula.
"Shakespeare would have done just fine on one of these tests," counters Vickers, without promising a 6 for the Bard.
Back at the sample-scoring session, meanwhile, an Illinois high school teacher named Bernard Phelan is keeping everyone entertained with his pointed comments. "This essay has the ring of empty assertion," he says of one effort, which ends up with a 2. "The student is telling us, 'I don't have an example, and I'm not about to provide one any time soon.' "
He awards another essay a 5. "Some kids write but don't think," he explains. "This kid thinks as she writes. There is some awkwardness here, but she moves fluently from topic to topic."
By 5 p.m., after eight hours of scoring essays, the eyes of the 13 panelists have begun to assume a glazed look. "The lighting in here is terrible," Vickers suddenly notices. "We need to get some proper lights in here tomorrow."
The panelists decide to tackle one final essay, which has received scores ranging from 2 (seriously limited) to 5 (reasonably consistent mastery). The essay is virtually illegible -- no marks are deducted for bad handwriting -- but it is two pages long and is sprinkled with academic-sounding words such as "commodity" and "value."
Ed Hardin, an expert with the College Board, makes a stab at reading the essay out loud. He had awarded it a 5 on the basis of his first impression and the sophisticated vocabulary but changes his mind as he tries to make sense of the stilted prose.
"Somebody is going to have to buy me a drink," he groans halfway through the reading.
It is getting late, and everybody is tired. The panelists agree on a 3 and call it a day.
