Sunday, November 22, 2009
MAKING THE GRADES
My Misadventures in the Standardized Testing Industry
By Todd S. Farley
PoliPoint. 253 pp. Paperback, $16.95
As college application season hits a fever pitch and standardized tests become the fixation of high school seniors and admissions boards everywhere, Todd Farley's memoir, "Making the Grades," argues for taking the results of these and other ballyhooed exams with a heavy dose of skepticism.
Farley spent almost 15 years working in the standardized testing industry for grades K-12, starting as an entry-level scorer and eventually becoming a test writer and scoring trainer who lived high on his expense account. His experiences led him to conclude that these tests are "less a precise tool to assess students' exact abilities than just a lucrative means to make indefinite and indistinct generalizations about them."
Throughout his career, grade manipulation was the norm. He and other leaders would change scores or toss some out in order to achieve "reliability," a measure of how frequently different readers scored a question the same way. Among scorers, he writes, "the questions were never about what a student response might have deserved; the questions were only about what score to give to ensure statistical agreement." Once, he and his fellow scorers changed standards midway through the scoring process when a representative from a state department of education objected to the large number of mid-level scores.
Ethics weren't the only problem. The guidelines used to assess the responses were often vague, and answers were awarded points based on absurd criteria. Farley's account is often downright funny, particularly in one case where students had to list their favorite food and describe its taste. The scorers hotly debated whether grass is a food, if pizza can be considered salty, and whether a food can be both sweet and bitter.
The book would have packed a greater punch if it had incorporated the experiences of others in the industry, allowing readers to know whether Farley's career was the exception or the rule. Still, his anecdotes add up to sharp criticism of an industry deeply entwined in our education system.
-- Sarah Halzack