By Jay Mathews
Washington Post Staff Writer
Monday, May 28, 2007
One in an occasional series on the culture of testing
Blank blue computer screens frustrated thousands of Virginia students this month during online state exams in a series of disruptions that underscored vulnerabilities in the educational testing industry. Such episodes, experts said, could prompt changes in how the nation's schools assess student performance.
Test software malfunctions in several states, coupled with staff shortages and cutthroat competition in the industry, have fueled a growing debate over whether to cut the number of tests taken under the federal No Child Left Behind law or adjust the testing calendar.
"The system has had a lot of pressure put on it," said Adam J. Newman, a managing vice president of the market research group Eduventures Inc. in Boston.
"The companies are really struggling," said Thomas Toch, co-director of the D.C. think tank Education Sector.
Under the five-year-old federal law, public schools must test all students in reading and math in third through eighth grades and once in high school. The law also calls for more science tests. The massive expansion of required testing fell onto the relatively small testing industry like a log landing on a twig.
An industry shakeout is underway amid fierce competition for state contracts. Several top testing company executives have been fired. Pearson Educational Measurement, blamed for the Virginia test disruptions, has acquired another major testing company, Harcourt Assessment Inc.
Fairfax County school officials complained in vivid terms about what they saw as sloppy work by Pearson, now the nation's biggest testing company. In a May 15 letter to State Superintendent of Public Instruction Billy K. Cannaday Jr., Fairfax schools Superintendent Jack D. Dale said that paper tests prepared by Pearson arrived several days late in loose, unsorted cartons and that calls to Pearson about online testing disruptions often went unanswered.
"We very much regret the disruption" in the Virginia Standards of Learning testing, Pearson spokesman David Hakensen said. "The irony is that Virginia and Pearson have been pioneers in developing the online model for the effective, accurate and scalable delivery of student testing."
Virginia began online testing in 2001. This year, the state estimated it would give about 1.2 million online SOL tests.
Failures in mid-May interrupted testing at schools in Loudoun, Fairfax, Prince William and Fauquier counties and elsewhere in the state. Some students who were halfway through or nearly finished had to start over with a new test another day.
Newman said new software often fails when demand surges, much as shopping Web sites often crashed in the early years of that online industry.
Testing experts said congressional leaders are discussing whether to encourage states to give tests earlier in the academic year to avoid a springtime overload, especially in May. (Many Maryland schools test in March; D.C. schools, in April.) They also said some lawmakers are considering whether to ease the midsummer deadline for results.
A spokesman for Rep. George Miller (D-Calif.), a leading sponsor of No Child Left Behind in 2001, said Miller and his staff had no comment. Other congressional sources said nearly every idea is on the table, including setting multiple testing periods in the school year and letting states pick the period they want to use.
But further complications could arise if Congress takes steps to encourage a new model of rating schools. That model would track individual students' progress from year to year. Currently, results for students at a given grade level are compared with results from students who were in that grade the previous year.
Another industry challenge: finding enough test-design experts, known as psychometricians, to create tests for each state. Companies frequently raid one another for talent.
"We are seeing psychometrican musical chairs," said Bob Schaeffer, public education director of the National Center for Fair and Open Testing in Massachusetts.
In a 2006 report on the industry, Toch wrote: "Psychometrics is a highly technical, mathematics-based discipline that doesn't pay particularly well by private-sector standards (about $120,000 a year in top industry slots and much less in state testing agencies)." He said education majors are discouraged from entering the field by their professors, "many of whom are opposed to the rise of standardized testing in public education."
Critics of the federal law point to technical difficulties as a reason to scale back testing mandates. Sue Allison of Marylanders Against High-Stakes Testing said reports of testing breakdowns across the country are numerous. Mickey VanDerwerker of Parents Across Virginia United to Reform SOLs said one way to reduce testing would be to sample students to gauge progress in a school or school system.
"You don't need to test every kid every year in every grade," Schaeffer said. "Testing is not going to go away, but you can do it much less frequently and get the information that you need for accountability."
But supporters of the federal law say universal testing is inherent in the goal of leaving no child behind.
Testing company executives say their industry is up to the challenge.
Douglas Kubach, president and chief executive of Pearson, said his company is working to build capacity for the mounting test load. "Over the past five years, Pearson has invested more than $100 million in research and development, innovation, quality and process improvement in our U.S. assessment and testing business," he said through spokesman Hakensen. "We have added capacity, expanding our scoring operations at 30 facilities in 15 states."
John Oswald, senior vice president for elementary and secondary education at Educational Testing Service, said: "We do not see any evidence of too heavy a burden on the testing companies." Mostafa Mehrabani, president of McGraw-Hill Assessment and Reporting, said: "As the industry continues to move toward digital delivery, there will be a number of benefits and efficiencies."
Staff writer Michael Alison Chandler contributed to this report.