washingtonpost.com  > Education > K to 12

Mining Scores for Nuances in Improvement

Though It Has Critics, New Tool Catches On Among Schools, States

By Jay Mathews
Washington Post Staff Writer
Tuesday, December 14, 2004; Page A12

Most days, whenever he has a free moment, Mark Twain Elementary School Principal Scott Ebbrecht can be found peering at the screen of the gray computer on his desk, trying to see exactly how well each student at his Westerville, Ohio, school is doing.

He is using something called value-added assessment, the hottest new tool in the national effort to improve public schools. At 485-student Mark Twain Elementary, a one-story modern brick building on East Walnut Street, the method has already brought results. Ebbrecht recently discovered that his top students, despite their high scores, were not improving as much as the value-added equations predicted, and he quickly made changes.

Mark Twain Elementary Principal Scott Ebbrecht discusses the school's value-added assessment software with third-grade teachers Angie Robinson, left, and Kim Glaser and the school district's coordinator of gifted education, Lisa Huelskamp, in Westerville, Ohio. (Will Shilling For The Washington Post)

Value-added, which uses test scores to compare each child's progress to predictions based on past performance, has its critics. Some experts say that the tests are too narrow and that the analysis ignores differences in subject matter. But 16 state school chiefs have asked U.S. Education Secretary Roderick R. Paige to let them use the system or similar measures to meet federal requirements, and districts in at least 35 states have shown interest in using it.

The data on Ebbrecht's screen are very technical, such as a line showing that his five lowest-performing third-graders are 11.3 points ahead of their predicted progress in reading, with a standard error of 5.6 points. But it works for him. "Without data, you're just another person with an opinion," he said.

Educational researchers have been experimenting with measuring each child's academic progress for several decades. The best-known recent research was done by William Sanders, a former University of Tennessee researcher now working for a private company in North Carolina. Ebbrecht had Sanders speak to a group of principals during one of his trips to Ohio.

Value-added assessment has also become a political irritant because some school boards and superintendents want to pay teachers based on how much value they are adding, as measured by individual student test scores, for students in their classes. In Ohio and most other states, the system is being used only to diagnose student needs, leaving the question of teacher pay for later.

"We use it to improve instruction, not to evaluate teachers," Ebbrecht said. Among his teachers, however, its potential for affecting salaries "is a big fear," he said.

Ebbrecht gets his data from the Web site of Battelle for Kids, a nonprofit group in nearby Columbus that is supporting a value-added pilot project for 200,000 Ohio third- through eighth-graders in 718 schools. The reaction has been positive enough to persuade state officials to make the method part of the assessment process for all Ohio schools by the 2007-08 school year.

"This puts positive pressure on the system and everyone in it," said Jim Mahoney, a former Ohio school superintendent who is Battelle for Kids's executive director.

Tom Luce, chairman of the National Center for Educational Accountability in Texas, applauded that development. "The ability to track students over time is absolutely essential," he said.

Other states working to include value-added or other growth data in their tracking systems include Arizona, Florida, North Carolina, Pennsylvania and Tennessee, said Lynn Olson, a senior editor at the Bethesda-based newspaper Education Week. Arkansas, California, Colorado, Louisiana and Minnesota are considering the assessment system.

Some districts in Maryland and Virginia are experimenting with value-added assessment, but neither state nor any D.C. school has adopted it.

The growth of value-added assessment has been accompanied by scholarly debate over its usefulness and validity. George Mason University educational psychologist Gerald W. Bracey, summarizing several studies in his research column for the December issue of the magazine Phi Delta Kappan, said value-added assessment "rests on what appears to me to be an untested hypothesis: good teachers raise test scores. Given the hysteria about test scores created by the high-stakes testing juggernaut . . . it is easy to see how that hypothesis might be mistakenly taken for an assumption."

David H. Monk, dean of the College of Education at Penn State University, said that as a former inner-city teacher, he is intrigued by the method's potential but has some doubts.

The value-added model "is entirely dependent on test results and can be only as good as the tests, which can miss important outcomes," Monk said. "The model is also retrospective and reveals more about where past successes occurred than about what needs to be done."

Many educators working with the assessment method said they appreciate its limits but consider it better than the way test scores are used now. Under the federal No Child Left Behind Act, schools are judged not by individual student progress, but by the average score attained by all students, and subgroups of students, at each school, compared with the average scores of different groups and subgroups of students the previous year.

Brad Jupp, a teacher-coordinator for a new teacher evaluation system in Denver that will use some value-added concepts, said he believes the new assessment method will spread because it provides "better measures of what schools and teachers do than simple performance measures."

Karen B. Wolf, principal of Wickliffe Elementary School in Wickliffe, Ohio, said she has found the value-added system useful in keeping effective teachers from becoming discouraged when they don't quite reach the state and federal test-score targets.

"You don't have control year to year over what kinds of students you are getting in your classroom," Wolf said. When her teachers saw in the value-added data that they were making significant progress with even low-performing children, they stuck with their methods and eventually reached all their targets, as well as won a statewide award for the school.

At Mark Twain Elementary, Ebbrecht met with his third-grade teachers last week to discuss enriched readings and other challenging assignments that he thinks will bring more progress for the high-achieving students that needed help.

With value-added, he said, he was able to ask and answer a vital question: "They are performing at an advanced level, but are they really growing as much as they should be?"

© 2004 The Washington Post Company


Business Schools

  •  Colleges and Universities

  •  Continuing Education & Professional Development

  •  Distance Learning

  •  Graduate Schools

  •  Law Schools

  •  Medical & Nursing Programs

  •  Summer Schools

  •  Technology Training