Putting Assessments to the Test

· Content validity deals with, not surprisingly, content. A key component, curricular validity, demands that a test actually cover material in the curriculum (especially important in high school graduation tests.)

· Criterion-related validity includes predictive validity. Gerald Bracey, an educational researcher and author of "Reading Educational Research: How to Avoid Getting Statistically Snookered," said that he does not know of any state that has tried to validate its tests against what happens in the future.

· Construct validity deals with the broad picture of whether a test assesses exactly what it is intended to measure; a science test trying to measure knowledge of geologic time might have questions that are so difficult to understand that what really is being measured is vocabulary and reading skills.

Another form of validity, identified in the 1990s, is "consequential validity," which says that a test's validity is determined by how the results are used. It has the testing world in a verbal brawl because some experts think it is essentially nonsense.

"You can have a good test of, say, mathematics, and have school boards make ridiculous policy decisions based on the scores," Bracey said. "To me, that says nothing about the test."

Complicating matters, educators say, is the fact that the pipeline of newly trained testing experts charged with improving standardized tests is nowhere close to keeping pace with the skyrocketing demand.

Training started falling 25 years ago, and there has been no big resurgence. And the capacity of the commercial sector to produce the vastly increased number of tests has significantly lagged, experts say.

Roger Farr is director of the Center for Innovation and Assessment at Indiana University, a special consultant on testing and assessment to the education company Harcourt, and an author several standardized tests. He said he thinks the country is placing too much emphasis on test results.

"Teach children to read and write well and the . . . tests will take care of themselves," he said. "What we've got to do is know what to teach kids. The goal of education is not coming up with answers. The goal of education is how you find answers."