Better data needed to accurately rate school systems

By Jay Mathews
Monday, July 19, 2010

Educational statistics expert Joseph Hawkins, one of my guides to the mysteries of test assessment, is impatient with the way the Montgomery County public school system is, as he puts it, "always telling the world how much better it is than everyone else." He finds flaws in its latest celebration of college success by county graduates, particularly minorities.

As a senior study director with the Rockville-based research firm Westat, Hawkins's critique has regional and national importance because it deals with the National Student Clearinghouse. This little-known information source might become the way school raters such as me decide which school families and taxpayers are getting their money's worth and which aren't.

The clearinghouse has a database of more than 93 million students at more than 3,300 colleges and universities. It originally specialized in verifying student enrollment for loan companies. Now it tells high schools how their alumni are doing.

Yeah, sure, Hawkins says, but "data from the clearinghouse is not completely accurate, especially if Social Security numbers for students are not obtained." Also, he says, some of the numbers Montgomery brags about don't look so good when compared with others.

"For example," Hawkins says, "MCPS reports that 26.7 percent of the African American graduates earned a degree in six years. Sounds okay, right? But according to the NCES [National Center for Educational Statistics, part of the U.S. Education Department], the college graduation rate for black kids is 42 percent. The graduation rate for white kids is 62 percent." Why, he asks, are Montgomery graduates doing worse?

Montgomery schools spokesman Dana Tofig works for Superintendent Jerry D. Weast, the most insatiable consumer and promoter of educational data in America, so Tofig is always ready for statistical queries. He says Hawkins is comparing "apples to pumpkins." The county is reporting the percentage of black high school graduates from the classes of 2001 to 2004 who got college degrees in six years. The NCES is reporting the percentage of black college freshmen (not including black high school grads who didn't go to college) who graduated in six years.

It is easy to stumble over these subtle distinctions, which is in part what Hawkins is saying. Montgomery has one of the best school systems in the country but has to be candid about the limits of the data supporting the valid argument that families are getting a good deal for the relatively high taxes they pay for these schools.

Washington area officeholders -- as well as real estate agents -- have been extolling the high test scores of our suburban schools for years. They rarely add that communities with such high average incomes nearly always have high scores. It is an iron rule of educational testing, Hawkins's home ground. We tend to assume a restaurant is good because the customers are well-dressed. In the same way, we think a school is good because the parents are affluent. But that is not always the case for all students.

If the National Student Clearinghouse can improve its data-gathering, we would have more indicators of which districts best prepare for college the disadvantaged students Hawkins and I often discuss.

Hawkins once worked for the Montgomery school system and applauds its many information-gathering improvements. "I'm glad that MCPS is investing in this data," he says. But he wants to see more of it, especially comparing the college success of its minority and low-income students and of those from other places.

If you think this overlooks kids who don't go to college, ask what skills good employers and trade schools want in their high school graduate applicants -- pretty much the same facility with numbers and words you need to get into a state university. I hope Montgomery and Hawkins will pursue their frank and friendly discussion to get us even better information on this.

For more Jay, go to

© 2010 The Washington Post Company