School Rating Scoundrels Club

By Jay Mathews
Washington Post Staff Writer
Tuesday, August 1, 2006; 11:58 AM

From the very beginning I knew that publishing a ranked list of public high schools, based on participation in college-level tests, would get me into trouble. I wasn't breaking any laws and I figured my critics, for the most part, would be very polite, since that is the way people interested in education usually are. But I knew I wasn't going to do my reputation any good. Here is the first paragraph of my introduction to my first high school list, the 1998 Challenge Index at the end of my book "Class Struggle":

"Nearly every professional educator will tell you that ranking schools is counterproductive, unscientific, hurtful, and wrong. Every likely criterion you might use in such an evaluation is going to be narrow and distorted. A school that stumbles one year may be fine the next."

And yet, I said, as a reporter and as a parent, I thought in some circumstances such ratings could be useful. In last week's column , high school social studies teacher Mark Crockett and I argued about this. Like many people, Crockett does not think that rating schools by Advanced Placement or International Baccalaureate tests in Newsweek and the Washington Post makes much sense. That discussion will continue, if the continual flood of e-mails questioning my sanity is any indication.

But it is important to note, I think, that I am no longer the only person doing this kind of comparative high school assessment. There are enough of us to form what we might call the School Rating Scoundrels Club. Fortunately, many of the new members are a lot smarter than I am.

They include Nicholas Colangelo, director of the Belin-Blank Center for Gifted Education and Talent Development at the University of Iowa; Mike Reno, a trustee of the Rochester (Mich.) Community Schools, and Tim Norris, personnel director of the Mountain Brook (Ala.) schools. There are also the several publications, including the Seattle Times, Westchester Magazine, Washingtonian magazine and others, that have used approximations of the Challenge Index in their own high school lists.

Colangelo has just published his second annual Iowa AP Index, to date the most ambitious effort I know of to apply the Challenge Index rating system to all high schools, public and private, in an entire state. His center's report can be found at

Colangelo said he thought paying close attention to each school's AP DATA would be a good way to encourage Iowa schools to be more challenging. Only four Iowa public schools qualified for the latest Newsweek list of the country's most challenging public high schools. Colangelo discovered that of the 389 public and private high schools in Iowa, only 213 had at least one student take an AP exam in 2005. Of that group, 187 schools--171 public and 16 private--consented to participate in the Iowa AP Index. "The top 25 schools range in class size from 11 to 378," said the report, co-authored by Colangelo, Susan Assouline, Damien Ihrig and Clar Baldus. "There are 20 public and 5 private schools in the top 25. The #1 school is Rivermont Collegiate High School, a small private school in Bettendorf. The biggest school (based on graduating seniors) in the top 25 is Iowa City High School in Iowa City, Iowa [378 seniors]. The smallest school is Russell High School in Russell, Iowa [11 seniors]." The University of Iowa researchers even found an Iowa school, Roosevelt High in Des Moines, that qualified for the Newsweek list but that I had missed, a mistake I plan to rectify soon.

Mike Reno, the school board trustee in Rochester, Mich., embraces Colangelo's view that more attention should be paid to individual school AP data. (Neither of them has tried to look at IB data yet, but I suspect that will come.) Reno said he suspected AP participation in his wealthy county was poor, despite claims from the school administration that it was tops in the state. So he started collecting his own data, resulting in a remarkable 41-page report, "Advanced Placement Participation in Michigan," available at his website,

Reno ranked 94 Michigan school districts by AP test participation. He discovered that Rochester was only number 24. Among 51 states and the District [see the list of states near the top of this column], Michigan ranked only 29th. The ratings, Reno concluded, were influenced as much by the emphasis educators and state officials placed on challenging courses as they were by parental income. "Many states demonstrate their commitment to AP in a very public way," he said, "such as tracking and publishing AP statistics on their websites and offering financial incentives to teachers and schools."

He concluded that "many Michigan children are at a disadvantage due to the concentration of AP participation in a relatively few number of schools. Over two thirds of AP exams are taken in districts representing just one-third of Michigan students. One-third of AP exams are taken in schools representing just seven percent of Michigan high school students." Reno invented his own measuring device, the Balanced Achievement Indicator, to show which schools are both bringing their students up to the minimum achievement levels required by the state and providing the higher academic challenge of AP.

At Mountain Brook, Norris is exploring the connection between AP participation and average scores on the SAT and ACT tests. He has gathered data on individual schools in many other states that share Mountain Brook High School's high performance levels.

The number of school rating scoundrels is growing. They probe and analyze and try to share what they find. I think they deserve a bigger audience. Nothing convinced me of this more than a letter from a very astute judge of high schools, educational consultant Jan Rooker, that was sent recently to the principal of New Canaan (Conn.) High School and the New Canaan school board. She was reacting to a story in the New Canaan News-Review quoting principal Tony Pavia as being skeptical of all such high school lists. He acknowledged he had not looked at the Newsweek list, which ranks New Canaan High at 911, in the top 4 percent of all U.S. schools but perhaps lower than what that very affluent community might expect.

CONTINUED     1        >

© 2006 The Washington Post Company