New Challenge Index: How to Score

Jay Mathews
Washington Post Staff Writer
Tuesday, December 13, 2005; 11:39 AM

This Thursday the Washington Post will publish its ninth annual Challenge Index rankings of all the public high schools in the Washington area. The list will also include the new Equity and Excellence rating invented by the College Board to judge, as the Challenge Index tries to do, which schools are doing the most to prepare their students for college. I wasn't sure how this new statistic would work, but it has impressed me by giving statistical weight to a small but important truth---there are great things happening at Annandale High School.

Annandale High is in Fairfax County. It is one of many public schools in the country whose reputation has suffered by an influx of low-income minority students, in Annandale's case, including many whose parents were not born in this country. Many Americans have a visceral negative reaction to schools with lots of poor kids who are not white or whose parents came from elsewhere. You may call it racism. You may call it ignorance. But it is rare to find such a school with a good reputation in its neighborhood, even when it has some of the best teachers and strongest courses imaginable.

Annandale, in my opinion, is one of those schools that is much better than its reputation. The Challenge Index, which I invented nine years ago to identify such schools, seems to show that. On the new list, Annandale has a rating of 1.329, which is the average number of college level tests it gave this year per graduating senior. That puts it in the top five percent of all U.S. public high schools measured this way.

But the Washington area has more than its share of great high schools, so despite Annandale's impressive national ranking, on the local list of 174 schools it is number 88. It doesn't stand out on the list in the way I think it should. Annandale International Baccalaureate coordinator Erin McVadon Albright is one of the most energetic and successful IB educators in the country. When she ran the IB program at George Mason High, more than 80 percent of students, a phenomenal number, took college level courses and tests. Albright then transferred from George Mason, which has few low-income students, to Annandale, where 39 percent of the student body qualifies for federal lunch subsidies and only 35 percent of the students are non-Hispanic whites. She began to build an IB program from scratch with students who had a lot of distractions in their lives.

I knew Albright was doing good work. The Equity and Excellence rating revealed just how good. The rating is designed to measure both the level of participation in Advanced Placement tests at each school, which is what the Challenge Index does, as well as how well its students are doing on the tests. It reports the percentage of all graduating seniors, including the many who never took an AP course or test, who have gotten at least one passing grade on one AP test sometime during their high school career.

On the Challenge Index list to be published in the Post's Extra sections, I put the Equity and Excellence percentage for each school next to the school's name. I can't use Equity and Excellence to rank schools yet because the College Board has not been able to calculate the rating for every school. Getting the right number of graduating seniors is still a problem for their statisticians, as it is for me, and I am gathering data on just 174 schools while they have 14,000 schools to worry about. But 72 percent of the local schools were able to give me their Equity and Excellence ratings, and some school districts, including Fairfax, calculated at my request Equity and Excellence ratings that included passing grades on both AP and IB, the two principal college-level programs in American high schools.

In May 2005, Annandale High gave 513 IB tests and 104 AP tests. That totaled 617 tests. I subtracted the three AP tests taken by Annandale students who had taken an IB test in the same subject, probably because they were afraid that some colleges that discriminate against the IB program would deny them college credit for their IB work unless they had an AP test grade, too. With those extra tests removed, Annandale's total was 614. I divided by 462 graduating seniors and arrived at a Challenge Index rating of 1.329. Fairfax County reported that 69 percent of the IB exams at Annandale received passing grades of 4 or above on the 7-point tests and 62 percent of the AP tests received passing grades of 3 or above on the 5-point AP tests. It said 38.1 percent of the graduating class of 2005 had at least one passing grade on an AP or IB test at some time during their time at Annandale High.

I put that Equity and Excellence rate of 38.1 percent (much higher than the national average of 14.1 percent) in parenthesis next to Annandale's name on the list, and checked out the ratings of the other schools.

I noticed something right away. Roughly speaking, the Challenge Index and Equity and Excellence ratings seemed to be measuring the same thing. The top ten schools on the Challenge Index included eight of the schools that would be in the top ten if I had ranked them by Equity and Excellence. The rest of the list followed that pattern, generally, but there were some exceptions, and Annandale was one of the most interesting. Most of the schools near Annandale on the list had Equity and Excellence rates in the teens or twenties, nowhere near 38.1 percent. I counted 44 schools ranked above Annandale on the Challenge Index that had Equity and Excellence ratings below 38.1.

So not only are Albright and the teachers at Annadale getting many more students involved in college level courses, but they are teaching them so well that they are doing better than even schools with their same high participation rates at finding ways for those students to master the material.

I have a theory about American teachers. I call it the hoops syndrome. Put any small disposable object in the hand of most people in this land that invented basketball, point them toward any open receptacle, and they will try to toss the object through the hoop. Teachers have the same irresistible impulse to help their students reach any distant but worthy goal. That is why I like AP and IB and the other high-level courses with independent assessments that are being introduced in our schools. Give a teacher a room full of students excited about hitting that target, and that teacher will try every possible way to make it happen.

At Annandale, Albright told me, everybody gets into the act---the IB teachers, the AP teachers and the teachers who have to prepare the lower grade students for those programs. They understand that a student who has never before had a challenging course may act bored or uninterested, when actually they are frightened. "Being apathetic and being terrified looks the same in an adolescent boy," she said.

Sometimes what would seem to be a simplistic concept in a high-income school has to be carefully explained at Annandale. "Take inference," Albright said. "Our kids don't necessarily come to school knowing how to do that." But if their teachers take the time to explain, many get it, and their Equity and Excellence rating goes up.

The new list makes me want to explore other issues: Why would two schools with almost identical student characteristics have very different participation and mastery rates? Why do some affluent districts wallow near the bottom of the list while some less wealthy districts soar? How can the experiences of the most successful schools be communicated to those that need help?

Take a look at the new list. Tell me what you find interesting. What I think it shows is that there are a great many teachers doing exceptional work, and that is something worth measuring.

© 2005 The Washington Post Company