Why I Changed the Challenge Index

By Jay Mathews
Thursday, December 11, 2008; 9:45 AM

The minute I saw that Coolidge High School in the District had given a startling 750 Advanced Placement tests last May, and that only 2 percent of those exams had received passing scores, I knew I was in trouble.

For 10 years I have been ranking high schools based on participation in AP, International Baccalaureate and other college-level exams. I call this the Challenge Index. It is the system used by Newsweek in its annual list of top high schools and by The Washington Post in its annual ratings of all Washington area schools, published today in The Post Extra sections and on washingtonpost.com.

Every year I receive thousands of e-mails about these lists, and my refusal to include test scores in the ranking calculations. Some readers praise me for recognizing schools that work hard to prepare students from poor families for college-level courses and tests, even if their scores aren't good. Others denounce me for giving high ratings to schools full of such students, because many people think low scores should disqualify a school from appearing on anybody's best schools list.

I like the nice messages, but the ones from people trying to straighten me out are more fun to answer. I tell them they are right. Average test scores are one valid measure of a school. But those averages are so influenced by the family backgrounds of the students that if you ranked schools by average parental income instead of test scores you would get pretty much the same result.

The Challenge Index gives low-income neighborhood schools with energetic and insightful teachers and administrators a chance to shine by showing their strenuous efforts to raise the academic level of their students, rather than bask in the glory of having classrooms full of doctors', lawyers' and accountants' kids.

I have a drawer full of e-mails from parents, students, educators and pundits saying I am right about this, but the Coolidge High statistics that appeared on my computer screen the day before Thanksgiving were a severe test of that point of view. Few public high schools have academic reputations worse than Coolidge's. A year ago my colleague Lonnae O'Neal Parker described the school in great detail as a chaotic collection of distracted teachers and inattentive students. Yet, six months after Parker's stories appeared, Newsweek published my 2008 Challenge Index list, on which Coolidge, despite its flaws, had a rating of 1.038. That meant it ranked among the top 5 percent of all U.S. public high schools measured this way.

I did some quick arithmetic when the new Coolidge data came in. I divided the number of tests, 750, by the number of graduating seniors, 137, to get the rating. It had jumped from 1.038 to 5.474. That would make it No. 1 in the Washington area. It would be ahead of perennial No. 1 H-B Woodlawn in Arlington, which this year had 399 AP tests and 76 graduates for a rating of 5.250.

As I said, the Challenge Index measures test participation, not test scores. The idea is to see which schools are getting as many students as possible to try at least one college-level course and test. Coolidge's participation rate was close to Woodlawn's. But their passing rates -- the portion of tests with grades of 3, 4 or 5 on the five-point AP exams -- were in different universes. Coolidge's passing rate was 2 percent. Woodlawn's was 59 percent, almost 30 times greater.

Coolidge Principal L. Nelson Burton, like a growing number of principals in impoverished neighborhoods, has decided that one way to rescue his students from the low standards that Parker described so well is to entice as many of them as possible into AP. Like IB, AP exams are written and graded by outside experts. Neither Burton nor anyone else in his school, or his school district, has the power to dumb down those exams. This is very different from what happens with final exams in regular courses, which are often made easier than they ought to be. Schools cannot afford to give out too many failing grades, no matter how poorly their students are doing. Parents would complain to the principal and the school board. Teachers who try to keep their classes, and their final exams, at a high standard are forced to compromise.

Teachers in AP and IB classes still give their students whatever class grades they like; the college-level test results do not arrive until summer, long after report cards have been handed out. But those AP and IB results are becoming public documents, reported on state government Web sites, so anyone can see (as long as all or nearly all AP and IB students take the exams) how well a school is doing when measured against an incorruptible high standard.

Burton and principals like him know they are going to see low scores in the early years of their AP programs. They accept that because they believe their students, teachers and communities should feel the icy blast of real college standards. Their students, they argue, need to struggle with long reading lists. They need to sweat through three-hour exams full of essay questions. They need to get a sense of what college is like that they cannot realistically get any other way. Those school administrators want their AP students to go one-on-one against the academic equivalent of pro basketball star LeBron James. Their kids will lose that game, they know, but gain a visceral appreciation of how much work they have to do to rise to that level.

I admire Burton and those other principals for exposing so many of their students to AP and IB, and some lesser-known but similar programs such as the Cambridge tests. If there were a less demanding but still well-structured and tamper-proof system like AP, IB or Cambridge that could be used to build their students up without so many failed exams, that might be a viable alternative. But there is no such option. The most frequently mentioned substitutes, such as state tests in Maryland or college admission tests such as the SAT and the ACT, have almost all multiple-choice questions. They provide little incentive for developing the critical thinking and writing skills that AP and IB tests demand, and that college students need.

CONTINUED     1           >

© 2008 The Washington Post Company