Insiders Report on the Challenge Index

By Jay Mathews
Washington Post Staff Writer
Monday, May 19, 2008 6:33 AM

This week, Newsweek magazine and its Web site Newsweek.com unveil this year's Top High Schools list, based on a rating system I invented a decade ago called the Challenge Index. The index ranks schools based on college-level course participation, adding up the number of Advanced Placement, International Baccalaureate and other college-level tests in a given year for a given school, and dividing that total by its number of graduating seniors.

Several weeks ago I asked students, teachers and parents to tell me how this annual ranking affected their schools. Here is a sampling of several points of view, both critical and complimentary.

* * *

So, with regard to your Challenge Index -- it really is a quick and dirty way of assessing schools. Very ambitious and probably very imperfect. However, there isn't anything else out there like it. I think the reason our school systems are not very good compared to other countries is that we underestimate the abilities of our children. I think too the education field is fuzzy -- not very good data or evidence to support the programs that are out there. . . . More and better research is needed. And of course there are the socioeconomic/family issues of some schools/districts that cannot/will not be fixed with just higher expectations.

-- Terry Adirim Montgomery County

I will give you two ways the index is affecting my school. First, the school system is unhappy with the results of our AP Calculus class. The basic problem is that the basic math skills are poor. This has been identified and confirmed within the system. So, our solution is to require all kids who are going to take AP Calculus to first take general calculus. I pointed out that the syllabus for both classes is essentially the same. So we will be offering Calculus Part 1 (general calculus) and Calculus Part 2 (AP Calculus) on a 4x4 block. The kids will get the same course twice.

The second area I've seen an impact is on scheduling. As I noted above we are on a 4X4 block. This past winter we were told that we were going to change the schedule to make education better for all. In every meeting all I heard about was how the new schedule would benefit the IB/AP courses. The problem with the index is that you manage what you measure. I want to see high achievement in our schools. I think we should do it by raising the bar of academic expectations for all children.

-- Gene Gartner Rappahannock High School (Va.)

When I first started teaching AP Psychology at Spanish River High School back in the early 1990s, I was told by the "experienced" vets that only one class of about 12 to 15 students would be feasible. I started with two classes of 25 each. Doom and gloom were predicted. We did pretty well that year so I expanded the program to three sections. Of course more doom and gloom and actual anger were directed at me. I was making a joke of the AP program. The kids would be hurt. How could I do such a thing? We did even better, so I expanded to five sections. We then hit 90 percent passing. The critics shut up.

For 10 years now I have taught about 135 kids AP Psychology, and I have averaged a passing rate of over 90 percent. In my opinion AP should be available to anyone who has the aptitude and the attitude necessary to be successful in any good class found in any high school in America. I have had students who took no other AP class but mine, and I have had students who took multiple AP classes every year.

AP is the best thing to measure in a high school because it is a real achievement test that marks proficiency at a college-level class.

-- Tom Di Figlio Boca Raton, Fla.

Schools that have magnet programs and IB programs that enable students to transfer away from their home school to the magnet or IB school cause the Challenge Index statistics to be artificially skewed. What happens is that the non-IB and non-magnet schools lose excellent students, which depresses their scores. At the same time, the IB and magnet schools gain students who would have been very likely to participate and succeed in the AP program at their home school.

A good example of the situation that I describe takes place at Richard Montgomery High School, which is the IB school for the central part of Montgomery County. Many of the top-performing students from Magruder High School and other schools that are within about six miles of Richard Montgomery are drawn into the Richard Montgomery IB program. This practice makes the surrounding schools appear to be lesser performers in the Challenge Index calculation and in [Montgomery County Public Schools] statistical reports, even though the surrounding schools are, in reality, often doing just as good of a job educating their students as the IB school.

-- Louis Wilen Olney

At the time that the Challenge Index (CI) was created, the choice of such a rating scheme made incredible sense. The more desirable factor of looking at the number of high school graduates from a particular class who had graduated from college within six years might be a more meaningful number, but one that lays outside the scope of collecting realistically. Initially the CI was the result of good education policies. As it did not yet exist, it was not looked at as a goal, but as a result. Students elected to take these courses because they felt well prepared as a result of previous courses they had taken. At West Potomac High School in Fairfax County, the primary courses devoted to AP work were electives, being European History, senior-year science (mostly AP Biology) and senior-year math (AP Calculus or AP Statistics). In addition, most college-bound seniors took either AP English Literature of AP English Language. For courses like U.S. history and U.S. government, there were both AP and college-prep level courses.

As with many things, once you start making measurements, you change the paradigm. Fairfax County Public Schools (FCPS) rated well on the CI, but as competition increased, the county wanted its schools to measure better. Raising the CI became a goal. In addition, the new superintendent wanted FCPS to benefit college-going throughout by graduating as many seniors as possible with 5 or more AP credits. One way FCPS did this was to make the only college-prep-level course for English and social studies for the junior and senior year an AP course. In addition, statistics and calculus were taught only at the AP level. Fourth-year foreign language and Latin courses were primarily taught at the AP level. There are many students that benefit from this push, but FCPS has never been able to show that all, or even most, students benefit from this push. Many college-bound students would benefit from more developmental, but still rigorous, instruction in their junior and senior year. I seem not to be able to convince anyone on the school board of this supposition. On the other hand, no one in the superintendent's office has pointed me to a juried paper that supports their policy. I have asked.

-- John Dickert Fairfax County

As a teacher in the Bellevue (Wash.) Public Schools for the past 15 years, I can tell you about the "down and dirty" effects of the index on our schools. Because our superintendent prized an outside assessment like the AP, he has pushed for "all AP, all the time" throughout the district's schools. As a teacher at the International School, one of Bellevue's top-rated schools (and at one point fifth in Newsweek's list of schools), the effect has been disastrous. . . . We were hard hit by having to push algebra into the middle school. All of our students who completed four years of high school math, as we required, would be forced to take at least one year of AP math.

The junior English class is now required AP, with only kids with special education status able to opt out. All 10th-graders must now take AP World History, with exceptions made only for special-ed students. These additional requirements have all been made by administration, without concern for teacher, parent or student input. On average, I'd say our graduating seniors take six to eight AP courses prior to graduating. You might ask, what's so very bad about that?

We are shortchanging our kids' childhoods. Upper-middle-class parents are pushing their students harder than ever to be all they can be: in the classroom, on the ball field, in the community. High school has become a sort of anxiety-laden marathon, but all kids are not created equal. Adolescents do not possess the adult coping skills required for this nonstop work environment. They are, after all, still adolescents. It would be so nice if they had time to hang out [no, not by studying notes from AP history], just hang out, shoot some hoops, find some balance. I have no idea what the long-term effects of denying our kids time will be, but I fear it will not be beneficial.

Not all kids welcome or are adept at doing college work in the high school. If kids' working at a college level in high school becomes de rigueur, why don't we just put the little buggers in college? Why don't we put middle-schoolers into the high school? Why can't kindergartners take on middle school work? Developmentally, kids are individual souls who progress in a sort of lopsided hobble toward adulthood. By assuming that one size [in this case, AP] fits all, we have demeaned our students' very identities and individuality.

-- Linda S. Boxleitner Humanities teacher, International School Bellevue, Wash.

I love the list, not least because we have been moving up and it can be noticed by parents, community and even bosses!

-- Stephen Williams College counselor, Eagle Rock High School Los Angeles

Let me say first that I am a friend of the Index. I think it is an excellent way to capture the intellectual and academic tenor of a school. As it happens, I have two sons who enjoy a challenge. And I believe that schools should offer that to any student who shows the desire to learn more and dig deeper. In some cases, it requires "outside the box" thinking -- schools need to put forth a "yes is the answer, what is the question?" attitude when thinking about these things.

Case in point: For many years, amongst its 20-plus AP offerings, Walt Whitman High School in Bethesda offered AP National, State and Local Government as a year-long class for sophomores. The general buzz was that this course was a good way to get your feet wet with the AP; as I understand it, the course is only a semester at the college level, but in the high school, they took a whole year to teach it. It was seen as a good introduction to the nature and rhythm of an AP course. My older son took it as a sophomore.

By the time my younger son was a freshman, Whitman had begun offering the course to freshmen. Whether or not this was a response to the overachieving competitive nature of the school community is unknown; what I do know is my son wanted to take the class. He did, and he enjoyed it and did well. Halfway through the year, we moved to Bainbridge Island. At the high school here, the same course is offered only to juniors and seniors; indeed they were not sure what to make of Dylan and his desire to complete the course. A meeting was held, and an exception was made, and he was allowed to finish the year. (For what it's worth, he got a 5 on the exam). However, as I understand it, that exception will not be made again -- the course is only available to juniors and seniors. Period.

-- Victoria Wilson Bainbridge Island, Wash.

I took 12 AP courses in my time at H-B Woodlawn in Arlington County. In fact, senior year I only took AP classes. I did very well on most of the tests and entered Georgetown 20 credits ahead of my classmates. Because of this, I was able to take a lighter class load each year, which allowed me to pursue other interests and opportunities: sports, internships, jobs and classes that seemed interesting but I didn't need to take. I also just took an entire semester off (what G-town calls "a leave of absence in good standing") to volunteer abroad (just got back from Ecuador!). Because of my AP credits, I will still graduate on time. And I saved a lot of money.

I also think that I learned more from my AP classes at H-B than I would have learned in intro courses in college. Well, mostly. Let me put it this way: I probably would have learned a lot more, and much harder, information taking Intro to Whatever at Georgetown than taking AP Whatever at H-B, but I would have forgotten it as soon as the final exam was over. I still remember a ton of what I learned in my AP classes, because of the way the teachers involved us in the material. Also, I took things that I never would have otherwise taken, just looking for challenges. For example, Dave Soles encouraged me to take his AP Chemistry class, even though I hated (and considered myself bad at) science. I ended up acing the class, getting a 4 on the AP exam and loving every second of it.

-- Rebecca Shinners Washington

The Challenge Index helped me point to a need for additional faculty training at Edison High School in Fairfax County. Edison is an IB school, and the training for IB teachers is more rigorous and expensive than that for AP. The school's index seemed relatively low, and when I, as academic chair for the PTSA, asked about, this I was told that they needed more trained faculty to be able to expand their offerings. The principal, Luke Fennell, immediately dedicated resources to train whoever on his faculty wanted the training (he had a large grant from a local business). And the number of trained teachers grew a lot, as did the number of IB students and the Challenge Index, too. The benefit went beyond IB, as those trained teachers also taught non-IB courses. The Index led directly to improving the skills of the faculty at Edison, which helped all of its students.

-- Dick Reed Fairfax County