Challenge Index

Jay Mathews
Washington Post Education Writer
Thursday, December 13, 2007 2:00 PM

Washington Post education writer Jay Mathews was online Thursday, Dec. 13 at 2 p.m. ET to discuss his Challenge Index rankings of Washington-area high schools

A transcript follows.


Jay Mathews: Hi. It is gloomy outside. A good day to sit indoors and sift some very revealing numbers on the level of challenge in our local schools. Keep in mind if I don't answer your question satisfactorily, or you wish a private chat, I am available at


Vienna, Va.: What's the difference between Langley HS vs. McLean HS besides the fact that Langley has less than 1 percent of kids that received subsidized lunches?

Jay Mathews: None, and even that difference is not very significant. It is hard to find two schools that are more similar. But I suspect there are some great teachers of special talent at each school that bring some intriguing variety to the mix.


Fairfax County, Va.: I wonder about quality control in AP and IB courses that allow anyone to participate. I remember reading an article by the English teacher at TC Williams saying that his AP courses were only that in name, because his class had too many mediocre students in it and his administration would not let him give them the poor grades they deserved.

Jay Mathews: Ah, yes. That is my old friend Pat Welsh, an AP English teacher who is also an author and frequent contributor to the Post's Sunday Outlook section. He and I debated this very point in Outlook two years ago.

I sense Pat is falling away from his former views on this question as he sees his school---finally---embrace the wisdom of the other Northern Va. high schools and open up AP to every student who wants to work hard and exercise his academic muscles for college. Pat did not seem to appreciate that in an AP or IB course where everyone takes the tests, a teacher cannot dumb down the level of instruction without being exposed in the test results. This means that AP and IB courses in No. Va., where all AP and IB students must take the tests, are almost never dumbed down.

My best argument against Pat was comparing his school's results to those of Wakefield High in Arlington, a school very close to TC both in distance and in demographics. Both schools have a majority of minorities and about half of their students are low-income. Yet Wakefield's AP participation rate was twice as high--all those mediocre students taking the courses! And those allegedly mediocre students were passing the exams at a much higher rate than the allegedly mediocre students at TC. TC has now embraced the Wakefield approach to AP, and its numbers are getting closer to those of its neighbor. I haven't heard Pat make the argument you cite in a long time. Instead he is discovering, I suspect, that a great teacher like himself can bring great improvement in those mediocre students, if he just applies his skills, and has a little faith.


Washington, D.C.: Mr. Matthews, in the past you've touched on the issue of why middle- and upper-middle-class parents will not send their kids to D.C. middle and high schools. We are African-American parents who sent our kids to D.C. elementary schools, and we are reluctantly resigned to sending them to private middle and high schools. This is not snobbery or elitism. We have looked at test scores, class sizes, parent involvement, and, more importantly, the atmosphere at D.C. public high schools and they do not seem to be places where, especially for African-American students, academic achievement is encouraged or celebrated.

Everyone talks about why it's good for schools to retain middle- and upper-middle-class families, but no one talks about why it's good for these families. Yes, it's possible to stay in D.C. public schools and get a good education -- I know many people who have done just that. But for many children, especially African-American boys, the culture of low expectations, large class sizes, disciplinary issues and parental apathy means that we will bite the financial bullet and go private. We're as liberal as they come and we've both worked in public interest jobs and believe in staying in D.C., but we're not willing to sacrifice our kids' education for some perceived social good. It may sound selfish, but I think most parents feel that way. We'll pay more than our fair share of taxes, put up with substandard libraries and rec facilities and all the other crosses we bear to live in D.C., but we're not fooling with our kids' education. I don't think that's snobbery.

Jay Mathews: Very well said. I have never commented in much detail on the issue you raise because I think your choice is a wise one, with a few exceptions. I have seen enough of both public and private schools in D.C. to be confident in saying that if you sent your child to Wilson, Banneker or the School Without Walls, you would be getting just as good an education as you would at Sidwell or NCS or St. Albans or Gonzaga. And you would find in those schools a critical mass of students that would allow your children to enjoy social pressures that push in a positive direction. (Remember, you are going to encounter some negative social pressures even in the best private schools.) Those schools may be far from where you live, and I would NEVER write off someone who decided to go the private route, since I did myself (I voted for a public school, but wife and kid outvoted me, and it worked out fine.) But I become more impatient with people who are surrounded by first class public schools, as many suburban residents are, and think the privates are better. In DC we also have some very promising small charter high schools that are doing good work, such as SEED and Math Science Tech. The KIPP people, my favorite charter educators, will soon start a high school in DC. But middle class folks like yourself may find that those schools, which focus heavily in raising low-income kids, are best left to that work while you seek other options. It will be a long time before most of the DC high schools cure the ills you described so well, and you have to do what is best for your kids. The Challenge Index is designed in part to show which schools in a bad district may be going against the tide, and so I hope parents like you take a look at places like Walls, or Wilson or Banneker before you make up yr minds.


Fairfax, Va.: Is there an argument for Fairfax County paying the AP and IB test fees for private school students? Why not, if their parents are county taxpayers?

Jay Mathews: I love this. A fresh question I have never heard before. I have been doing the Challenge Index for ten years and this is a treat for me. My first answer is that even if it made some sense, and it might at some level, it would never fly politically so there is not much point in considering it. My second answer would be more mischeivous. If a private school promised to make those AP and IB courses and tests open to all students who wanted to take them, as the public schools in Fairfax now do, and promised to provide free educations to a certain number of low-income Fairfax students in addition to their current scholarship commitments, a deal might be struck. But the taxpayers will never go for it, and the faculties of many of those schools would not stand for opening AP or IB to all.


Falls Church, Va.: How do you think the budget shortfall is going to affect schools? My son is scheduled to attend part-time kindergarten next year, and I think money should be taken out of the budget for other schools that offer full-time kindergarten. It's unfair that not all schools offer full time, or only part-time kindergarten right now. I'm a taxpayer and I'm paying a lot.

Jay Mathews: It is a good question. If we had all the money we needed, I would support not only kindergarten but pre-kindergarten for all. We have not yet reached that financial nirvana, I think the best course is to provide free all-day kindergarten, and pre-kindergarten, to low-income students whom the research shows will greatly benefit from it. The research also shows the children of middle-class, college-educated students do not need pre-K or K, or even that good of an elementary school education. Their children emerge in great shape no matter what kind of primary schooling they get because of the effect all those good conversations and book reading and museum going and other parental enrichments have on them.


Woodbridge, Va.: With the financial burdens schools are facing -- tax revenue decreasing, state funding coming up short -- how can these counties justify spending thousands of dollars on these AP tests for students who are not passing the class? I think funds should be available only to students who are passing the class. I'm not advocating that students shouldn't take AP classes, just that the money should be better spent.

Jay Mathews: It is hard for me to explain this without taking you inside the classrooms of the great AP and IB teachers who are working with those students who are not passing the class, but your suggestion would be disastrous. It is the ill-prepared low-income student who is going to benefit most from the difficult class and the demanding final exam. He or she will build academic muscles that will make it far more likely they will succeed in the work place, or in college. They might have trouble passing the class, and the exam, but they will be much better off having tried it than being consigned to the pablum courses that we often reserve for such kids. It is those kind of students that we are serving worst in our public schools, and your suggestion would take us back to the dark ages, which ended in 1998 when Fairfax County started the move to opening AP to all in this area.

If by not passing the course you mean a student who is not trying--doing no homework, not participating in class, not exercising anything---then I would have no trouble giving that student a choice of either getting to work or being assigned elsewhere. But it would be stupid to drop a kid who is trying to get better. And for the cost of just $84 per test, it is the greatest bargain in American education today.


Falls Church, Va.: If there is no difference between Langley and McLean, then why is the former ranked No. 4, while the latter is ranked No. 16?

Jay Mathews: You want me to bemoan my decision to rank schools, because that is the problem. Ranks that close to each other, particularly when we are talking about 186 schools, have no meaning. It is the same as the current college football rankings. As has been proved every week lately, on any given day the 16th ranked school can beat the 4th ranked school. They are really not that different at all.

But we tribal primates, with the tribal pecking order in our DNA, think small differences in rank are important. It is not a rational feeling, but it is there in our viscera. I can't do anything about that, so I exploit it. Based on 40 years of working for newspapers, I know if I didn't rank, no one would pay attention the issues raised by this way of assessing schools, which I think are very important. Editors would not even publish my results. I realize that people are going to reach the wrong conclusion about the differences between Langley and McLean, but that is the price I am willing to pay to show them that, for instance, there are a few schools in places like DC that are doing great work. That leads to useful questions like, how can that be? What are they doing that other DC schools are not doing? And then we get some progress, I hope.


Falls Church, Va.: I think your calculations for the Challenge Index should be expanded to not only include the number of AP/IB courses divided by students but to include the number AP/IB courses with passing grade divided by students. This is different from your equity in excellence figure.

Also, you should devise an equation that factors in the passing score relative to the number of students. This fine tunes your Challenge Index. These data points may change your rankings considerably.

Jay Mathews: Please email me (see address above) and explain further how your ideas are different from the equity and excellence percentage. They sound like the same thing to me.


Virginia: I, like other Foreign Service brats, went to overseas International American Schools. We took many IB classes. They were tough and helped us a lot. Then we came back and our new peers know more about beers than we do. Seemed to be a cultural issue.

Jay Mathews: Please email me more. I think this may be a bit of wry wit that is over my head, but I do want to understand the question.

If you are suggesting the IB courses here are different than they are abroad, that is hard to imagine since you took the exact same five-hours exams that IB students in the US took.


Washington, D.C.: Why are D.C. and P.G. schools so low? Whatever happened to academic excellence there?

Jay Mathews: It is very simple demographics. Academic achievement correlates very closely with parental income and education.

The likely reasons for that are many. Low-income parents were usually not successful in school and don't know how to guide their children to be successful. They have a lot of distractions in their lives. They can't afford trips to museums and to Europe. They never learned to love books when they were children. DC and Prince George's have the largest portion of low-income public school students in this area, so they are going to have, on average, the lowest test scores. Schools with lots of low income children also find it difficult to recruit and keep the best teachers, because it is harder to reach such kids and raise them academically. But it can be done. The Challenge Index reveals some schools that are doing it, like Stuart and Annandale and Wakefield and Wheaton and Springbrook and SEED. We also have some charters like KIPP that are getting great results with these kids. They just need more time and encouragement to learn, to make up for the fact that their parents cannot provide those ingredients in the same way college-educated parents can.


Virginia: I'm a TJ parent who knows that TJ does not have all the best and brightest students in the region. Far from it! And with some of the new policies, particularly the grading policy, the school is discouraging intellectual risk-taking and demoralizing its more curious and able students. So I thnk what I am hearing from TJ kids is probably heard by parents of highly able students at other high schools, too, and might resonate with other parents and educators in this discussion.

Some of the best TJ students comment about how disappointing their AP courses are. For example, one student was disappointed that there were so few questions about the subject matter for its own sake, not whether the particular topic will be on the exam. I've also heard complaints about the lack of discussion concerning connections between the subject under study and other subjects studied by the students. And, finally, I've heard quite a few complaints about AP teachers delivering a recycled package of lectures designed to garner the top scores on the exam but failing to foster creative thinking and fresh ideas.

Colleges seems to be reconsidering the value and the place of AP courses. Might these student comments shed light on what is bothering college professors and administrators about the AP program?

Jay Mathews: What you are describing is not the inadequacies of the AP program, but the inadequacies of some of the teachers who are in AP classrooms. I have seen the best of the breed in AP and IB, and none of their students have any complaints like that, quite the opposite. So a word or two to the TJ principal might be in order, since TJ students deserve the best possible teaching, as do all students.

As for the colleges occasionally being critical of AP, the data and a lot of reporting by me indicate this is a combination of professional jealousy and fear of financial consequences. All evidence suggests the average AP class is a better educational experience--more focused, more challenging, better taught--than the average college introductory course that AP is designed to replace. So many more high school students are taking AP and getting credit for the exams that college administrators worry that they will have to cut back on intro courses, and have nothing with which to employ their graduate students. Also, college administrators, with no experience with AP and no data, just assume that college teachers are always better than high school teachers. Those of us who have been to both high school and college know better.


Reston, Va.: Jay,

In your national ranking, schools that offer only IB exams are noted with one star, those that offer both AP and IB have two stars. Yet, according to state data, all of the one-starred IB schools near the top of your list offer both IB and AP classes. Washington Lee, one of the top IB schools on your list had 53 IB Diploma Candidates and gave 549 AP exams as well. Without those AP exams, they couldn't have made it high on your list, I imagine. Schools that offer only IB are at a great disadvantage because most tests are given at the end of a two-year sequence. Juniors rarely take IB tests. Why were these schools listed as IB only schools?

Jay Mathews: Sorry. My fault. I have to be much more careful with those asterisks. You are right. W-L has both, and it appears in today's list we indicated it just has AP. But you are actually quoting the Newsweek list asterisk system, not the one in the Post today. In the Post we have two categories, schools whose totals include IB tests, one asterisk, and schools that include other college-level tests, like Cambridge, two asterisks. Almost all those designations are correct on today's list. But I have to squint more and make sure there are no errors in the future. Maybe it is time for me to break down and get some reading glasses.

As for your suggestion that schools that just have IB are at a disadvantage, I don't think so. IB students are likely to take more tests than AP students, because IB students who want to get the IB diplomas must take six tests, far more than most AP students will take. Schools that have only IB still rank on average much higher than schools that only have AP. And they all have the option to become AP and IB schools if they want. I say, the more choices the better.


TJ: Jay -- All this hype about TJ. Do they disclose how many applications they get each year? I know many, many more than they accept; but, would love to get an idea of the numbers rejected/accepted.


Jay Mathews: They do disclose the numbers. In my book Class Struggle, I report that in 1995 they had 2,700 applications for 400 places. I think the ratio is still about the same, but if you call them, they will tell you. It may even be on their Web site. It is very hard to get into TJ, which is why it is the highest performing public high school in the country.


Alexandria, Va.: It would be helpful if you could include a similar listing of high schools in the area that did not make Adequate Yearly Progress under the NCLB federal law. I would be curious if any of the top schools in your listing would be considered failures under the law. Did you look at this as a factor in your listings?

Jay Mathews: It is not a factor in the list because, although I think No Child Left Behind was an improvement on previous federal support for public schools, the AYP label is in many cases a very deceptive indicator of the quality of the teaching and the level of challenge at that school. If you have lots of low-income students, you are almost certain to miss AYP. Many of the low-income schools who do well on the Challenge Index have missed AYP. Most of the high-income schools have not, although a few have because they tripped over small details in the very complex evaluation process. I like the intent of AYP, to have everyone focus on achievement of low income kids. But we have not yet produced a sure cure for low achievement that is attainable by educators in most of our schools, so to factor that in would be to penalize them for a problem they can't fix, no matter how hard they try.


Alexandria, Va.: The fact that some schools are "requiring" AP/IB students to take the pertinent tests is stated as if it is a great idea; I think it should have been a given all along.

Does the challenge index count students who take the test but score 0-2? What good is that?

Does the challenge index also calculate students who do not take an AP course but do take the AP exam and score 3 or above?

Jay Mathews: On your first point, I heartily agree. On your second, please see my answer to the question above about why we should not kick kids out of AP classes that they are failing. On your third, the Challenge Index does not note the score of a test. Our attendant equity and excellence percentage does. We do count any test taken by any student at a school, whether they took the course or not, but the number of students who take the exam without taking the course is, nationally, way way below 1 percent. I took an AP test once without taking the exam in 1997, when I was 52. I did okay, but I would have loved to have taken the course.


Fairfax, Va.: The challenge index is determined by dividing the number of tests (including those taken by non-seniors) by the number of graduating seniors. Why not just count seniors? It seems like padding.

Perhaps on a related note -- some individuals take several tests; many take none. People might think schools have more individuals taking tests than they really do.

Jay Mathews: It may seem like padding, but it is not. Dividing by the number of graduating seniors is just a convenient way of leveling the playing field so that large schools, which naturally give lots more AP tests than small schools, do not have an advantage because of their size when we figure the rating. I could have divided by total school enrollment, but that number changes depending on what month you count. The number of graduating seniors is a one time, hard number, and also has the advantage, particularly for inner city schools, of measuring a school's AP and IB efforts against just that core of students serious enough about school to graduate. Those are the students who are reachable by the good teachers who are really the ones being rated by the index.


Montgomery County, Md.: My son attends one of the schools consistently ranked in the local Challenge Index top 10 from year to year. While the school's opportunities and achievements are impressive, there is a downside for kids (read, generally, boys) who don't have the maturity to handle college level work at the age of 15 or 16. The typical high achiever at my son's school takes at least one AP course as a sophomore (and some as freshmen), three as a junior, and God knows how many as a senior. For a kid like mine whose maturity hasn't caught up with his intellect, none before junior year and only one then is more than enough. Yet my son's self esteem really has taken a beating because he sees other kids doing so much more and so thinks there is something wrong with him. My belief is that what is wrong will be fixed by the passage of time, but that doesn't help him now.

Jay Mathews: This is also a new question. I can't speak with confidence about a student I have not met, and about whom I know very little. My first question would be, why limit him to just one AP in junior year? Why not one demanding academic AP, and then one of the more enjoyable, less academic courses, like AP psych or music theory or studio art? Given the great number of APs in your school, you didn't see anything else he would enjoy? Some of these courses are difficult, but the greatest benefit of an AP is that it tends to be better taught than a regular course, and prepares a student for college. I realize that in schools like yrs there may be a social element that makes a student feel out of it, but he does have a chance to take two more senior year, and three is enough for any kid going to college. Sadly, the vast majority of US high schools have the opposite mindset. Most average students are barred from or discouraged from taking AP, which I think you will agree is a much more serious problem than yours.


Bethesda, Md.: Once again, you've fallen into the trap of overemphasizing a single measure that says relatively little about the overall quality of any high school's programs. While AP/IB courses are a valuable challenge for those students who can handle them, your index totally ignores programs that serve the average or special needs students who aren't going to keep up in AP classes. Why not also rate how well the students do on the tests -- are you automatically better off for taking AP classes if you don't learn enough to pass the test? At the very least, weight the rankings based on the scores.

The U.S. News rankings aren't much better, but at least they're a little more comprehensive than yours. How about a raking system that takes things like overall outcomes (career and college placement, and 1st year success) and extracurriculars into account?

(P.S. -- this isn't sour grapes. My kids went to one of the top rated schools on both lists, took lots of AP courses and thrived in college.)

Jay Mathews: These are almost all good suggestions. Unfortunately we have no way of measuring extra curricular participation or career or college placement or even first year college success on a high school by high school basis. Once we develop a test tracking system for all students, we may be able to record first year college success by high school, at least in state colleges. That will be a welcome addition.

Your other suggestions, like test scores, take me back to the reason I developed this measure. Those scoring rates are mostly measures of parental income and education. The richer the school, the higher it will rank. The Challenge Index comes at schools from a very different angle, and is a much better measure of what is going on inside the school, as opposed to what is going on inside each student's family.

You say: "your index totally ignores programs that serve the average or special needs students who aren't going to keep up in AP classes." Your statement assumes that average and special ed students can't do AP. The Challenge Index, inspired by a school--Garfield High in East Los Angeles--that was full of students considered average or below average doing great in AP, assumes the opposite. There are plenty of schools on today's list that prove you wrong. Visit Wakefield High in Arlington some time and prepare to be surprised.


Chevy Chase, D.C.: Mr. Matthews: It seems to me that your approach to evaluating schools includes an element of "race norming," i.e., they practice grading performance differently depending on race. This is evident in your view that SAT scores reflect student body make up rather than school quality, and also in your focus on the number of AP tests administered rather than on the scores received. Isn't that a form of bigotry? If a school administers a lot of AP tests but does not get a lot of high grades, isn't it really meaningless to call that school a successful school?

Jay Mathews: Nope, because the list exposes difference AMONG nearly all white affluent schools, just as it exposes differences AMONG nearly all minority low-income schools. There is no racial element to making the good decision to open up AP and IB courses and tests to all students who wish to make an effort and give those students the time and encouragement they need to succeed. Take a look at today's list. Notice the "%lunch subsidy" number for each school, which is the poverty indicator. Notice that there are some schools with lots of low income students (which in this community also means lots of black and/or Hispanic students) that do well on the list, and some that do not. Notice there are some high-income that do well on the list, and some that rank much closer to the bottom. That proves pretty conclusively, I think, that we are not talking about race here. We are talking about which schools encourage good, challenging teaching, and which do not.


Rockville, Md.: You've argued that taking AP courses is valuable even for students who don't take the test or take it and fail it. Your point -- and I believe you've made it explicitly -- is that the AP curriculum is more challenging, with or without the test. Given that, shouldn't you be advocating that states adopt curricula modeled on AP for all of their students?

Jay Mathews: Exactly. There are some educators, such as former Bellevue, Wash., school superintendent Mike Riley, about to relocate here to work for the College Board, who have long argued exactly that. It will take a while, but I think we will get there. Standards tend to rise in this country. When I was born, people would have thought it was crazy to suggest that most Americans would eventually go to college. Hopefully we can get to a place where most Americans also graduate from college.


Who's on First?: In the National Rankings you list WT Woodson and Langley as No. 62 and No. 69 respectively, numbers below:

62 W. T. Woodson-- Fairfax Va. 3.941

69 Langley-- McLean Va. 3.716

But in the local listing their order is reversed with different scores for each Challenge Index:

4 Langley-- Fairfax 4.280

10 W. T. Woodson-- Fairfax 3.907

Do you use a different methodology for the local vs. national rankings? If so, why? And if not, how do you explain these differing outcomes?

Jay Mathews: The national rankings in Newsweek, first published last May, are based on 2006 data. The Post rankings released today are based on 2007 data. Woodson maintained its very high rating, but Langley had a significant increase. At that level, it is good news all around. Both of those schools are in the top one half of one percent nationally, and I suspect will be there when we do the Newsweek list using this same 2007 data next spring.


Jay Mathews: Thanks for the great questions. Please email me personally if you have more. And happy holidays!--jay


Editor's Note: moderators retain editorial control over Discussions and choose the most relevant questions for guests and hosts; guests and hosts can decline to answer questions. is not responsible for any content posted by third parties.

View all comments that have been posted about this article.

© 2007 Washingtonpost.Newsweek Interactive