Challenge Index

Washington Post education writer Jay Mathews was online Monday, May 19 to discuss his 2008 Challenge Index rankings of high schools across the country.

A transcript follows.

____________________

Jay Mathews: Good morning. I was a little concerned about scheduling this live chat so soon after Newsweek released the new list. Some of us don't get our copies of the magazine in the mail until later today, or tomorrow. But that is just the thinking of a dinosaur, born long before the Internet era, who doesn't realize that the smartest people these days are the ones who, unlike him, go to the web, and don't wait for the sliced up dead trees to arrive at one's doorstep. So I better get started. You people are busy, and have a lot of other web sites to visit.

_______________________

Richmond, Va.: I have read the criticisms of the methodology used to calculate the Challenge Index, and I have read your defense. As an AP Calculus teacher in the Richmond (Va.) suburbs, I think you need a cold dose of reality splashed in your face. Every year, fully half of the students in my AP class have no business being there. These students are in my class because of a horrible combination of parental and student over-estimation of their abilities and a school (and school district) policy designed to boost our ranking in your index. Rather than being challenged by college-level coursework, these students would be much better served in a class that either reinforced concepts from earlier classes or moved a much slower pace. Instead, these students will struggle mightily, get a 2 or 3 on the AP test, be know better prepared for college, and become disenchanted with higher math. But we scored higher in your rankings this year, so I guess all is okay.

Jay Mathews: This is one of the most interesting questions I have ever received, a terrific way to start the chat. You are a pro who knows your environment, but you have to answer a question for me. If what you say is true, and half of our kids should not be in Calculus, then how do you explain what happened at Garfield High School in East Los Angeles when I was there? You may have seen the movie, but I would be very grateful if you would skim my book, Escalante, and tell me what you think. THAT Calculus teacher believed that giving such kids the extra time and encouragement needed to master the material would yield great benefits, and the data suggest he was right. He had more Mexican Americans succeeding at that one school than some entire states did. His pass rate on the AP test was about 66 percent, above the national average, and he had plenty of 4s and 5s. He argued that not only would good teaching bring more kids, even ones that appeared ill-prepared, to that level, but it would add academic muscle to those kids who could not get to that level, but would learn a great deal anyway. When they retook calculus in college, they would be far more likely to succeed, having struggled with it in high school, than they would if left in, as you put it, a slower class.

And where would they find this slower class that would still stretch them and keep them moving forward? If you have one at your school, please tell me about it. I have visited about 200 high schools around the country, and I am rarely finding such classes. Schools tend to put kids they think are ill-prepared for difficult math in classes that do not prepare them for more difficult math. They don't think those kids can handle it. It would be nice if this were not so, but I don't see any way to change the culture to make such courses appear. All we have is AP and IB, and I can put you in touch with dozens of Calculus teachers who have made their AP and IB courses work for such kids.

But we need a longer exchange on this. I could learn much from you. Please email me at mathewsj@washpost.com, and I welcome anyone else who wants to keep this conversation going to email me there too.

_______________________

Central Va.: Do you have any sense how state standards of learning positively or negatively impact the standings of high schools? I notice quite a few Virginia high schools in the rankings, but few in my area (none actually in the county where my child would attend). I confess I am no fan of SOLs and would seek out alternatives that do not place so much emphasis on them, such as IB, Governor's school or private school. My child attends Montessori. Thanks.

Jay Mathews: The Challenge Index ratings in Newsweek have nothing to do with state tests or state standards. Those tests and standards, I think, are better than nothing. They do motivate schools to get more kids up to a basic level. They raise the floor, as schoolteachers like to say. But what AP and IB do is raise the ceiling. Kids can shoot for college intro course competence, which completely changes the dynamics of the schools that open those courses to all, and thus make it much more likely they will appear on our list. You are right not to focus on how a school does on the SOLs. That just tells you what percentage of the parents are affluent, the iron law of standardized test scores. Instead look for schools that make our list, who are trying to get ALL kids, rich and poor, to a college level. The reason many of the schools in your area don't appear on the list, I would imagine, is that the culture hasn't changed yet in your schools. Up here in the Washington area, nearly everyone has accepted the idea that AP and IB should be for all kids who want to work hard. In most neighborhoods, including yours, I would guess, this is considered nuts. Most US high school administrators think AP should only be for A students, even though the research suggests it is the B and C students who can get the most out of it.

_______________________

Newark, Del.: You mention that you left out schools with SAT and ACT scores that were too high to compete, but much of the top of the list are schools with "Gifted and Talented" or "Magnet" in the name. I can't believe I'm saying this, but perhaps the standards for what counts as a 'normal school' should be lowered?

Jay Mathews: I am open to any and all suggestions. I decided from the beginning to exclude schools like the Thomas Jefferson High School for Science and Technology because the list was designed to recognize schools that got their average students involved in AP and IB, and schools like TJ have no average students. There was a problem with just excluding magnet schools. Some magnet schools, like Banneker in DC, were full of students who would be considered average in the suburbs. Banneker's average SAT score, math and reading, is under 1100. Many of our suburban schools average over 1200, and they are not magnets. So I decided to find out which suburban regular enrollment schools had the highest SAT or ACT averages, and only exclude those magnets and charters who had average scores significantly above theirs. That means the cutoff now is 1300 on the SAT and 29 on the ACT. If you are a magnet and you are above that score, we celebrate you superiority by putting you on our separate Public Elites list. If you are under it, you go on the regular list. Notice some famous magnets, like Lowell in San Francisco or Boston Latin, are below several regular enrollment schools on our list.

_______________________

Fairfax Va.: I have been a frequent critic of yours in the past and I still am.

It all boils down to this: I don't believe every kid is capable of AP. I don't believe every kid is capable of college.

What I do believe is every kid should be challenged, but to think that every kid can be successful at the AP level is ridiculous.

And for the thousandth time you should not have to slow or water down an AP course for the kids who don't belong. This is detrimental for the kids who actually belong there. There are plenty of other classes for the kids who are not ready for a true college level AP course.

Jay Mathews: If I thought the philosophy I learned from hundreds of successful AP teachers--that all kids should be given a chance to take AP---produced watered down AP classes, I would. like you, question my thinking on this. But it turns out that doesn't happen in well run schools that understand AP and keep track of those classes, and require all AP students to take the AP exams. The best example is Fairfax County, Va., which ten years ago announced that all students were welcome in any of their AP classes, and all AP students would be required to take the exams. I waited for my phone to ring with angry phone calls from teachers and parents and students about all the dumbed down AP classes that resulted. So far, ten years later, no such phone calls, or emails. Fairfax's experience, and the experience of hundreds of others schools that have tried this, show that you cannot dumb down an AP or IB class without getting caught, as long as everyone takes the exam. No self-respecting AP teacher wants to be shown up as not preparing his or her kids for that test, and so the dumbing down does not occur. It DOES occur, a lot, in schools where kids can avoid the exam without penalty, but the list---which counts exams--was designed to discourage schools from doing that, and I think we have had some success in that venture. If your kids aren't going to take the exam, no one can tell if you have dumbed down the course or not. That is a bad situation, which is why I applaud schools that say, if you are in AP, you must take the test.

But you sound like you have some facts on your side. Email me what you know. Is there a school that has done all the right things and is still dumbing down AP? Send me your data. It will help me.

_______________________

Ellicott City, Md.: My daughter goes to a school on the list -- I'm extraordinarily unimpressed. My experience as a parent has been that the teaching for AP classes is very uneven, and the school administration is more interested in getting good "numbers" (and appearances on lists like this) than in what the kids actually learn. I wonder how many kids who test out of college classes will miss important learning because what's covered in college wasn't covered in high school?

Jay Mathews: We have plenty of evidence to show that the AP and IB classes are, on average, better and wider and deeper learning experiences than the average big state university intro courses that they replace. The AP and IB kids spend more time with their instructor, their instructors are more experienced teachers, their final exams are longer and more encouraging of thought and analysis, etc. AP regularly gives AP exams to college students who have just completed an intro course, to make sure they are mimicking the college's grading system.

Some AP courses are not, as you say, so well taught, but as I pointed out above, AP, IB and Cambridge are the only courses in high school where the students, the principal and smart parents like you can actually check to see how good the teaching was. If a class full A students gets low AP exam scores, you know something is wrong with the teaching. You do not have that advantage in assessing the teaching in any other high school courses. Talk to some kids who have taken AP or IB, and gone to college. You will learn that in the vast majority of cases, doing AP or IB was an enormous advantage in their adjustment to tough college classes.

_______________________

Mt St Joseph High School, Baltimore Maryland: Mr. Mathews;

I am more than a bit confused about your use of SAT numbers. Current SAT test is out of possible 2400. 1500+ is Maryland average, here at the Mount it is 1600+. My 7th grade daughter just posted a 1350 this past winter.

What's up?

Jay Mathews: As I said in that answer above, I am only counting reading and math, two of the three tests. I don't count the writing because it is still a wildcard, not used by many colleges, and it is easier to just go with the two tests we have looked at for several decades. I will try to make that clearer in my answers. Thanks for helping me do that. I am looking at just the two tests that total 1600 top score, not the three tests that total 2400 top score.

_______________________

Annandale, Va.: Where can I find information about good grade level (i.e., elementary and middle schools) schools in the area? Are there any rankings for these schools?

Thanks

Jay Mathews: There are, sadly, none that are very helpful. I did do a story in the Washington Post magazine last year about 30 good middle schools in this area, passed on emails from readers and some other data, but that was just a sample. I think the best way to judge a middle school is what percentage of its 8th graders complete algebra I. (50 percent is a good number. The national average is 25 percent.) But nobody puts out a list with that data. Maybe we will some day. You can go to any district or state Web site and find which schools have the highest test scores. That has some use, but mostly it is telling you which schools are located in the most affluent neighborhoods.

_______________________

washingtonpost.com: "Unstuck in the Middle" by Jay Mathews (Washington Post Magazine, April 15, 2007)

_______________________

Annandale, Va.: What strikes me about the list is, compared to some other areas, notably in Texas (!), our area seems to lack schools open to all students, who might be motivated by a specific interest like health professions or arts. Is this a true deficiency? Is there an argument against them?

Jay Mathews: What a smart question. Sadly, MOST of the special interest high schools you will find are in urban districts where the administrators thought they might produce better teaching and learning if they focused on certain career fields. In most cases it does not work. I have visited some DC schools that have these academies and they are mostly a joke. In a few states and cities, and Texas is a good example, they have not just slapped a new career-oriented name on the school but gotten serious about hiring energetic, skilled teachers and raising the level of instruction, including heavy doses of AP and IB. Those successful cases you find on our list, but most such schools nationally are failures. We do have a couple such local schools appearing on the Newsweek list this year, McKinley Tech, a DC magnet, and Thurgood Marshall, a DC charter. Both have career-oriented programs, and include a lot of AP. But their scores on the tests are not very high yet, so they have a lot of work to do. I don't actually think career-oriented high schools are necessary to providing a good high school education. What students that age need are the reading, writing, math and presentation skills that will get them college places or good jobs, and any well-run school can provide those, no matter what it calls itself.

_______________________

Bethesda: I want to thank you for your list. We have been considering a move to another part of the country and we have used your list exclusively to identify the towns where we want to relocate. It may not be perfect but it's probably the most efficient manner for finding high performing schools.

This seems like a simple question but I just want to make sure I am making the right assumption. Using HB Woodlawn (index 5.63) as an example, can I assume that the average graduating senior has taken 5 AP classes over four years?

Jay Mathews: Yes, that is about right. The mathematically relationship is a bit more complicated, since a lot of juniors and some sophomores take AP test, but what you are saying is close enough. I like to say that any school can make the Newsweek list if just half of its juniors and half of its senior take one AP test. It is sad that only 5 percent so far have reached that modest standard. Thank you for the kind words, and let me know how your new school works out for you.

_______________________

Metro Detroit, Michigan: I couldn't help but notice in your rankings that private schools were absent. Do you do this for private schools? I went to Roeper school in Birmingham (MI) and took about 9 AP exams on my own (almost all of my classmates took several APs as well). I'm sure some of the other private high schools (Cranbrook among others) would rank highly as well. I point this out only because one of our rivals, International Academy, is technically a public school but competes with other private schools. It is ranked 12th in your list, which strikes me as an unfair assessment given its small class sizes relative to most other public schools. When you only have 125 hand-picked students per class (versus more than a thousand for some public schools in the area), its a lot easier to get ludicrously high rankings by this statistic. You might want to consider size when compiling this list.

Jay Mathews: If you notice, the Newsweek list is accompanied by a piece I did on the size issue. I pointed out that the portion of smaller schools at the top of the list has gotten much bigger, a sign of the growing popularity of small schools, such as charters and magnets. My view is if smaller size works better, I don't want to discourage it on the list. I want to show its effects and suggest the even more schools move in that direction.

As for the lack of private schools, I would love to rate them, but the private schools themselves--particularly the best known and most expensive independent schools--quake at the very idea. The National Association of Independent Schools put out an all-points bulletin when I started doing the list warning its members not to send me their data. They said the ratings might appeal to what they called "consumer-conscious" parents, but could not adequately convey the inherent wonderfulness of their unique contributions to children's lives. They have a point. The list has a narrow focus, but as the parent above noted, it has its uses. Those privates are saying they think parents like you and me are too stupid to look at a list in a magazine and make an intelligent judgment on the use of the information. Since so many private schools stonewall us, we decided not to try to rate them. But times may change.

_______________________

Fairfax, Va.: All things being equal, I would MUCH rather my kids go to Robinson HS, Lake Braddock, Oakton, Madison, over HB Woodlawn, which is consistently ranked among the highest echelons of your list. Why, because I am a racist elitist who hates poor people? NO! because I am a Hispanic parent that wants the best for my children, and the best is not a school with a poor graduation rate, high rates of violence, a school where a large portion of the students don't speak English, but are all artificially pushed to take classes they aren't qualified in order to increase the schools ranking.

Jay Mathews: You should send your kids to whatever school you and they are comfortable with, but your message leaves the impression that HB Woodlawn is a school with a poor grad rate, high levels of violence, and a large portion of kids that don't speak English. You must be mixing it up with another school. Woodlawn has hardly any dropouts at all, very few students who have any trouble with English, no gangs, no high rates of violence. It is a very small school with a deep academic tradition. Only students with some academic ambition ever sign up for the lottery to that school, since they know its high standards.

_______________________

Bellevue, Wash.: As a parent of two high school students who attend a school in a district that emphasizes AP and IB and therefore has a number of school very high on the list, I have a concern about the cumulative effect of the push towards an AP curriculum. In the effort to push students towards these classes, we now have no honors classes in 11th and 12th grades. This means students either have to take what they perceive as "the dumb classes" or AP. I have no problem with 1 or 2 AP classes, but this year my daughter had to take five AP classes as a junior and next year will be more of the same. Some students can handle this pressure, but I think others would be better served by taking a mixture of AP and honor classes. What is your feeling on this?

Jay Mathews: You should check out this great story on the issue in today's paper by Dan De Vise:

http://www.washingtonpost.com/wp-dyn/content/article/2008/05/18/AR2008051802461.html

Many honors courses over the years have, sadly, lost their reason for existence. In some cases, they have maintained some rigor and vitality, mostly because they are taught by a teacher with a passionate commitment to the subject. But in many cases they have become little more than refuges for middle class students who did not want to be in that regular course which they considered full of fuss-offs, and did not want to work as hard as they would have to in AP. That is really not a good reason to keep a course on a high school schedule. I suggested in a column last year that those high schools eliminate NOT the honors course, but the REGULAR course, and finally give all of those alleged dummies in the regular course the good teaching they deserve, with an honors or AP choice. Here is the column:

http://www.washingtonpost.com/wp-dyn/content/article/2007/07/16/AR2007071601288.html

AP and IB courses, in my view, are less stressful in the long run because they are engaging and well-taught, usually, and add value to a student's time in high school. They in the end make COLLEGE less stressful, and that should be included in the equation. Your daughter, however, is not, I suspect, taking all those APs because there are no honors course alternatives. She is taking them because she wants to get the best possible education. Smart girl.

_______________________

Fairfax, Va.: Would you support an initiative to build another school similar to Thomas Jefferson, in Fairfax (or Loudoun or PG County)?. I don't think I would have any problems with the way you rank schools, if we gave high achieving children more opportunities to separate themselves from the pact. Thoughts?

Jay Mathews: I think that would be a fine idea. Loudoun has started a program like that. Although I might want to have an admissions system that was less stuck on only taking the very highest scorers. And I can think of plenty of ways we could help genuinely gifted students right now, such as not being so resistant to acceleration.

_______________________

Reston, Va.: I am a foreign language teacher at a high school. Like several questioners before me, I question the value of your rankings as well.

Students (and parent and administrators) seem to think that AP courses are the magic elixir to get INTO a college - rather than good preparation for college. I have seen students focus on AP courses at the expense of Math or Foreign Language courses that are required for entrance to most colleges.

The thinking (of students and parents) seems to be "if one AP course is good, 5 is better". This does not seem to be your rationale for your ranking system, but it is a consequence of the value schools place on your rankings.

I believe the only true statistic that would evidence how well a school is preparing students for college would be one that showed the number of graduates from that school who actually successfully completes a 4 year college program in 4-6 years.

Jay Mathews: I like your suggested rating method a lot. Sadly, we lack the data to make it happen, and probably won't be able to get it until every student in the country has a distinct electronic tag, and that doesn't sit well with American views of privacy rights.

Some students overdo AP, but they are a TINY minority, less than 1 percent of all students. Notice that most US high schools don't have much AP taking going on at all. That is the real problem. About half of kids going to college have not taken a single AP course, one reason I think why we have such a high college dropout rate.

This sentence interests me: "I have seen students focus on AP courses at the expense of Math or Foreign Language courses that are required for entrance to most colleges." I don't understand how that could happen. Math and language courses come in sequence. You can't take an AP course in those subjects unless you have reached a certain level, usually 4th or 5th year, where you will have already completed the college requirements for having a certain number of math or language courses. See my email address in question one, write me and tell me what you meant.

_______________________

Windsor, Conn.: Last year the College Board audited all AP classes nationwide due to complaints made by post secondary institutions regarding a lack of uniformity in the rigor of AP classes. Apparently, many student transcripts showed AP credits but the student's ability was well below AP standards when taking advanced courses in college. The College Board has since required all AP teachers to submit a syllabus to prove that the rigor is indeed of AP caliber. However, once submitted, the College Board can not be present to ensure that the teacher is following the more difficult standards. As your first questioner stated, just because the student is in the class does not mean the student is capable to perform in AP class. More importantly, are the classes of the schools mentioned of equal rigor? The one California teacher is an exception. Don't use their one in a million academic increase to hide behind a questionable methodology. I can drive in a 25 minute radius from where I live and find significantly divergent requirements for academic success just within basic courses let alone AP classes.

Jay Mathews: You have been sadly misinformed about the audit. There was absolutely no evidence that student with AP credits were performing below college and AP standards. You can't get the college credit unless you score well on the AP exam, and almost all the available data shows that students who have done well enough on AP to get college credit do BETTER in college than those without those credits. See the Saul Geiser student at Berkeley, or the National Center for Educational Accountability study at U of Texas.

You say: "The College Board can not be present to ensure that the teacher is following the more difficult standards." But it doesn't have to be present. EVERYONE can check to see if the teachers has followed the more difficult standards by seeing how those AP students score on the AP (or IB, or Cambridge) test. No other high school programs have that feature of an independent, incorruptible exam. Which is why I thought the audit was a big waste of time, but the College Board people wanted to play nice with their college members, who seem threatened by the fact that so many students these days with AP credit are successfully jumping over their intro courses. I say the colleges should learn to appreciate how much better prepared AP or IB students are, and wake up to realize they are in a new century.

_______________________

San Diego, Calif.: The fallacy of using the number of students taking AP classes as a measure of school performance is evident from your number five school, the Preuss School. They were found guilty of wide-spread grade tampering and watering down AP classes in an audit last fall. Here is one article: Principal at Preuss abruptly steps down (San Diego Union-Tribune, Dec. 19, 2007)

Jay Mathews: I know all about that case, but notice the hard core of AP incorruptibility that survived even that scandal. The principal may have been able to change report card grades and other data, but she could not change the kids' AP test scores, which are sent directly to the colleges designated by the College Board. Nor could she alter the good teaching that was going on in those AP classes by great teachers. Talk to some to the people at that school, and you will learn what was really going on. It does not change at all the good results of all the hard work by students in a school that only accepts low-income kids. They earned their ranking on our list. (And by the way, check yr facts on the list. We count TESTS, not classes. There is a big difference, as several of the answers above show.)

_______________________

Arlington, Va.: You did clarify to some extent, but could you please explain to the Fairfax poster that HB Woodlawn is an ARLINGTON school, and there is a highly competitive lottery system to get into it? Fairfax was definitely getting HB confused with some other school.

Jay Mathews: Sorry. You are right. Indeed, a student who wanted to attend any of the Fairfax schools listed could not attend HB, unless the student moved to Arlington and put her name into the lottery. Maybe the reader will send me an email and tell me which school in Fairfax she was thinking of. I am willing to bet it will turn out to be a much better school than he or she thinks it is. I have no quarrel with people choosing not to send their children to schools with lots of low-income kids, but I do object to them labeling such places as bad schools, without ever spending any time in them, or looking at their data.

_______________________

Rockville, Md.: Hello-Thank you for you ranking. My question is that your list seems to include specialty schools and prep schools. Is that correct? If so, wouldn't those schools tend to attract more talented kids and pull from a wider geographical area as compared to a typical public high school where, if you live in the district, you're in the school, if you don't you can't go? I would be interested in a list of just the latter. Thank you.

Jay Mathews: Please see my answer above to the person who asked about why we excluded some high-scoring schools. We do list them as the public elites, and describe them on the Newsweek web site. But your point is a good one. We should look more closely at the specialty schools on the list. WE do NOT, however, have any prep schools, at least as I understand the term. That means private schools to me.

_______________________

Reno, Nev.: My daughter and her friend went to Wooster which has an IB program. After two years she and her friend (valedictorian) transferred to another school with an AP program. Their complaint was that the classes had both IB and non IB students in the same class. The classes were dumbed down and separate grading curves were used depending on whether a student in the class is an IB or not.

My other child attended another school with an AP program. The requirement for taking an AP class is that you must pay the AP exam fees. The school does not care whether you actually take the exam and moreover does not release the score distribution or averages but gives weight (adds .1 to one's GPA/semester) in determining class rank irrespective of whether the student has taken the AP exam or how well they performed.

Jay Mathews: How very interesting. Please email me at mathewsj@washpost.com and tell me which schools exactly these were, particularly the school with the weird AP test rules. They don't make sense to me, and are the first of their kind in my experience.

I have also NEVER heard of an IB course that included IB students and non-IB students. I can see where there would be a temptation to dumb down, but if you will read my column link above honor classes which I posted a few questions above, you can read me describe a GREAT AP teacher, Jack Esformes, who had both AP and non-AP students in his government class, and didn't dumb anything down. As usual, it all depends on the teacher.

_______________________

Rome, N.Y.: Would national accreditation of dual credit courses offering community college credit as well as high school credit change your position as to their consideration in the Challenge Index?

Jay Mathews: I would need a system, like AP, IB and Cambridge, that guaranteed the final exams would be long and challenging, with lots of free response questions, and would be written and graded by experts who do not work for the high school system the students attended. National certification would not be enough for me. The data indicate that, on average, dual enrollment final exams are not as challenging as AP. That is a problem for me.

_______________________

Rockville, Md.: Despite trying her best, my 5th grade daughter is middle of the road academically. And that's OK by me because I know she is working hard and is on grade level.

So my question is, what's wrong with doing high school courses in high school and moving on to college courses in college? What happens in college, are they expected to do graduate level work by sophomore year?

Students are already being pushed to do 6th, 7th and 8th grade work in the 5th grade, and my daughter can't handle it. She tells me she feels dumb. And that's not fair.

Jay Mathews: People tend to forget something vital about AP, IB and Cambridge courses, which is important in my answering your good question. These do not substitute for most college courses. They substitute only for college INTRODUCTORY courses. There is a big difference.

It is very helpful to take such courses in AP or IB or Cambridge form in high school because it makes the transition to college--where more than half of students drop out--so much easier. And students who do well in AP, IB or Cambridge can skip the intro course in college and move on to the exciting second and third and fourth tier courses, which AP does NOT duplicate, and which are the true college courses.

The country continues to move forward. My dad graduated from high school in 1929, when he was among the minority of students to get a diploma. He didn't graduate from college, but was still ahead of the game. These days someone like him is at a disadvantage in arranging the kind of life he wants. By the same token, our high schools, most of them, are stuck at a relatively low level of late 20th century instruction that no longer matches the world we live in. Time to move forward, and encourage all college goers to take some of those intro courses in high school.

_______________________

Baltimore, Md.: I have some questions about the quality of the AP exams, themselves. Although I have no doubt of the improvement upon an average public school course, I question if AP test excellence really reflects what students should get out of a college class. For example, I earned a 5 on my US history AP. However, I did not have to write a research paper or read scholarly articles for that class; it was purely textbook and test prep based. Perhaps even more problematic, was the strong focus on political history, rather than the social history that was the focus in universities at that time. Who holds the college board accountable for the quality of their exam? Simply mastering facts and writing an essay should not be all that is asked of America's top students.

Jay Mathews: The college board would, I suspect, love to improve those courses and exams, making them like the IB programs that do require a good long research paper in high schools. But they can't get ahead of the colleges, who set the standards for the intro college courses which AP mimics. All of the flaws in AP you cite are found in the intro college courses that students take in college if they don't take AP. Some good AP teachers do add a paper or other extras to their teaching, but in general, AP has to wait for the colleges--I am talking about the big state colleges that most Americans attend--to get their act together. Some are trying, but it is going to take awhile. Selective colleges have better intro courses, but only about 10 percent of American students get to attend those schools.

_______________________

Bowie, Md.: I am relocating to Ohio, to a part of the state where there are no schools on the Challenge Index. Is it possible for me to find CI info on the schools in the area somewhere? If not, what is the best publicly accessible data that I can use to choose a school district?

Jay Mathews: You can call the guidance department of any school and ask how many AP or IB exams they gave in the most recent May tests administration and how many seniors graduated in May or June. Then divide the first number by the second, and you have their Challenge Index. Some schools come pretty close our 1.000 standard in Newsweek, and may be fine places to attend. But if the number is way below 1.000, you ought to look very carefully before enrolling there.

_______________________

A classroom is only as strong as its weakest link...: That being said, what do you say to the high achieving high school student that is taking an AP course as a way to challenge themselves, but the teacher my instruct at a slower pace because a quarter of the class is there more for social engineering purposes then for their academic prowess? Or perhaps it would be better phrased this way - how can a GT kid or just a really high achieving kid shine, if policy dictates that we are all equal academically, even if individual level of effort is certainly not equal?

Jay Mathews: I would complain to the department chair and the principal. If that doesn't work, I would get a book of old AP exams, check out the questions, and either read ahead in the course or do some of your own reading, feeding off the questions you see in the exam. And I know some students, faced with mediocre AP courses, sign up for online AP courses, which turn out in many cases to be very good, and can be done at your own pace. If you are that quick, you can do two or three. Email me on this. Am I right to assume this is a course that does not require everyone to take the AP test?

_______________________

St. Louis, Missouri: Have you come across many (or any) schools on the list that do not offer AP classes but have students who take the AP tests? Our high school claims they do not want to be "restricted" by the AP curriculum so they offer what they think are superior "honors" classes. I find this to be a bit strange.

Jay Mathews: Actually, there is a group of about 50 schools, calling themselves the Excellence Without AP network, that do just that. Google that title and my name and you will find some of my columns on the group. They are almost all very small and very expensive private schools. And they can, because of their high-achieving clientele, provide AP level courses without AP. But as you suggested, it would not work for most schools. A honors course that said it taught at AP level would work for me if it required all students to take the AP exam, but if that is the case, why not call it AP? A "honors" course in a regular school that blows off the exam risks being dumbed down, even if it promises otherwise.

_______________________

Winston-Salem, N.C.: I think a point that seems to get lost in the arguments over whether to rank and why school X may be better than Y despite their ranks, is that regardless of rank, a higher score indicates more opportunities for the best student in a given school. With all the emphasis placed on SOL and No Child Left Behind, is has ironically been the top students that have been left behind. From a macro view, the education system and the country will only thrive when the best students are challenged and allowed to excel. Perhaps rather than complaining about how student are being pressured into taking AP course, prepared or not, we should be figuring out how to make the classes sufficiently difficult to challenge the best students. Thanks for your work. It's much appreciated.

Jay Mathews: Bless you. You have made my day. One sign of a great school, in my view, is a lot of complaints about AP pressure, which among the good students of my acquaintance is a way to bond together, just like I bonded with my fellow draftees by complaining about the basic training drill sergeant. Our real problem is, as you indicate, in most of our schools AP pressure doesn't exist, or any other academic pressure of any sort for that matter.

_______________________

Bethesda, Md.: It is my understanding that Fairfax County Public Schools pay for their students' test fees. Is this true? If so, it seems like yet another variable to contend with in this ranking system.

Jay Mathews: They do, as do most of the other school districts in Northern Virginia. I would penalize them for that if I thought requiring all AP students to take the test, and paying the fees, was a bad thing. I think quite the opposite. It is a great policy that ensures every AP student will get the full dose of college trauma that they need, including the three hour exam. So I applaud such schools and hope the Challenge Index motivates more schools to adopt the same policy, as do many of the successful AP teachers who have led me to think this way.

_______________________

Rockville: Jay, Thanks for doing this Challenge Index - it is a great tool for parents and students. From the "local" perspective of the DC area, what high schools most impress you? For instance, is RM's program (at 32) markedly better than Churchill, or Howard County's (arguably) finest, Centennial?

Jay Mathews: Howard County is changing, but for a long time even their best schools on the list were restricting access to AP. I have to visit Centennial before I say anything about it. RM is a wonder, a school so challenging that if the kids in its IB magnet program all suddenly transferred to another school, the remaining non-magnet kids would still be taking enough AP exams to get the school on the Newsweek list. I have many favorite high schools, particularly those that do the best job in challenging low income students, and giving them the extra time and encouragement they need to meet those challenges.

_______________________

Savannah, Ga.: When the AP tests are mandatory, is financial assistance provided?

Jay Mathews: Usually, but there are some schools--I wrote about two in New Jersey last week--who do not pay the fees except for students with a real need. Here is that column:

http://www.washingtonpost.com/wp-dyn/content/article/2008/05/12/AR2008051200496.html

_______________________

Baltimore, Md.: With an increased emphasis on AP exams that this ranking system encourages, it seems as though high school teachers too, would be hampered by the same low expectations of introductory classes at major public universities that you say limits the College Board. One of the bonuses of your ranking system is that it encourages schools to open up their AP classes, something I fully support. However, one of the weaknesses of your system, is that it undermines strong non-AP classes, which perhaps exceed the mediocre expectations of huge college introductory classes. I believe that this is why private schools, where such classes abound, choose not to participate in your survey. I wish there was a more nuanced way to evaluate the strength of schools -- one that showed the actual work of the students during the year.

Jay Mathews: I share your wish. But notice that the vast majority of private schools have embraced AP, even as they complain about it. Any way to produce good teaching for our kids is fine with me. But in this stage of history, for most schools, it does not happen very often without AP, IB or Cambridge. The pressures to dumb down are just too great.

_______________________

Clarksville, Maryland: As a parent, I have to agree with the teacher from Richmond. My children attend a Blue Ribbon HS and the entire focus of the school is on AP classes. They are assigned to the best teachers so many students are pressured into enrolling, even though it's not in their best academic interest. Contrary to popular thought, not every kid is a genius and can handle 5 AP classes at once. These lists are more for bragging rights between administrators (and parents at the grocery store.) There is absolutely nothing wrong with taking regular or honors level classes. These kids have much less stress and will succeed just fine. My daughter is now a sophomore in college, excelling in her program, and never took an AP class.

We hear more and more about overly scheduled teenagers who are on the verge on breakdowns. No wonder substance abuse has continued to increase. Those of us who were in high school 30 years ago did not have the pressures of high stakes testing and have done just fine. My husband and I both attended nationally ranked universities and have careers that are professionally and financially rewarding.

This isn't Hollywood and Mr. Escalante was truly a hero for the kids at Garfield HS. But that was one highly committed educator working on a single test. In reality, our kids don't have the luxury of focusing on just one subject. There is nothing wrong with challenging kids, but at what - and whose - expense??

Jay Mathews: You make a good point for those of us in the top 10 percent of the country in income and education. We are going to set standards and expectations for our kids that will put them on just the right course. They will attend selective colleges and do great things, having had those advantages, whether they took AP or not. They will also suffer stresses, but I think often such stresses are exaggerated, and the source of the stress is not AP classes but we parents. Our kids see us choosing to lead stressful lives, and in many ways liking it. So they say, I want some of that. Usually no harm, but we have to be careful that our kids don't overdo, and not try to blame it on AP.

To return to my main point, what about the 90 percent of families who do not have these advantages? What about their children? Their high school reading and math scores have seen NO improvement in 30 years. Those kids, on average, spend 2 hours a day watching TV and only one hour doing homework. Their high schools demand little more. Don't they deserve a bit of a challenge in their school day?

There are lots of Jaime Escalantes out there now in many schools, proving that he was not a fluke. Anything that encourages their work, as the Newsweek list does, is a good thing.

_______________________

Richmond: I have a thought about the 1st comment. Perhaps the problem is that the way some schools structure classes prevents the teacher from helping students learn at their own pace. If the teacher is pressured to constantly move through the material, a student who gets lost will probably never catch up. Struggling under those circumstances is no good for anyone. No wonder the students don't like math or feel they can't do well. I don't think most public school teachers are taught to teach this way.

Jay Mathews: Bingo. Absolutely right. A lot of kids, as I said above, are smart, but need MORE TIME and more encouragement to learn. That was one of the most important things I learned from Jaime.

_______________________

Re: Poolesville HS: Poolesville High School is the only "Whole School Magnet" in Montgomery County, Maryland. Should it be on your list since not just anyone can attend and therefore there may not be "average" students?

Jay Mathews: It is not a Whole School Magnet. The magnet there is terrific, but many of the Poolesville kids are there because it is their home school, and do not participate in the magnet. Montgomery County does not have a whole school magnet.

_______________________

Newark, Del. again: Makes sense -- I guess my definition of "magnet" is higher than those who are naming schools. Thanks!

Jay Mathews: You are most welcome.

_______________________

Fort Lauderdale, Fla.: I was intrigued by your index results. Having grown up in Michigan (with close ties there still) and living in Florida, I took a quick peek at those two states' results -- and was baffled to see the stark differences in the rankings. A couple of the best schools in Michigan, that I have passing knowledge of, rated in the mid-300s, while many Florida schools were rated within the "best" 100. Two of our local, "urban" schools fared as well as schools in the tony suburbs of Detroit. No school north of Lansing even made the list.

My first thought was "Gee, our local schools are doing far better than I imagined". After looking at your methodology, however, it became clear that Mark Twain was correct: "There are three kinds of lies: lies, damn lies and statistics."

As any fourth-grader should know, a larger numerator will increase a fraction, as will a smaller denominator. Your index is skewed in favor of schools with larger numbers of attempts at AP tests, as well as those with lower graduation rates.

Wouldn't a measurement that is based on the number of tests passed divided by the size of the incoming freshman class be a better indicator of a school's true excellence? At least it would measure actual performance relative to the complete "market" the school serves.

While Americans love lists of the "top whatever". I would think that state-to-state comparisons are complex at best (and more likely, impossible) due to differences in the different incentives given for test-taking. The Florida FCAT process puts inordinate emphasis on standardized testing -- with school districts and teachers compensated, directly or indirectly, based on test results. I'm not sure if all states have the same types of policies and incentives.

This index seems to be a case of giving an "A for effort", which is in keeping with the current culture of "every participant is a winner".

Jay Mathews: Good question. This is part of a long debate I am having with Andrew Rotherham, co-director of the think tank Education Sector. If you google both of us together you will find some of our discussions. Andy agrees with you. A school with low test scores and high dropout rates should not be on a top schools list. I argue that such schools have those problems because they have so many low-income students. We as yet have no proven solution for those low scores and high dropout rates in such schools. Some of those schools have great principals and teachers, but they have little power to shift the burden of demographics. However, they DO have the power to bring AP or IB to many more students, as the Garfield High teachers did, and change a lot of lives among those kids who won't drop out and will be in the upper range of those low scores.

Your way means we have to assess schools the old way--if they have a lot of low income kids, they are bad. If they don't they are good. You are not assessing the school, but the demographics of the student body. The Newsweek list assesses schools in a better way--which ones have staffs that are trying hard to raise the level of every child, by introducing more challenging instruction? You might try reading my book Class Struggle to get the argument in full, but I am convinced it is better than yours, based on conversations with many teachers who have made those schools work for many kids.

(PS, one of the benefits of dividing by the number of graduating seniors is that I am rating those urban schools academic efforts against just the number of students serious enough to graduate, not the kids who rarely show up or just roam the halls. If you can find an urban school of normal size that has solved those problems, I will write about it and adjust my method. Until then, I think this way of looking at schools is the most accurate reflection of the quality of the teaching, not the size of the parental bank accounts.)

_______________________

re: undermining non-AP classes: Even if districts are just trying to improve their Challenge Index rank, if they are forward looking this means improving the preparation in early grades that gets kids ready for AP classes in HS. That would be a good thing. Then even kids who don't take the AP class would have gotten better training in early grades. I do worry that many districts are not very forward looking, or don't know how to make these improvements. If it was easy we'd all be there already.

Jay Mathews: This is a very wise message. Thank you.

_______________________

Re: Poolesville HS: Here is the link to Poolesville High School. http://www.montgomeryschoolsmd.org/schools/poolesvillehs/

It says right in their Mission Statement that they are a "whole school magnet". This is a recent change I believe.

Jay Mathews: Thanks for getting back to me on this. It is, like many things, a matter of definition. You are quite right. Poolesville calls itself a whole school magnet. I just spoke to magnet coordinator Billie Bradshaw who explained that this means all students in the school, even the ones who came to Poolesville just because it is their local school, have the CHOICE to join the interdisciplinary academic program that makes up the magnet if they want to. (They don't have much choice in the 9th and 10th grades, but do in the 11th and 12th grades if they want to opt out of the fancier classes.) That is different than the county's other magnet, so I endorse their title, and will be more careful in the future when I discuss it, because of your good messages.

_______________________

Annapolis, Md.: I like the Challenge Index. Not because I think it produces better students or schools, but because it encourages schools to open opportunity for every student. Those who advocate restricted AP classes might not have thought of who the gatekeepers are, and who gets shut out. My own experience in Hampton Roads in the late 80s was that kids identified as GT in 6th grade were assumed to be smart right through high school graduation. They didn't have to do anything special to get into AP or honors classes in high school, and they surely weren't always "qualified". The students who hadn't been tagged in middle school had an awful time getting into good classes in high school. The practical consequences were that our AP and honors classes were filled with white kids, some of whom weren't interested and didn't work for it, while only a few black kids were allowed in. (At the time, there were very few Hispanic students in the school, and I don't remember any in the AP classes.) Many of us black students had similar experiences of tussling with the guidance counselors to let us into the classes we needed to take to be competitive for college, all while being told that they were just looking out for us. Apparently, they were afraid we would fail, and wanted to protect us from that.

Whatever their motivations really were, I'll never forget that my mother had to call the superintendent of the district to get me into calculus, because the counselors and teachers just didn't want to believe that I could succeed in the class. As it happens, Calculus was my worst AP class by far, but it certainly helped me to do a lot better in college, where I had to take many advanced math courses for my engineering degrees.

Jay Mathews: What a terrific message. And as you know, sadly, most high schools in this country still follow the system you describe so well.

_______________________

Rockville, Md.: I will say one thing is depressing me. I live in a community where the north side goes to Gaithersburg and the south to RM. People are so obsessed about getting into RM that they ignore the fact that Gaithersburg is perfectly fine. RM 32, Gaithersburg 652 I think. Sounds large but that is still the top 1000, and out of many thousands of schools. It is a matter of scale and when compared against 10000, the difference between 32 and 652 is oh so small. One more way of competing I guess.

Jay Mathews: You are so right. I give the same lecture about the ranks. Gaithersburg is in the top 3 percent of US high schools. That sounds very good to me. If I didn't rank we would not have this problem, but if I didn't rank, no one would pay attention to this issue. We are tribal primates, our DNA wired to look for pecking orders, and we journalists know we have to rank or we won't get published, and no one will read what we have written on such obscure topics as AP courses.

_______________________

Falls Church, Va.: Do most kids who have taken the APs and placed with a 5 skip the intro classes when they get to college, or use their prior experience as a way to ease the overall transition to college and take the intro again? My son is graduating from TJ with a host of these things, and I am not sure what to advise him.

Jay Mathews: Most 5s earned by students at TJ and other schools are in subjects that the student doesn't plan to pursue in college, and thus will have no need to skip. The social studies types like my kids are not going to take any more hard sciences, even though they took AP physics, bio and chemistry. The science types are not going to be doing much more explorations of the college English departments, even though they often took AP English. They will often use the credit to advance to the next level in their majors, but if you have looked at college curricula lately, many of the courses are electives that do not fall within the ranks of the major course sequences from which AP derives its intro courses. And since your kid went to TJ, he is totally on top of this. I am sure he, like my kids, will listen politely to your advice, but like them he really doesn't need it.

_______________________

Cardozo High, D.C.: I'm just curious exactly how some of these high schools made the list. I live across the street from Cardozo, and was appalled to read last summer's education report in the Post about the state of schools in DC. They highlighted students at Cardozo and touched on all of the academic and funding problems the school had. How does a high school like this make it in the top 1300 high schools in the whole nation? Is it that our education system is so poor that a school with so many problems like Cardozo, can make it, or is it because the methodology of your study somehow allows for a school like Cardozo to make the list?

Jay Mathews: I have spent a lot of time at Cardozo. It makes the list because it has had for several years a core of AP teachers who absolutely refused to give into the low standards and hopelessness that plague most DC schools. They nurture kids, give them extra time and show them some of the research, reading and writing skills they will need in college. Cardozo English AP teacher Frazier O'Leary is a nationally known expert on how to do AP in our most disadvantaged schools. As all such schools, they endure many setbacks---well described in our recent stories. Most of their kids do not pass the AP exams, but their scores have been going up, and the educations they get at that school are clearly better than at other DC schools that do not take AP very seriously.

_______________________

Arlington, Va.: In terms of underqualified students taking AP courses... maybe what teachers need is more/better training in how to offer courses that BOTH challenge well-prepared students AND support poorly-prepared students. I am a teacher who faces the problem of teaching students who, as your first writer said, "have no business being in AP," but at the same time I believe STRONGLY that all students should be offered the chance to take AP courses. It is up to me to figure out how to support all students, without calling attention to the huge gaps in their readiness/"ability". I don't always know how to do that. I have seen teachers take the "sink or swim" approach, and don't believe it's the best way to handle the situation.

Do you know of any good training for public school teachers that addresses this issue? The AP trainings I have been to, run by the College Board, have been all about the curriculum (as is appropriate!), and not about teaching it to underprepared but dedicated students.

Jay Mathews: What a smart comment. I will copy it and send it to AP director Trevor Packer, who is pushing for more such training and has found a great educator, Mike Riley of Bellevue, Wash., to lead that charge. I don't know of any specific training that does better than others. I have found the best training comes in schools like Wakefield, right there in Arlington, that has been a leader in teaching AP to low income kids. Email later this week and I will tell you what they said.

_______________________

Arlington, Va.: Wow. I think you just showed your true colors: "I am rating those urban schools' academic efforts against just the number of students serious enough to graduate, not the kids who rarely show up or just roam the halls. If you can find an urban school of normal size that has solved those problems, I will write about it and adjust my method. Until then, I think this way of looking at schools is the most accurate reflection of the quality of the teaching, not the size of the parental bank accounts."

What about the kids who roam the halls and rarely show up? Oh, yeah - conveniently, they drop out, so they don't count towards the Challenge Index, so you get to write about how some urban schools are just so wonderful. Now I understand just how perverse this index is -- a school with a high dropout rate but who can push a few AP tests at the few remaining is a great school? What about those kids who never even finished? They didn't even get a high school diploma! And those schools are considered successful? How about being honest -- publish next to the statistic about what proportion rate a free lunch the dropout rate for the school (e.g. what proportion of kids from the freshman year didn't make it to graduation). That would be a start at some honesty.

Jay Mathews: I am trying to be as honest as I can. I just posed the question: do we have a solution for those dropouts that the great teachers in these schools can apply? I would be very grateful for your honest answer. If you know of one that has worked in an average inner city school, tell me about it and I will write about it. In the meantime, I don't see any point in rating schools on a game they cannot win. It is like blaming the residents of Florida for having a lot of hurricanes. Someday we may learn to control bad weather, but at the moment it is out of our hands, as is the high dropout rates in such schools. We are making a LITTLE progress, enough to keep trying, by starting smaller and more intimate inner city high schools. But the teachers and principals at the schools we have now lack the power to make their schools smaller. So I think we should judge them on jobs they can do, like getting more kids into AP, and hope the school boards that have the power to change the structure of schools start working on that.

_______________________

Rockville, Md.: With US News now out with a High School ranking, how do you feel their ranking stacks up against yours? What are the main difference between the Challenge Index and what US News uses?

washingtonpost.com: America's Best High Schools (usnews.com)

Jay Mathews: They are new, and well-intentioned, but so far we are way ahead. They can only do ratings in 40 states because they cannot get the top-down state data they use from ten states, plus the district. They don't give any credit for IB. They are trying to rate schools from a hundred miles up, using state data. We get our data from each school, or school district, in all 50 states and the District. That means we can present a more individualized sense of each school, and show the quality of some inner city schools that US News writes off--my friend Andy Rotherham advised them--because they have low test scores and high dropout rates.

_______________________

Anonymous: I would be curious to know if the 50% of the students in Richmond's AP Calculus teacher's class are quietly absorbing the material or if because they are overwhelmed they disrupt the teaching to the prepared students. If so, I might prefer to send my children to a school which ranks lower on the list.

Jay Mathews: Good questions. I hope he or she writes and tells me more. Email me latter in the week and I will let you know if I got more info.

_______________________

Gaithersburg, Md.: If a school ranks high on the Challenge Index, that could possibly mean that there are students in the class who aren't prepared to work hard, since the index doesn't indicate whether the student even passed the test. Administrators who want their school to go further up the list could be encouraging students who aren't even prepared for the classes. Is it possible to do a second ranking that shows the percentage of students who take an AP class and actually pass the test compared to the entire number of graduates? I think both indices would be helpful. Thanks.

Jay Mathews: Check out the Equity and Excellence percentages for each school on the Newsweek list. That shows you what percentage of the senior class got at least one passing score on an AP or IB test some time in high school, and is a good general measure of mastery and participation.

But let me comment on the beginning of your comment---very few AP students fail to try. They generally do their best, because they know the courses are important, although not all pass the exams. It is the trying that builds academic muscle for college. That is why we count tests, meaning they went the whole way, and didn't avoid the exam, as many do.

_______________________

Washington, D.C.: Have you ever thought of doing (or helping someone else get the support to do) a ranking of US high schools on other criteria that might be more broad measures of how much they challenge their students?