Friday, Dec. 12 at Noon ET
Jay Mathews: Challenge Index
Friday, December 12, 2008; 12:00 PM
Post education writer Jay Mathews was online Friday, Dec. 12 at noon ET to discuss his 2008 Challenge Index rankings of area high schools.Challenge Index: 2008
The transcript follows.
Contribute to Jay's discussion group: Admissions 101.
Jay Mathews: Good morning. I can't think of a chat I have more eagerly anticipated. Some of my best ideas come from readers, particularly those that think I have gone bonkers. Please don't be shy about exposing me to your views. I am in a particularly good mood because I am sitting in my brother's home in San Mateo, Calif., after having visited yesterday with my first grandchild, Ben Mathews, age 2 days, and his proud parents. The future looks bright, including a chance to chat with you today. As always, if I fail to get to your question, please send it to me individually at firstname.lastname@example.org. It may take me a while to get back to you, but I will respond. You may see me below ask you for more info. Please use my email address to do that. And if you prefer to call, my number is 703-518-3012. Leave a message if I am not there. I check phone messages obsessively, even when on vacation.
Arlington, Va.: For the record, I think the Challenge Index would benefit from an asterisk explaining that HB Woodlawn in Arlington, Va. is a lottery-admission program, not technically a school, with just 76 students in the senior class. It's a great program for many kids, but it's not really comparable to a neighborhood high school that serves thousands.
Jay Mathews: It is an interesting idea, but I am reluctant to do that for two reasons. The first, minor, reason is that, as you can see, we already have a lot of asterisks to denote what kind of college-level courses each school uses. The second and by far the more important reason is that I don't accept your premise. There are plenty of neighborhood schools that are JUST like Woodlawn, or in some cases have demographics that make them even more likely to have lots of AP and IB test takers. Consider that despite being a school of choice, with of course that lottery factor to keep it from loading up with just the children of the aggressive parents who get in line first, 13 percent of Woodlawn students are from low income families. You will notice that that is a larger portion of low-income kids, who traditionally are less likely to try AP or IB, than half of the other two 20 schools on this year's main list. Secondly, its small size is an advantage, and the school has created a family culture that makes great use of that advantage, but there are other schools in the area of similarly small size that are NOT schools of choice. The best local example is George Mason High (with a 7 percent low income student population) in Falls Church. Across the country, you will find many other single high school districts in affluent neighborhoods who have the same culture as Woodlawn simply because of the small size of the district and the high cost of the houses. They are essentially schools of choice, and harder to get into than Woodlawn because you got to have the dough to live in a district like Scarsdale. Should I put an asterisk next to their names too? I think the demographics of each school is a more important influence on what kind of school it is, and how hard the teachers and administrators have to work to build the college-level program, and the lunch subsidy percentage, which we do give for each school, is the best measure of that.
DC: What does it say about a school such as Bowie HS in an "upscale part" of PG County that has a decent Challenge Index score, but a gosh awful HSA passing rate where kids are having a hard time passing basic algebra, government, biology, and English?
washingtonpost.com: Bowie High School (greatschools.net)
Jay Mathews: It means the administrators and teachers have done a good job of providing challenging courses, but have to work much harder to get the level of mastery up at all grade levels. If what you say is true---I don't have the Bowie HSA data in front of me---I think it is pretty embarrassing for a school with only 17 percent low-income students to have trouble meeting fairly modest state standards. They have to set a higher standard for all students starting with ninth grade, and stick to it. Most American high schools don't do that, so Bowie is reflecting the majority view that high schoolers should not be pushed too hard. But the fact that they have a lot of AP courses is a good sign, and they should now focus on making sure those kids are ready for the AP exams, and everyone is ready for the HSA exams.
Washington, D.C.: I really appreciate your changes to the Challenge Index. Not only did you resolve a serious flaw in measurement, but you also celebrate growth and effort - something that any high-needs educator must do to promote effort-based achievement.
Rather than question the quality of AP/IB tests, I wanted to ask you to give your definition of two words that frequently appear around Index discussions: "challenge" and "rigor."
Jay Mathews: Thank you for your kind words. I am anticipating that that will not be the majority view, but this is just the start of the chat, and because of the arrival of Ben I have not yet read any of my email since the lists came out. I love your request for a definition of those two words. Nobody has asked me that question before. I would define challenge, in this context, as the ingredient in a course that entices and pushes a student to a new level of achievement, and this is different for each student. An algebra I course is challenging for most 6th graders. It should not be challenging for most 11th graders. The course should stretch the important muscles that lead the student to be able to write clearly, analyze intelligently and come up with his or her own ideas about the material being learned.
Rigor to me is the ingredient in a course that makes sure, once a challenging standard has been set, the students have many opportunities to reach it. It means a teacher who turns a class session into a deep conversation, calling on every student to respond to issues and concepts being taught. It means a fair amount of homework (more than the 50 minutes a day that is the average total homework for American high school students) and frequent writing assignments, among other things. Do your definitions differ from mine? Some writers I respect, like Alfie Kohn, think rigor is a much-misused word to denote educational quality, and would not like my definition, but that is why I like reading Alfie, to keep my mind sharp. He gives my life some needed rigor, and challenge.
Winter Park, Fla.: Arguably the strongest feature of the Challenge Index is its simplicity. (Compare it to the byzantine formula USN&WR uses for colleges!) However, this is possibly its greatest weakness--it's easy to game a simple formula.
There's been some evidence of this happening in several districts down here, with students being pushed to take AP tests they've not been prepared for. (And, at least as I see it, the preparation for the AP test is the most important part.) What do you see as the best way to combat such gaming of the system--or is there really any way to get around it?
Jay Mathews: Many people agree with you that the simplicity makes it easy to game the system and get on the list, but I don't agree. I have thought about this important issue you raise, and watched carefully for signs of it appearing, since the list began ten years ago. I address this point in my Class Struggle column this week, which you will find on this web site if you search for my name.
Many people think, for instance, that many of the schools that I put on the new Catching Up list were gaming the system by shoving kids into AP classes even though they were not ready for them. I argue in the column that that is not true. Those administrators were desperate for a way to get their very low standards up, and figured out that even if those kids struggled in AP and flunked the AP exams, they would be much better off than with the non-AP courses the school could come up with on its own. The fact that those students are in a college level course with an incorruptible exam makes a major difference, as long as they all take the exam so that they and the teacher are shooting for that difficult target.
Theoretically, you could imagine a principal declaring that everyone in his school was going to have to take AP and the AP tests, but telling his teachers to just give those kids the same old thin gruel and don't bother to stress themselves to try to prepare those students for the exams. My view, based on three decades interviewing and watching high school teachers and principals, as well as parents and students, is that such a scheme would produce an immediate revolt by students, parents and teachers. That principal would be out of a job. It does not take parents and kids long to figure out the value of AP and IB, and I have never seen a case where they let a principal mess with it like that.
Washington, D.C.: Why are AP exams so important? Do we really need our kids entering college with Sophomore/Junior status? They will be working 40+ years, wouldn't it be nice to spend 4 years in college? I took (and passed) 2 AP exams in high school, but repeated those classes in college. The depth of learning could not compare. High school AP classes teach you how to pass a test, a real college course should teach you how to think.
Jay Mathews: I could not agree more strenuously with your last point. There are some weakly taught AP courses out there, but in most cases--if you spend the time to look at them carefully--you find they are BETTER than most of the college intro courses they are designed to replace. Remember, we are not comparing them to average college courses, which are by definition better, but average college INTRO courses, which are supposed to be identical to AP and IB, but are in most cases inferior. Look at the standard state college intro course---two lectures a week in a hall with 200 other students, one session a week with a TA who has little teaching experience, an exam that is often just multiple choice questions. Compare that to an AP course with just 25 students meeting every day with an experienced teacher who is MUCH better at teaching this introductory material than the average TA, or college professor for that matter. Compare the 3 hour AP exam, with strong emphasis in the exam scoring on the free response questions, with the standard 2 hour college intro course final, and you will see what I am getting at.
As for the need to get soph or jr status, almost NOBODY uses AP or IB to graduate from college early. I agree entirely with your premise--who would want to do that? Only kids in extreme financial difficulties would see that as a good option, since they would not have to pay the tuition for that 4th year. What AP and IB do most importantly is give high schoolers a taste of college trauma so they are ready for those college courses in a way many college freshmen are not. If they wish to take the college's intro course after taking the AP course, that is fine. That will give them a good solid grade to start their college career. If they did well enough in AP or IB to jump past the college intro course their freshman year, then they will have more room in those great four years to explore more topics.
Let me know if you can where you went to college. I think that the top ten percent of colleges in selectivity do offer intro courses that are better than most IB or AP courses. Those schools can often afford to have full professors teach intro courses of 25 to 30 kids, with no TAs, and that is a treat. But for most American college students, the intro course experience is not like that.
Anonymous: That principal would be out of a job. It does not take parents and kids long to figure out the value of AP and IB, and I have never seen a case where they let a principal mess with it like that.
Then you haven't looked closely at Locust Valley, NY (LVCSD) where we have had 4 Principals in 5 years, ALL of whom bowed to the Board of Ed who considers your List such an honor that it has even dedicated a page on its official district website. This despite the FACT that last year, LVCSD had a 0 percent pass rate for AP U.S. History, and this year a 10 percent pass rate.
Jay Mathews: Hi Lisa. Dear Readers, this is one of my favorite email correspondents, who has failed to tell you that Locust
Valley is an IB school, where the passing rates on those exams are far above zero. They give AP classes in only a few subjects, as a warm up for or supplement to, IB, which many schools do. They should do a better job in teaching those AP courses, as I have told Lisa, but her local high school does NOT have a principal shoving kids into college-level courses and telling the teachers not to teach to the standard. If he or she did, Lisa would lead that revolt and she would win. She lost her last race for the school board, but if she had such an issue on her side, she would win in a landslide.
Alexandria, Va.: Thank you for doing the Challenge Index each year, and especially for making the data easy to sort. My son's school (Hayfield) moved up slightly in the region's rankings, but was the lowest in the county on the pass rate. My sense from him is that there are a small group of kids who take a full load of AP classes in both their junior and senior years. The pass rate gives a more accurate indication of how broadly AP reaches throughout the entire student body.
Jay Mathews: That's not the passing rate, 30.9 percent, but the equity and excellence rate, the percentage of all seniors who had at least one 3 or above on an AP exam sometime in high school.
Fairfax parents make themselves crazy unnecessarily comparing their schools to other Fairfax schools. It is like having your kid start for the Dodgers and be unhappy because he doesn't make as much money as Manny Ramirez. Fairfax County has probably the strongest AP and IB program for any large district in the country. That 30.9 percent is twice the national average. Hayfield's challenge index rating puts it in about the top 3 percent of all US high schools. If I were you I would be very happy with that.
Proud 1992 W.T. Woodson Graduate: Please explain, so that someone might sit up and notice, why the Fairfax County school system consistently ranks in the top tier and the DC school system does not. By the way, I briefly served as a substitute teacher at the Model Secondary School for the Deaf on the Gallaudet campus and the quality of the education there would give you the blind staggers. Perhaps a future story for you.
Jay Mathews: How interesting. Gallaudet was amazingly good or amazingly bad? See my email address above. Tell me more.
As for yr good question, it is mostly a matter of the disadvantages of poverty, which produces a culture, both in the families of the students and the politics of the district, which does not encourage learning at a high level. Although it is important to say that Fairfax had the dumb luck--that is the only factor I can figure out is different in its case---to have had some very smart school boards and superintendents in recent decades who opened AP and IB to all and supported that decision in ways more other affluent districts have not. And in the same way, DC policy makers have been among the worst, compared to other urban districts, in trying to change the culture in their schools. Although in my view the latest mayor has truly made this a priority and is moving in the direction that other, better, urban school systems have moved.
Bayville, N.Y.: "Across the country, you will find many other single high school districts in affluent neighborhoods who have the same culture as Woodlawn simply because of the small size of the district and the high cost of the houses. They are essentially schools of choice, and harder to get into than Woodlawn because you got to have the dough to live in a district like Scarsdale. Should I put an asterisk next to their names too?"
Scarsdale? You're going to cite Scarsdale, which threw away perfectly good AP offerings for an esoteric feel-good curriculum that Scarsdale alone claims is superior to all others? And yes, for that privilege, to buy a home or rent in the Scarsdale area you need to make a good living. HOWEVER, the Scarsdale public schools must accept any child whose residence is within the Scarsdale district boundaries! There is no "lottery", no "admissions test".
I've said it before and I'll say it again - MAKE TWO LISTS - one for GENERAL public schools and the other for magnets and charters. They are two distinct and separate animals.
Jay Mathews: And yet, without that admissions test or lottery, they get a mix of students who are in my view even better prepared and eager for AP or courses like that as are the kids at Woodlawn. Scarsdale, for instance, has a higher passing rate on its AP exams than Woodlawn does. I agree with you that Scarsdale's switch to its new non-AP courses was a waste of time and effort, but it really didn't change anything. The Scarsdale faculty is deeply invested in the change and will not do any research that compares what they did under AP and what they are doing under the AT program, but such a comparison I am convinced would show that kids are in each case getting the same fine teaching, and notice they are still taking the AP exams and doing well.
It is extremely helpful to have AP courses under one's belt. I took the AP calculus BC class/exam in high school. I got a 2 - which meant that something on there was correct. I had no idea what was going on in that class the whole year! So I thought...
I got to college, and basically, that AP course had covered 2 semesters of calculus. I sometimes went to class, but if I planned not to go, I would do the homework the evening before to ensure I knew what was going on. Since the actual class wasn't necessarily helpful (as you said, hundreds of students...I think 300 or more in my classes - no TA taught smaller class) - the AP experience was awesome.
---bachelor's and master's degree in math...
Jay Mathews: You have added a very telling story to this discussion. Thank you.
Alexandria, Va.: Congrats on your first grandchild.
My questions deal with Coolidge; was the increase due to more students enrolling in existing classes or were more AP subjects added? What was their passing rate last year (when either fewer students or fewer AP subjects/courses)?
Do you think money is better spent at this level (high school and open enrollment AP courses) or at the elementary and middle school level, preparing kids when they are younger for challenging high school courses?
Shouldn't remediation occur when kids are younger, rather than in high school where they might end up still struggling in college (ties in with your recent post on grad rates in college and the need for tutoring there).
Jay Mathews: A great question. The Coolidge passing rate last year was the same low figure it was this year, just 2 percent, when it gave just 109 tests rather than the 750 it gave this year. The increase this year was due to both factors, more courses and more kids taking the courses, but the latter factor was by far the more important. They really made AP a priority for all.
As for what level we should spend our money, keep in mind it is not more spending that is most important (although it will help) but better leadership that produces better teaching. And it is clear if you get that in the lower grades, you have much bigger impact than if you wait until high school. The KIPP charter school people, who have produced the most successful inner city schools in the country (commercial alert: look for my new book about them, Work Hard. Be Nice., just $11.50 on Amazon), have started to build preKs and elementary schools. Their results are so good they are saying this will allow them to get great results in their middle schools with ordinary mortals as teachers, not the hero teachers that have been doing such good hard work the last few years to raise kids from such a low point.
Mount Vernon Farms, Va.: 1. Is it possible that the original point of the AP program to give high school students a taste of college, and not to have as many students as possible complete a semester's worth of credits before arriving?
2. Can you comment on the phenomena that a tool you designed to measure educational success is now driving educational policy?
Jay Mathews: The original point of AP was to keep college freshmen who had graduated from very exclusive private schools or affluent public high schools from getting bored in their freshman college year, taking intro courses that covered material they had already learned at their very good high schools. As the program spread, getting the credit became a selling point. These days I think the most important point about AP is your number one point, but sadly the vast majority of high school administrators don't agree with me, and still tell C students who are going to college and could use that experience that they will not be allowed in AP because their grades aren't good enough.
2. Your premise is wrong, but don't feel badly about it. It is hard to believe that any major publication, like Newsweek and the Post, would allow a reporter to design a tool not only to measure success but to drive educational policy. But it is true. I made it clear from the beginning that I thought the information in the list was useful to all kinds of readers, but I particularly wanted policy makers to look at it and change the insane policy of keeping average students out of AP. That intent made sense to my editors at both publications. I was writing oped and columns on this so they were used to me spouting off in that way. I think it has had some impact, at least as support for AP teachers who have been fighting the policy for years before I came along, but as I said above, in most places that policy has not changed.
Washington, D.C.: What makes a top-flight elementary school?
Jay Mathews: You need first and foremost a great principal who was once herself or himself a great classroom teacher, and knows how to recruit and train more great classroom teachers, and has the power to do so, and get rid of those teachers who fail to improve after being well trained. Unleash someone like that on any elementary school and you are going to have a top-flight institution.
West Falls Church, Va.: Wouldn't it be better to rank schools based on the Equity and Excellence figure rather than the Challenge Index figure since the latter can obviously be artificially elevated.
The E&E calculation is a better measure of success. In fact, how would the rankings change? Does your online posting allow sorting based on the E&E calculation?
Jay Mathews: That day might come. But for now it is too difficult to calculate accurately. The College Board sends the number to each high school based on its data, but they have to base the number on numbers a clerk put on an order form and they are often wrong. IB doesnt calculate it at all, and if you have both AP and IB, it is difficult to come up with the number. Those problems are fixable, but the remaining issue is that the number would be far more influenced by demographics than the Challenge Index rating. Poverty keeps test scores low. We don't have anything to tell teachers to do that will cure that ill in any significant way, yet. But those same teachers CAN raise the level of challenge in their courses simply by letting all kids who want to work hard enroll in college-level courses. You might call that "artificially" elevating their number. I call it a wise policy that will significantly improve the academic skills of those kids, even if they don't pass the exam and don't raise their school's equity and excellence percentage.
If I did rank by E and E now, the list would have far fewer schools at the top with high percentages of low income students. Schools like W-L, Wakefield and Fairfax High, are in the top 20 of the new list. Their E and E percentages are in the high 30s and 40s, very good, but they do not match the affluent schools with E and E ratings in the 60s and 70s, and thus would fall out of the highest rungs if we measured your way.
Jay Mathews: Thanks for the great questions. But nobody got very nasty! I will have to search my email for that. It is still early in California, just 10:30. Time to take my 91-year-old mom, the former teacher, to one of her favorite haunts, the library. She is going to clean out another shelf of Nora Roberts in big type. Have a great weekend.
Editor's Note: washingtonpost.com moderators retain editorial control over Discussions and choose the most relevant questions for guests and hosts; guests and hosts can decline to answer questions. washingtonpost.com is not responsible for any content posted by third parties.