Washington Post Education Reporter
Tuesday, December 4, 2007 2:00 PM
Washington Post education writer Jay Mathews, who does the paper's annual Challenge Index rankings of national high schools, was online on Tuesday, Dec. 4 at 2 p.m. ET to determine what makes a school great, how to measure those factors, and the value of their competing systems.
The transcript follows.
Jay Mathews: Hi. This is an exciting time for us school raters and rankers. High schools have been overlooked in the past as not worth subjecting to such comparative examination -- except of course when talking about their football teams. The Post's, and Newsweek's, Challenge Index has had the high school rating business pretty much to itself the last ten years, except for some silly exercises at Worth magazine and the Wall Street Journal finding which private schools are most likely to get your kid into Harvard. I welcome U.S. News & World Report's new high school list, a very serious effort which I hope will focus more attention on the way we are educating teenagers. The data show that age group has had the least academic improvement over the last 30 years. And I am delighted to see we have some good questions here.
Philadelphia: Is there any evidence that more access to AP tests is helping large numbers of students graduate earlier? If not, why not? It seems like with the vast increase in AP test availability, and with the passing of those tests, three years for college should be the new norm -- but that does not appear to be the case. I think AP, dual-enrollment, and programs for college classes in high schools are all great, and have important intangible benefits, but saving a year of college tuition is an enormous benefit as well. Thanks for your thoughts.
Jay Mathews: This is a great start -- one of those rare questions I have never heard before. The answer is no. AP and IB long have used the prospect of earning college credit as way to motivate high school students to take the courses and tests, but most students, even those who have had to scrape together every last cent for college, prefer to stay the full four years. Instead of using the AP or IB credits to graduate early, they use them to create room on their schedules for electives they have always wanted to explore or graduate courses that will get them started on a career plan. Some older, or unusually mature, students, who already have their lives in order and know exactly what they want out of college, may use the credits to get out early, but the vast majority of kids that age prefer to take the four years to try lots of options and develop a sense of what they want their lives to be. That strikes me as a good way to go. In my view, the best thing about AP and IB is not the college credits, but the way they add juice to the high school experience, giving teens a chance to experience great teaching in preparation for a demanding final exam.
Boise, Idaho: Would you consider creating an index where high schools are listed for the number of concurrent/dual-credit courses offered in partnership with a college/university?
Jay Mathews: I would if I could figure out a way to do that which would only count concurrent/dual-credit courses that are genuinely challenging and provide the kind of high-level instruction that AP and IB do when students are required to take the AP and IB exams. Many dual enrollment community courses are just as good as AP. I took a fine calculus course at my local city college when I was in high school. But my conversations with college admissions officers and experts on community college offerings suggest that too many of these dual enrollment course are not so challenging. Some do not even require final exams to receive the college credit. For The Post's Challenge Index,we do count dual enrollment courses that have final exams that match AP or IB. We require such courses to have a final exam at least two hours long and some free response questions. The exam must be written and graded by someone who does not work for the high school, as is the case with AP and IB. Just a few schools seek to have such tests counted, so I have time to check out each exam. On a national level for the Newsweek list, we tried to do the same three years ago but discovered only the most affluent high schools had the resources to get us the information we needed. So we shelved that idea until we could find a better way.
Bayville, N.Y.: Hi Jay. Who do you think will sell more magazines this year, U.S. News & World Report, or Newsweek?
Jay Mathews: That sadly, is an easy question to answer. Newsweek will sell many more magazines than U.S. News, as it has done for many decades, but both magazines will likely post sales that are less than previous years. Such is the state of modern mass print journalism. We print people are in a slump, hoping our Web sites will keep us in business. I think our and U.S. News's high school lists will help both of us in that regard, which is good.
Syosset, N.Y.: As an educator and parent, I have many problems with the rankings system you have put into place. This might sound like sour grapes to someone if my school district weren't represented on your list, but it is, and has been for many years. I find that the districts push children into AP-level classes just to increase their standings on that list of yours.
Although I agree that AP and IB programs prepare students for college, I am concerned with the number of ninth- and tenths-graders who are being placed in these advanced courses. I have a hard time imagining 14-year-old children pursuing through a college-level textbook. I know of many instances where the classroom tests are "dumbed-down" in order to pacify parents, and these children not doing well on the AP exams.
Shouldn't the teachers and districts be held accountable for a correlation between class grades and AP scores? If this were done, many districts would think twice about pushing these courses on their student population before they are developmentally ready to handle them. Oh, I forgot, they wouldn't be able to be on that top-high-school-in-the-nation list then, would they? Something to think about, isn't it? Maybe you should add another component to your simplistic formula to incorporate scores on those AP exams given by the College Board. I think by doing so you will see a major shift in which schools make the list and which don't.
Jay Mathews: You have put your finger on a very lively and important issue, and also revealed another advantage of using AP and IB to rate schools. No other courses given in high school provide an incorruptible national standard against which you can judge the classroom grades given out by the teacher. I can't judge the quality of the AP courses for ninth- and tenth-graders in your school district, but I have taken a close look at the ones in Montgomery County, Md., where I live, and they are first rate. They tend to draw the most academically ambitious students in that age group, and those kids are excited about the chance to do AP. These are usually government, history and biology courses, and in Montgomery County the test scores are pretty good. I would not approve of shoving kids into AP who don't want to be there and are not being well taught, but keep in mind you won't really know how well they are being taught if you don't have the AP scores to guide you. One of my sons took an AP course at that age, and liked it. I know I would have loved to take an AP U.S. Government or U.S. History in 9th or 10th grade. It would have been a big improvement over the standard social studies courses in those grades. E-mail me and give me more details on your courses, and what you are seeing.
Syracuse, N.Y.: You mentioned community college dual enrollment programs -- what about dual-enrollment programs from universities like Syracuse, Indiana and Minnesota?
Jay Mathews: Those seem to be better run and more standardized (please correct me if I am wrong about that), but I would need to judge them, as we judge AP and IB, by their final exams.
It is difficult to get from them the data we need for the high schools that have students participating in them.
Rochester Hills, Mich.: Hi Jay. The first thing I noticed about Rotherham's list was the schools that are not on the list. It was unclear whether they were "filtered" or just missed. For example, the IB-based International Academy, which is arguably one on the best public high schools in the country, received a "bronze" medal. Some of the top-performing high schools in our state were completely missing. Do you see this as a flaw in Rotherham's formula, or is it just a bug in his processing?
Jay Mathews: I think it is both a flaw and a bug--a bug that is hard to cure. They are eliminating schools in the first run-through based on state test scores. If your students overall do not exceed statistical expectations, or if your minority kids do not score proficient at averages above those for those minorities in general in your state, then you don't appear on the list. The US News folks are very honest about the fact that this means a potential top-100 school can miss the mark by just a little and not appear at all. So many of their top schools are very high income or very selective that they have very few minority kids, and when you are subjecting such a small number of kids to such intense statistical examination, weird results can occur. And of course my argument with Andy Rotherham continues. He is intentionally cutting out big urban schools with low test scores but great AP and IB programs that are changing lives. That strikes me as a bad move. Until we find a way for those teachers to raise the test scores of low income kids, we are penalizing them for not solving a problem that no one yet knows how to solve.
Los Angeles: What are your views on Catholic schools? Do you believe that they give a better education to their students then public schools do?
Jay Mathews: In many instances they do. There is much data showing this is particularly true in the inner city, but some scholars think the difference is they get kids with more motivated parents than regular public schools. I am impressed with the number of great teachers I have encountered who attended, or taught in, Catholic schools. It isn't just a job for them.
St. Paul, Minn.: Public Schools in the Midwest (Minnesota, Wisconsin and Iowa) always are near top for average SAT/ACT scores from high school seniors, and are always near the top in other measures as well. Why is there a lack of Midwestern schools on your list? Does this show a flaw in your ratings system? Shouldn't the majority of schools on your list be from the Midwest?
Jay Mathews: Those schools are on the top of those lists for demographic (smaller percentages of low-income students) and choice (on the SAT, only the better Midwestern kids take those tests) reasons. They don't get on the Challenge Index in as large numbers as some other areas because they do not let average students take AP, which is a problem in most U.S. schools.
Washington: I had enough AP credits to graduate from college early (though only a semester and not a year early) but my parents thought it was better to stay in and get the fullest out of the college experience. I also would've been only 20, which is young. At the time I went to college (Class of '97) my school was $25,000, so it definitely would've helped my parents financially for me to graduate early. I did exactly what you said -- took different elective classes.
Jay Mathews: Me too. The American college experience is an enormous gift, one we want for as many children as possible. Why cut it short?
Washington: Are you not surprised that many of the high schools in this area that made the list are from wealthy areas (Langley, Churchill, Oakton, Whitman, etc.)?
Jay Mathews: That is what happens when your system focuses on test scores, but to be fair to U.S. News, those schools appear high on the Challenge Index too. The difference is that the U.S. News list cuts out many of them that are very similar to Whitman and Oakton, and easily make the Newsweek list. The U.S. News people are showing me their data and I may be able to explain that better in a few weeks. It has to do with the cut-off points for their first screen, which can accept a very affluent school like Whitman but cut out an only slightly less affluent school like Walter Johnson, both in my town of residence, Bethesda.
Bowie, Md.: Thomas Jefferson made the list as the No. 1 school because they are an exclusive magnet program. On the other hand a great school in Maryland -- Montgomery Blair -- didn't make it because it also serves as a neighborhood school. What are your impressions of this?
Jay Mathews: At first glance, it likely has something to do with the fact that Blair is 29 percent low-income, and Thomas Jefferson less than 1 percent low-income. The U.S. News folk have tried to level those playing fields, but experts like Dan Koretz at Harvard point out that that is very difficult to do that well with the usual tools, and it does not seem to have worked well here.
Washington: Is there a place to find private high school rankings? Thanks.
Jay Mathews: Oh yeah. You saw a reference to it in my column on this Web site today. Check out the Wall Street Journal last Friday and their list on which high schools are most likely to send kids to eight very selective colleges. They are almost all private schools, very expensive ones. I don't think that is a very useful way to look at high schools, but a beloved family member works for that paper, so I won't say another word.
Falls Church, Va.: Full-time kindergarten is taking a phased approach in Fairfax county. Wealthier areas, such as McLean, don't get full-time kindergarten until the bitter end, while "poorer" areas such as Fairfax get it immediately. How do you think this affects the intellectual growth and challenge of kids that age, especially if you're comparing a kid from full-time vs. a kid from part-time?
Jay Mathews: The wealthier families don't need a public kindergarten. They usually give their kids an even better education at that age than the public schools do. The less wealthy areas do need it, so giving them priority makes sense. Well-run kindergarten, and pre-K, are in the minds of many people whose views I respect the best way to solve our most stubborn issues, low achievement and high dropout rates later on in school.
Washington: Hi Jay -- what a wonderful appreciation of Sally Smith and the Lab School in the Post. Why can't teachers in regular D.C. schools be allowed to show this much creativity and spunk? Maybe Michelle Rhee and Mayor Fenty should spend some time at the Lab School and figure out what's going on over there that can be used in the District's underperforming schools.
washingtonpost.com: The Teacher at the Head of the Class (Post, Dec. 4)
Jay Mathews: Amen, Amen. She was something else, and personified the spirit of creative innovation that I am finding in some of our best charter schools.
Washington: Maybe the usual tools aren't the best ways to measure scores. Have we considered the quality of the assessment in accurately measuring what students are learning?
Jay Mathews: We have, and it is not that great. Most state tests are fairly cheap, designed to just give a quick snapshot of how kids are doing. I think that is okay, and that the way to improve schools is to focus on deep and rich programs with very authentic assessments like AP and IB.
Philadelphia: Thanks for answering my question. I expected that to be the case and I agree with your assessment of the situation. But I think if more complete information were available, the credits savings would be more than just a carrot, but a tangible benefit. With the help of AP credits I graduated in three years. I am immensely grateful for those credits. I did sacrifice some parts of the college experience, but I also saved $10,000 of debt, which to me was worth it.
I know that route isn't right for everyone, but for students who don't think they can scrape together four years of tuition and fees, or who can't make it through a full four years, AP credits could make college graduation a realistic goal. With more complete information from counselors, schools, and news sources, students could make that choice without wading through bureaucracy or giving up because they assume that the four-year route is the ideal option.
Jay Mathews: You are absolutely right, and I apologize for overlooking people with your perspective. There may be many more of them out there than I think. AP and IB do give us extra options, and I should appreciate that.
Alaska: Hi. My family and I are moving to Alaska. Now that I have two little kids, I'm starting to play attention to how good the schools are, but I don't really know what to look for. There isn't a high school in Alaska on Mr. Matthews' list, for instance. What should I look for? Are AP and IB classes important (my high school didn't have them)? How should I evaluate violence and intimidation levels? Thanks!
Jay Mathews: One of the nice things about the Challenge Index is you can do it yourself. Once you find a neighborhood that looks good, ask the local high school how many AP tests they gave last May and how many students graduated last June. If the first number is not pretty close to, or even better above, the second number, you might want to look elsewhere. I haven't checked Alaska's state ed department web site, but many states have those AP and graduation figures available for every school.
Falls Church, Va.: I was interested to see that Falls Church City's high school, George Mason, was not on U.S. News's list. It consistently appears very high on your local list and was ranked 60th nationally using your/Newsweek's criteria. This just makes me wonder how lists of this type are really used. I like your Challenge Index because it keeps my property value pretty high! It seems to be used by home buyers moving in to the area -- but I don't know that it really speaks to my impression of the strengths and weaknesses of the high school and the school system as a whole.
Jay Mathews: That is an easy question to answer. George Mason is a very successful IB school, one of the best in the country, but the U.S. News list doesn't use IB figures. They tell me they hope to add them next year.
Greensboro, N.C.: With rankings based solely on the number of AP exams taken, can't they in a sense be "bought" by districts willing to commit their dollars to that purpose?
Jay Mathews: They can indeed add many more AP courses and pay the test fees, but that is being bought in a very good cause. Whatever their motives, anything that exposes more kids to AP teaching and AP tests is a good thing, allowing them to develop academic muscles that will help them in college, and in life.
Alexandria, Va.: No Child Left Behind defines a successful school as one that has made "adequate yearly progress." Do you know how many of your top schools -- lets say the top 200 -- have made AYP? If any of them have failed to make AYP, what does that say to you about the debate between those who want to make major changes in the law and those why say "stay the course"?
Jay Mathews: A few of the top 200 on my list did not make AYP, such as Edina, Minn., which everybody knows is one of the best schools in the country. Schools like that get tripped up by very small differences between, say, their special ed achievements and the federal targets. And there are the big urban schools with lots of low-income kids that I mentioned earlier, and usually don't make AYP. It would be nice to tweak the law to let those schools pass muster if they show a certain amount of improvement, even if they don't reach the target for that year.
Washington: Jay, I graduated from Bethesda's Walt Whitman High School in the 1980s. Since then the school has fluctuated between being a great school and merely a very good school. At the same time, Wootten and Churchill have just climbed up, doing better and better, and dark horses like Fairfax's Thomas Jefferson have gone from "good school" in the 1980s to virtually the best school in the nation. Because $28,000 per year is really expensive for private school, how can a parent determine where to buy a house now to focus on the best high school?
Jay Mathews: I have been watching Whitman, Wootten and Churchill carefully since 1997, and I don't see any significant change in their relative quality. They were and are first-rate schools, mostly because they draw from very affluent and well-educated families. E-mail me and tell me what you know about Whitman that suggests otherwise. I think if a high school is on the Newsweek list, which all of those are, you are wasting money to go private rather than sending your kid there.
Washington: Why do you continue to measure the number of tests taken instead of the actual resulting scores on the tests?
Jay Mathews: We do report the test results in our equity and excellence percentage, given for each school. But the test results, as I said, are not a measure of the school, but a measure of the student body. The wealthier and better educated the kids' parents, the higher the test scores. The Challenge Index does a much better job showing which schools are trying hardest to raise the level of all kids.
Syracuse, N.Y.: Why all the focus on just AP, IB and other standardized tests? Why not also include -- at least as a sidebar -- high schools that also partner with college and universities?
Jay Mathews: Many of them are quite good, but there is little quality control on those partnerships. Some are just high school teachers given a college course and asked to teach it at a college level, without an outside exam to guide them. That is a recipe for something less than promised.
Falls Church, Va.: How did U.S. News and World Report's rating criteria differ from yours? I moved to this area specifically for my kids to attend McLean High School, and McLean High School was nowhere to be found on the list.
Jay Mathews: I need to study their very complex system more to answer your good question completely, but in short they are looking at a very narrow range of schools like McLean and keeping many of them off the list for small differences in their state test scores and their very few minority kids' state test scores. It is complicated, but if you called the U.S. News folks I wager they would tell you that you are not going to regret sending your kid to McLean High. It does very well on the Challenge Index. I have been to the school several times, interviewed its great teachers, seen what its graduates do in the world. Don't worry. The U.S. News list is brand new and they have to work out some kinks.
Jay Mathews: Thanks for the great questions. I hope you send some to U.S. News too. My friends over there want to be noticed, just as we do at The Post and Newsweek. If they give you answers that intrigue you, pass them on to me at firstname.lastname@example.org.
Editor's Note: washingtonpost.com moderators retain editorial control over Discussions and choose the most relevant questions for guests and hosts; guests and hosts can decline to answer questions. washingtonpost.com is not responsible for any content posted by third parties.