By Jay Mathews
Washington Post Staff Writer
Monday, March 24, 2008
5:54 AM
The Challenge Index, my device for assessing high schools on college-level course participation, was born 10 years ago this month in The Post and Newsweek. At the beginning it was mostly a way to draw attention to a book I had written, "Class Struggle: What's Wrong (and Right) with America's Best Public High Schools." I feared that my prose was far too stuck in the minutiae of classroom life to win much of an audience but hoped that a list of schools ranked in a new way might tweak some curiosity.
In May, Newsweek will again publish its annual Top High Schools list, using the Challenge Index rating method, just as The Post published its annual Challenge Index list of D.C. area schools in December. These lists have taken on a life of their own. Newsweek's Top High Schools was the most visited feature on the Newsweek.com Web site last year. The Post's local list is also popular, and both are targets of controversy, producing by far the most questions and comments coming to my e-mail boxes.
Is this good? I would like you to tell me. These past 10 years I have been quoting regularly from the lists' most acidic critics, as well as their warmest friends. But the arguments on both sides have grown stale and predictable. I have a new idea for advancing the debate.
First, I would like to ask all high schools that have strong Advanced Placement, International Baccalaureate or Cambridge programs and have NOT gotten the Newsweek list entry form to e-mail highschools@newsweek.com right away and request one. If you gave at least as many AP, IB or Cambridge exams last May as you had graduating seniors last year, you should qualify for the Newsweek list. We gather all of our information for the list directly from the qualifying high schools. We have sent out thousands of forms, but we don't want to miss anybody. If you know of a high school that you think has been overlooked, please forward this column to the principal. I figure the more schools on the list, the more varied and interesting the opinions of the list.
Second, I would like all readers of this column who have had some firsthand contact with the effects of the Challenge Index in an actual high school to send me at mathewsj@washpost.com your view of this way of assessing high school quality. I want to quote your observations in future columns. The more real-life stories I get, the better.
Whether you like the Challenge Index or not, I hope you can put aside your favorite theories about it and instead focus on how it has affected, or not affected, for good or ill, teaching and learning in your school. I love data. I love anecdotes. Give me what you've got. I heard much speculation about the index in its early years and produced much of it myself. But after 10 years, I am hoping we have some interesting and useful facts to share.
For those of you new to this argument, most criticism of the index focuses on its simplicity. Each school is rated and ranked by the ratio of just two numbers: the number of college-level exams given at the school to all students in all grades divided by the number of graduating seniors. Critics say it is absurd to judge a school by just two numbers. They say the list also encourages schools to assign every student to college-level classes without any thought to whether such courses are good for them, and produces classes full of students who struggle and fail the exams, possibly ruining their records and college chances.
Supporters of the index say those fears are groundless. American high school students cannot be forced to take a college-level course against their will. Instead, they say, the Newsweek list gives recognition and political cover to principals and teachers who have been fighting apathy and racial and class bias in an effort to expose more students to challenging assignments in reading, analyzing and writing. They say the Challenge Index exposes the many schools that are denying college-bound students a chance to prepare for college, and helps even students who struggle in the difficult courses to build their academic muscles.
I will need to attach your name and the name of the school to whatever you tell me, but the people you are describing can be anonymous. I hope to prepare a good collection of observations on all sides that we would post on both washingtonpost.com and Newsweek.com.
I realized long ago that whatever the lists' benefits or drawbacks, they were a terrific way for me to connect with educators, students and parents. I have learned a great deal from what you have told me so far. Please don't stop now.
****
****
Editor's note:
Below is an article on the debut of the Challenge Index, published on March 22, 1998, in The Washington Post Magazine.
THE CHALLENGE INDEX
ARE WASHINGTON AREA HIGH SCHOOLS CHALLENGING THEIR STUDENTS ENOUGH, PUSHING THEM TOWARD THE MOST ADVANCED WORK THEY CAN HANDLE? THE AUTHOR HAS DEVELOPED A SYSTEM TO EXPLORE THAT QUESTION - - AND A WAY TO MEASURE SCHOOL PERFORMANCE
By Jay Mathews
Nearly every professional educator will tell you that ranking schools is counterproductive, unscientific, hurtful and wrong. But I am going to do it anyway, not because I believe my system is scientifically infallible, but because I think it provides insight into one of the most significant emerging issues in American education: whether our high schools are working hard enough to challenge and elevate students. Just last month, the results of a new international exam in math and science showed that American 12th-graders ranked close to last among the 21 nations that participated. Those dismal results provoked dismay among educators and politicians and prompted Education Secretary Richard W. Riley to declare, "We need to have higher expectations for our students." I could not agree more. That's why the reporting and analysis that follow focus not on how good area students are -- the way others often rate schools -- but on how hard each school tries to make each student better.
The importance of challenging students, and the reluctance of many public schools to do so, first attracted my attention in 1986, when a quiet boy with square features and short brown hair named Greg Rusu tried to enroll in an Advanced Placement calculus class at Garfield High School in East Los Angeles. I was working as a reporter in Los Angeles at the time and became interested in his story.
Rusu's salient trait was his stubbornness, a gift from a father who had gotten the family out of Romania by going on a hunger strike. Rusu wanted to take calculus, even though he would only be a junior and would first have to survive a summer of trigonometry. His counselor and his computer science teacher advised against it. Such a lethal dose of mathematics, they said, would leave no time for his other courses, ruin his grade point average, kill his chances for a college scholarship and anger his parents. The calculus teacher, Jaime Escalante, later the hero of the film "Stand and Deliver," had a different reaction. He listened to the boy's request while correcting another student's paper. After Rusu was finished, the teacher said, "Yup," and handed him the necessary form. Rusu eventually received the highest possible grade on the AP calculus test and was accepted at a fine engineering school.
A decade later, having visited dozens of other high schools across the country, I am convinced that students like Greg Rusu are still receiving little encouragement from most of their teachers and counselors. As often as not they are shunted off to courses that are too easy and, for many, too boring.
This tendency reveals itself in the way American high schools handle their Advanced Placement (AP) and International Baccalaureate (IB) courses. Both programs administer difficult examinations to students who have completed high school versions of college-level courses, from AP economics to IB social anthropology. A few teachers and schools use the courses and the tests to motivate students who would ordinarily be given much lighter fare. But most educators do not like to push children that hard. The reasons are varied and complex, and illustrate, among other things, I think, the corrosive effect of ill-considered kindness in American schools.
This is, in some respects, a crisis of American democracy. The AP and IB tests were born of supremely elitist motives -- to ease the ennui of prep school students and diplomats' children. But teachers at a few inner city schools have hammered this gilded tool into a device for enlivening and deepening the education at schools filled with ill-prepared and unmotivated students. What is puzzling is the failure of many suburban schools to democratize their best courses in the same way.
Some teachers say they fear that students asked to do the hard work these courses require will lose interest altogether and drop out. Some teachers say that parents are likely to protest too many demands, and assume their child will get a bad grade and hurt his or her college chances. Some teachers, already drained by long hours teaching ordinary classes, do not think they have the energy to pull students up to an AP or IB level. Some principals and department heads say they do not have enough teachers willing to be judged by their students' performance on examinations written by national and international organizations over which they have no control.
Many educators argue that AP and IB courses should be reserved for the very best students, who would be annoyed and distracted if there were too many students in class who asked basic questions and struggled with the workload. Too many marginal students in the class, some principals have told me, will reduce the percentage of students passing the AP and IB tests, and tarnish the school's reputation in the eyes of the district superintendent and the best colleges. Some state school boards, including Virginia's, aggravate this misplaced emphasis by ordering schools to report their annual AP and IB pass rates. When more than 90 percent of the students taking such tests receive passing scores, we know two things: (1) conscientious teachers are doing good work, and (2) many students who might struggle with the tests are being discouraged from taking them.
I once thought the reluctance to welcome students into difficult courses was a failing of schools like Garfield in low-income neighborhoods with inadequate resources and teachers who did not expect much. Children at elite public high schools were unlikely, I assumed, to be troubled by low expectations and artificial barriers to doing their best. Then I began to interview educators at places like Darien High in Connecticut and New Trier High in Winnetka, Ill., where the homes of some students' parents are worth more than a typical Garfield parent's lifetime income. I found that many of the best schools had fallen into the bad habits of the worst.
When my family and I moved to Scarsdale, N.Y., in 1992, I discovered that my son could not be admitted to honors English or AP U.S. history at Scarsdale High unless he scored well on a qualifying test. Instead of using its talented teachers to bring as many students as possible into those classes and up to that level, the school decided to treat those courses like the best linen. Only certain guests were good enough for it. At New Trier, I found a culture so wedded to an academic pecking order that students were placed in five different levels of courses based on past scores and teacher recommendations. If a student or her parents objected, they had to sign a statement saying they were acting against professional advice and "assume full responsibility for the consequences of this placement." The grading system was so distorted by status consciousness that students who slept late, missed homework and did only C work in a top-level course received more grade points than students three levels below who worked hard and received a B.
At Mamaroneck High School in Westchester County, N.Y., I met a junior, Kerry Constabile, who was barred from AP U.S. history because her grade point average was not quite high enough. She became so upset that she assigned the course to herself, borrowing homework from her AP friends, reading extra books and eventually passing the test. If she had gone to Wilson High in the District or Wakefield in Arlington or any of a number of less affluent schools with more encouraging faculty, she would have been welcomed into AP. Her problem was that she attended a school that, in some ways, had too good a reputation to risk on her.
The teachers and counselors who oversee all this are kind, thoughtful and otherwise admirable people, with their students' best interests at heart. "If it has not been a good year academically or emotionally, to pressure kids to take the AP test is wrong," said Dale Mills, head of the guidance department at Glen Burnie Senior High in Anne Arundel County.
Otherwise attentive and encouraging parents exacerbate the problem by shrinking from the threat of a bad grade. One teacher told me that even at the Thomas Jefferson High School of Science and Technology in Fairfax, with the highest-scoring students in the area, administrators are afraid to push more students into science projects because some will fail and make their parents unhappy. As a father, I have known that fear, but in fact, because AP and IB test results are not linked to classroom grades, teachers in these advanced courses can mark report cards any way they choose. The student does not have to send the test scores to colleges, although many university admission officers are pleased to see a student try something demanding, even if the results fall short of what the student had hoped.
To provide some way to compare how schools handle this issue, I devised a simple measure, which I have dubbed the Challenge Index. The index makes an effort to show how much a school tries to persuade students to take AP or IB courses and the related tests. AP and IB are the only consistent measures of instructional quality for U.S. high schools, and are alike enough to be interchangeable for the purposes of the index. The Scholastic Assessment Test (SAT) is a much less useful alternative because it tests skills learned before most students reach the ninth grade and does not require mastery of any particular high school course.
To create the index for a given group of schools, I calculated the number of AP or IB tests given at each school, then divided by the number of students in the June graduating class, so that big schools would not have an advantage over small ones. Using the graduating class as an indicator of school size tends to level the playing field for schools that draw from low-income neighborhoods with heavy dropout rates, because the strength of each AP or IB program is measured only against the number of students who are committed enough to earn a diploma.
Having derived an index number for each school, I gave each school a rank. Like every other numerical measure of complex human institutions, the index is in some ways narrow and distorted. A school that stumbles one year may recover the next, for instance. Yet I think the exercise illuminates interesting and unexpected differences and exposes some unexamined assumptions. For one thing, in the Washington area, the index reveals that some public high schools are making remarkable efforts to stretch the minds of their students and others are not doing as well as they are equipped to do.
I have left off the list two schools, Jefferson in Fairfax (index 4.879), and Banneker in the District (index 1.305), because, unlike other area schools, they are allowed to select all of their students on the basis of academic merit. Schools like Richard Montgomery in Montgomery County and Eleanor Roosevelt in Prince George's County select some of their students this way. But as long as no more than half of a school's students are admitted by test scores and grades, I decided to put them on the main list.
The index does not record how well students do on the AP and IB tests, only how many tests they take. A school that draws from poor or troubled neighborhoods has an opportunity to rank as high on the index as a school full of lawyers' children. Even so, it's obvious that one of the most important factors in any school's performance, even on this index, is the prosperity and education levels in the neighborhoods from which the school draws its students. Schools from rich areas tend to do well; those from poor areas, poorly. But there are fascinating and potentially instructive exceptions in both categories: privileged schools that do less well on the index than you might expect, and underprivileged schools that do as well as their wealthy counterparts, or better.
At Garfield in East L.A., nearly every student who wanted to try an AP course was allowed to do so. Several students who preferred easier fare were dragged in as well. Students who failed the tests were congratulated for their efforts and invited to try again. Many students told me they acquired new confidence in themselves by the mere act of sticking with the course and taking the test. Many of the least eager students found that they did much better than they expected. The program became fashionable, giving it the ultimate high school weapon, peer pressure.
Only a few Washington area schools with Garfield's relatively high percentage of low-income students have adopted such an aggressive approach. Central in Prince George's County, Wilson in the District, Wakefield in Arlington and Mount Vernon in Fairfax are among them. The Fairfax and Arlington school systems do particularly well, in large part because two recently departed superintendents, Robert R. "Bud" Spillane in Fairfax and Arthur W. Gosling in Arlington, believed that AP courses were shams if the students were not pushed to take the exams. Prince William County School Superintendent Edward L. Kelly requires that all students in core AP and IB courses, such as English, chemistry, mathematics and history, take the tests, in return for which the district pays the $75-per-test fees.
And what of schools of excellent repute, drawing from prosperous and well-educated areas, that do not score as high as you might expect? James E. Person, principal of Park View High School in Loudoun County, said many students in AP courses fail to take the AP tests because "they get stretched pretty thin and are involved in a ton of other things." Jack Graham, the new principal of Osbourn High in Manassas City, said teachers are frustrated by the number of students who take the course but choose not to take the tests. "We need to do a better job of selling that," he said.
Whitman High in Montgomery County has a five-year-old building that resembles an upscale mall and a national reputation for excellence. It has twice been named the best high school in Maryland by Redbook magazine. Its students win prizes and go off to Ivy League schools each year in astonishing numbers. Its index of 1.170 puts it among the top 2 percent of U.S. schools -- and yet it falls below schools like Langley in Fairfax, Yorktown in Arlington, and also below many schools around the country that have fewer resources and more low-income students.
Why is that? Some Whitman students say those who receive a B in pre-calculus are not encouraged to take AP calculus, although principal Jerome Marco denies it. Writing tests are given to students who wish to enter AP English, and those who don't meet a high standard are not encouraged to enroll. The initial AP English assignment is often so difficult that some students immediately drop the course. This suggests that the school has chosen to teach at least some of its AP courses at such a high level that they exclude students who, with patient teaching, could do reasonably well on the AP examination.
Marco says many students in AP courses fail to take the AP tests after they have received their college acceptances and no longer need to decorate their transcripts. He said teachers are right to steer students away from classes that are too hard for them but that he will open any course to any qualified student who insists. One way out of this predicament might be for schools like Whitman to establish AP-plus classes for their ultra-talented students, so that they could comfortably offer the ordinary AP curriculum to applicants who are merely bright.
Some teachers discourage students from remaining in AP classes or from taking the test when they are struggling with the subject matter. This helps inflate the school's percentage of passing scores, a source of pride for faculty and administrators, but does not inspire good teaching.
Compare this to the anything-goes spirit that animates students, teachers and counselors at Falls Church's George Mason High, where more high-level courses are offered to a larger slice of the student body than at any other public school in the Washington area. It is the only high school in its district. It has fewer than 500 students on a little campus wedged between Interstate 66 and Leesburg Pike. But in some ways it outshines every other school I have ever visited.
Michael Hoover, who teaches English at George Mason, says a few teachers warned of severe consequences when the school decided to open its IB classes to all students. He was told the percentage of students passing the tests would drop. They did at first, then rebounded to new highs. He was told that parents would object to diluting the high-level classes. Instead, they embraced the idea.
Erin Albright, the IB coordinator, said 35 percent of George Mason's juniors and seniors took IB courses and tests five years ago. That percentage has doubled. She said many teachers had long suspected that they could demand much more from students than they did. "Nobody ever came back and told me they worked too hard in high school," she said. "We showed the students that you can challenge yourself at the highest level and be successful."
George Mason overcame the warm complacency that envelops most schools in nice suburban neighborhoods. But it is an isolated example. Urging teenagers to dive into deeper academic waters seems to be what our national education leaders want, but such daring still inspires little more than discomfort in many of our best public high schools.
****
****
Below is the first Challenge Index that appeared in the Post magazine
CAPTION: How demanding are Washington area high schools? Here each public high school is ranked by a number rating that reflects how much a school encourages students to take Advanced Placement (AP) or International Baccalaureate (IB) courses and their internationally recognized tests. The rating is not a measurement of the quality of a school overall or the achievements of its students and faculty -- it is strictly focused on AP and IB courses. The rating is determined by dividing the number of AP and IB tests taken by students at a school in 1997 by the number of students in the school's graduating class last June.
Compared with high schools across the country, Washington area schools overall are among the most encouraging of AP and IB testing. For instance, any school with a rating of 1.000 or above is in at least the top 2 percent of all U.S. high schools in encouraging students to take AP and IB courses.
Most schools use AP tests and some use both AP and IB. Schools measured by their IB tests are marked with an asterisk. Also listed are the name of the school district and the percentage of a school's students whose family incomes are low enough to qualify for federally subsidized lunches and who also apply for that program. The number of subsidized-lunch applicants is provided as a rough indicator of the socioeconomic factors that may affect a school's performance or expectations. The number of such applicants usually understates the portion of poor children in a given school.
School/District/Index/Percent Subsidized Lunches
George Mason* Falls Church 3.146 (10%)
H-B Woodlawn Arlington 2.676 (13%)
Richard Montgomery* Montgomery 2.316 (13%)
Langley Fairfax 1.987 (1%)
West Potomac Fairfax 1.597 (25%)
Madison Fairfax 1.415 (6%)
Yorktown Arlington 1.414 (14%)
McLean Fairfax 1.398 (10%)
Wilson D.C. 1.281 (23%)
Churchill Montgomery 1.253 (2%)
Oakton Fairfax 1.238 (8%)
Wootton Montgomery 1.234 (3%)
Woodson Fairfax 1.196 (7%)
Marshall Fairfax 1.193 (25%)
Montgomery Blair Montgomery 1.177 (24%)
Eleanor Roosevelt Prince George's 1.173 (19%)
Whitman Montgomery 1.170 (2%)
Centreville Fairfax 1.160 (10%)
Mount Vernon* Fairfax 1.156 (33%)
Walter Johnson Montgomery 1.123 (6%)
Bethesda-Chevy Chase Montgomery 1.084 (11%)
South Lakes Fairfax 1.069 (16%)
Robinson Fairfax 1.055 (8%)
Washington-Lee Arlington 1.039 (37%)
School Without Walls D.C. 1.012 (0%)
Central* Prince George's 1.010 (42%)
Poolesville Montgomery 1.009 (4%)
Hylton Prince William 0.991 (7%)
Chantilly Fairfax 0.989 (8%)
Centennial Howard 0.986 (2%)
West Springfield Fairfax 0.984 (6%)
Stonewall Jackson* Prince William 0.946 (17%)
Woodbridge Prince William 0.944 (15%)
Quince Orchard Montgomery 0.943 (10%)
Falls Church Fairfax 0.938 (35%)
Brentsville Prince William 0.936 (5%)
Lake Braddock Fairfax 0.930 (13%)
Herndon Fairfax 0.904 (13%)
Gar-Field Prince William 0.885 (15%)
Loudoun Valley Loudoun 0.857 (6%)
Fairfax Fairfax 0.833 (18%)
Sherwood Montgomery 0.774 (9%)
Stuart* Fairfax 0.757 (54%)
Broadneck Anne Arundel 0.748 (2%)
Wakefield Arlington 0.722 (47%)
Severna Park Anne Arundel 0.715 (2%)
Liberty Fauquier 0.711 (11%)
Fauquier Fauquier 0.674 (7%)
Magruder Montgomery 0.674 (15%)
Bell Multicultural D.C. 0.653 (n/a)
South River Anne Arundel 0.650 (3%)
Paint Branch Montgomery 0.647 (10%)
Einstein Montgomery 0.633 (27%)
Springbrook Montgomery 0.630 (23%)
Osbourn Park Prince William 0.621 (11%)
Westlake Charles 0.615 (7%)
T.C. Williams Alexandria 0.613 (33%)
Edison Fairfax 0.606 (25%)
Annapolis Anne Arundel 0.603 (11%)
Lackey Charles 0.578 (20%)
Lee Fairfax 0.571 (22%)
Rockville Montgomery 0.563 (20%)
Broad Run Loudoun 0.548 (3%)
Glenelg Howard 0.521 (2%)
Loudoun County Loudoun 0.510 (11%)
Leonardtown St. Mary's 0.508 (12%)
Hayfield Fairfax 0.508 (23%)
Chesapeake Anne Arundel 0.504 (3%)
Northern Calvert 0.497 (6%)
Mount Hebron Howard 0.486 (6%)
Oakland Mills Howard 0.469 (11%)
Watkins Mill Montgomery 0.459 (15%)
Atholton Howard 0.454 (4%)
Kennedy Montgomery 0.441 (20%)
Oxon Hill Prince George's 0.436 (23%)
Wilde Lake Howard 0.426 (15%)
Old Mill Anne Arundel 0.424 (7%)
Damascus Montgomery 0.411 (5%)
Seneca Valley Montgomery 0.397 (15%)
Northeast Anne Arundel 0.394 (6%)
Potomac Prince William 0.391 (21%)
Park View Loudoun 0.382 (10%)
La Plata Charles 0.380 (13%)
Great Mills St. Mary's 0.368 (23%)
McDonough Charles 0.362 (9%)
High Point Prince George's 0.358 (49%)
Annandale Fairfax 0.337 (33%)
Arundel Anne Arundel 0.333 (3%)
Bowie Prince George's 0.322 (15%)
Calvert Calvert 0.308 (10%)
Surrattsville Prince George's 0.294 (15%)
Northwestern Prince George's 0.290 (53%)
Southern Anne Arundel 0.276 (7%)
Hammond Howard 0.273 (6%)
Osbourn Manassas 0.269 (9%)
Gaithersburg Montgomery 0.268 (19%)
Chopticon St. Mary's 0.266 (10%)
Stone Charles 0.261 (14%)
Parkdale Prince George's 0.238 (53%)
Wheaton Montgomery 0.238 (34%)
Laurel Prince George's 0.236 (31%)
Suitland* Prince George's 0.230 (34%)
Douglass Prince George's 0.215 (16%)
Meade Anne Arundel 0.200 (17%)
Cardozo D.C. 0.169 (n/a)
Eastern D.C. 0.168 (52%)
Howard Howard 0.159 (7%)
North County Anne Arundel 0.159 (9%)
DuVal Prince George's 0.150 (32%)
Glen Burnie Anne Arundel 0.146 (8%)
Fairmont Heights Prince George's 0.144 (37%)
Friendly Prince George's 0.135 (18%)
Largo Prince George's 0.122 (20%)
Anacostia D.C. 0.111 (58%)
Ellington D.C. 0.107 (19%)
Gwynn Park Prince George's 0.106 (12%)
Coolidge D.C. 0.093 (36%)
Roosevelt D.C 0.083 (52%)
Crossland Prince George's 0.067 (34%)
Potomac Prince George's 0.066 (47%)
Ballou D.C. 0.049 (n/a)
Dunbar D.C. 0.041 (43%)
Forestville Prince George's 0.036 (42%)
Bladensburg Prince George's 0.017 (50%)
***
Two schools, Thomas Jefferson in Fairfax (4.879) and Banneker in the District (1.305), were omitted because they are allowed to select all of their students on the basis of academic merit. Ratings could not be calculated for Long Reach and River Hill in Howard, Blake in Montgomery, Patuxent in Calvert, and Potomac Falls in Loudoun because they are new schools that did not have full senior classes in 1997. Three schools in the District, Phelps, Spingarn and Woodson, like 47 percent of American high schools, did not give AP or IB tests in 1997.
Sources: State, district and school officials, Washingtonian magazine (for subsidized-lunch information).
¿ The Washington Post
View all comments that have been posted about this article.