Dear Extra Credit:
Since your professional reputation is based upon your Challenge Index, it can hardly be expected that you would be objective when criticisms based upon actual experience and observation are made of its findings. To your readers without a personal stake in the issue, however, it is obvious that your defense of your system often merely supplies more evidence of its shortcomings.
You gave Richard Montgomery High School an index of 5.0294 based upon the fact that 2,047 Advanced Placement and International Baccalaureate tests were taken in 2004, a year in which there were 407 graduating seniors. You then rejected the suggestion by your reader that this high index was due to the fact that Richard Montgomery hosts an International Baccalaureate program. You made this argument on the basis that there were only about 400 IB students but there were 815 AP test-takers at Richard Montgomery that year, so 415 non-magnet students made a contribution to the high index. Of course they made a contribution, but how much?
Your correspondent made the statement that the IB students take all AP-level classes. You didn't address this statement, so I presume you agreed with it or at least thought it was likely enough not to warrant doing any work to challenge it.
Theoretically this could mean they could take AP tests in all their subjects. That this did not happen is clear from the fact that 400 IB students times five or six classes would account for all or more than all of the 2,047 IB and AP tests taken at Richard Montgomery, leaving no room for the 415 non-magnet students who took tests.
But it is quite likely that the IB students did take several tests. Why wouldn't they? They have done the preparation work, they are in the IB program in the first place because they are motivated high-achievers, their families in general have the income to pay the fees, so why would they stop short of the payoff? If the IB students averaged four tests each, that would account for 1,600 of the 2,047 tests at Richard Montgomery. Even if they took only three (out of their five or six classes), that would account for 1,200.
What would it do to the Richard Montgomery Challenge index if 1,200 to 1,600 IB and AP tests were removed from the 2,047 total, leaving only the tests taken by the non-magnet students?
Your column also issued your standard challenge to anyone to come up with a better way to measure preparation for college than your Challenge Index. A suggestion that would accomplish this has already, I am sure, been made many times. Just multiply the number of tests taken by the average score achieved by the students at the school, before dividing by the number of seniors. That way, the indicator would not be based solely on quantity of performance; quality of performance would also play a role.
Of course, this would not completely correct flaws in situations such as Richard Montgomery's, in which both quantity and quality are biased by the select nature of part or all of the student body. In such situations, perhaps the only way to make adequate comparisons between schools would be to place schools with select students and students without in separate rankings. The rankings for schools without select students would be a fairer and more accurate indication of how well schools with students with a normal range of abilities do in preparing their students for college than if they are compared with schools that recruit top students from a wider area than their geographical jurisdiction.
At present, your Challenge Index on balance probably does more harm than good, so hopefully some day you will take the obvious steps needed to refine it into something of real value.
I appreciate your suggestions and will ponder them as I work to improve the index (keep in mind, though, that I said the number of students outside the IB magnet program who took AP tests had to be higher than 415, and the actual number would affect your calculation). But the latest figures from Richard Montgomery support the notion that even without its IB program, the high school would rank very high in AP test participation.
Richard Montgomery IB coordinator Carol Solomon's best estimate is that 1,041 of the school's 1,677 AP tests in 2005 were taken by non-IB students.
Richard Montgomery had 398 graduating seniors in 2005. Let's pretend none of the 84 IB seniors were there and subtract them from the graduating class. That gives Richard Montgomery a Challenge Index rating of 3.315, based just on AP tests taken by non-IB students.
That is not as high as its 2004 rating of 5.029, in which the IB students were included, but it is still high enough, if it had had such a rating in 2004, to be ranked 10th in the Washington area and 51st in the country, and in the top 2/10 of 1 percent of all U.S. public schools. Those non-magnet Richard Montgomery students seem to be doing very well all by themselves.