When I started what The Washington Post now calls the America’s Most Challenging High Schools list, I was told not to trust data from schools and school districts. They’re sloppy and sometimes dishonest, people said. It won’t work.
That was 19 years ago. The doubters were wrong. The educators I deal with have proved to be unfailingly honest. Mistakes are rare. But the biggest so far just happened. The IDEA Public Schools charter network in Texas told me it provided incorrect numbers of Advanced Placement tests at six of its schools for the 2017 list published in May.
As a result, the five IDEA schools that were in the top 10 have dropped several places on the corrected list. “We messed up,” said IDEA founder and chief executive Tom Torkelson.
The IDEA network educates more than 30,000 students, mostly from low-income families. It prepares children from the earliest grades for the challenges of AP in a way I have never seen before. It emphasizes technology and data, but this time the network apparently tripped over its own eagerness to measure in new ways what its students are doing.
I had asked the IDEA officials, as I do all schools, to provide the number of AP, International Baccalaureate or Cambridge tests they gave at each school in 2016. They already had in their system a report that listed the number of AP tests their students had taken over several years. That was mistakenly sent instead of what I asked for. They reported, for instance, the IDEA Frontier College Prep school in Brownsville gave 1,940 AP tests in 2016, when the actual number was 1,427.
The formula for the Challenge Index rating that determines each school’s rank is: divide the number of tests by the number of graduating seniors. Frontier College Prep had 99 graduates last year. Because of the error, its rating dropped from 19.596 to 14.414 and its rank dropped from third to sixth.
IDEA San Juan dropped to No. 10 from No. 5. IDEA Mission dropped to 12 from 4, IDEA San Benito to 13 from 6, and IDEA Quest to 21 from 9. A sixth IDEA school on the list, IDEA College Preparatory, dropped to 90 from 84. All of them remain in the top one-half of 1 percent of U.S. schools measured this way.
I invented this list and am still responsible for gathering and verifying its data. It is my fault that I did not detect the error earlier. I was saved by the work of Carol Burris, executive director of the Network for Public Education and a former New York state high school principal of the year.
Burris is one of the most knowledgeable education critics in the country and a prime source for my recent columns on the BASIS charter school network. She looked at the IDEA numbers, compared them to the number of AP tests that each IDEA student usually takes, and found a discrepancy I had missed. She told me about it. I asked IDEA to explain. They rechecked their data and quickly confessed that they had erred.
Going forward, I will be warning schools to be extremely careful with how they report their AP data. IDEA officials said they were able to verify that they submitted the correct data for the 2016 list, when they had six schools in the top 50. College Board documents confirmed their corrected figures for the 2017 list.
I am sorry for my mistake. I started the list to celebrate those schools that had discovered that even ordinary students, if taught well, could thrive in a college-level course. Research shows that even students who did not pass the AP exam are often better prepared for college than similar students who did not take AP.
Rankings draw attention because we are all tribal primates endlessly fascinated with pecking orders. I thought schools that were inspiring average students to grapple with difficult concepts deserved recognition as much as football and basketball powers did.
What will remain more important to me, however, is reporting in detail on what these great educators are doing to help so many teenagers raise their level of reading, writing and thinking.
Correction: An earlier version of this report included an incorrect ranking for IDEA College Prepatory. It dropped to 90.