Tulane University released a report this month which said that high schools in charter-heavy New Orleans were “beating the odds” with test scores and graduation rates higher than expected for most students. The report got some attention in education circles because New Orleans is the poster city for charter schools. Furthermore, Tulane is a well-regarded research university so one could rightly assume that the research it publishes is sound.

Not, apparently, always.

In what one New Orleans newspaper called a “high-profile embarrassment,” Tulane effectively said, “Oops, never mind,” and retracted the report on charter schools that had been released just days earlier. What happened?

The initial report, titled “Beating-the-Odds: Academic Performance and Vulnerable Student Populations in New Orleans Public High Schools and published by  Tulane’s Cowen Institute for Public Education Initiatives, was viewed as proof that charter schools in the city were achieving better than predicted results for high-needs students. It was posted on the institute’s Web site on Oct. 1 and received some attention in the media, including this story in the Times-Picuyane that was published with this headline: “Most New Orleans public high schools beat the odds, study says.” (At least 80 percent of the schools in New Orleans are charter schools.)

The authors of the study took data and came up with a prediction of how students in different demographic groups would do on the ACT exam as well as on class exams. Graduation rates were also predicted. A comparison of what the students actually did led the authors to conclude that schools were doing better than expected. But there was, apparently, a problem with how the “expected”  part was calculated.

Not long after the report was posted on line, it was taken down. Why? According to The Times-Picyaune, top officials at the institute realized the research was bad. It quoted institute Executive Director John Ayers as saying, “Officials determined the report’s methodology was flawed, making its conclusions inaccurate. The report will not be reissued.” The institute plans to “thoroughly examine and strengthen its internal protocols” to ensure its future reports are accurate and have been appropriately reviewed, he said, adding, “We apologize for this mistake.”

Ayers did not explain what the “mistake” actually was, the newspaper reported. But the controversial method that was used to come up with the predictions is known as VAM (value-added methodology), which purports to be able to take various data points, plug them into complicated formulas and “predict” performance.

VAM has become popular in recent years in education reform circles, especially for the task of measuring the “value” of a teacher by using student standardized test scores in formulas that can supposedly factor out all outside influences on a student.  Assessment experts — including the American Statistical Association in a report last  April — have emphatically said these formulas can’t actually do this with reliability and validity, but school reformers have pushed this approach and now most states use VAM as part of school and teacher evaluation .

Tulane’s Cowen Institute has been supporting the charter movement in New Orleans for years. According to its Web site, the institute is “an action-oriented think tank that informs and advances solutions – through policies, programs, and partnerships – to eliminate the challenges impeding the success of K-12 education in New Orleans and beyond.” As educator and blogger Mercedes Schneider wrote in this post, it has “been promoting the New Orleans Charter Miracle since 2007 … trying since then to sell the ‘transformed’ post-Katrina education system in New Orleans.”

This is clear in an explanation on the institute’s Web site under a section called “Situation Assessment,” in which it talks about how awful the New Orleans public schools were before Hurricane Katrina in 2005 (which they were) and how the system was replaced with a collection of mostly charter schools as part of a “radical” experiment that is showing great promise.  It then goes on to talk about the “very promising” improvement that could be a national model for reform. Yet, for whatever progress has supposedly been made, the all-charter Recovery School District in New Orleans still earned a “D” grade from the state for 2013 and remains one of the lowest-performing districts in the state.

In addition, lawsuits have been filed alleging that the district has failed to guarantee equal access to educational services to students with disabilities, and more recently, the U.S. Education Department said it had opened an investigation into a civil rights complaint on behalf of African American parents in New Orleans alleging racial discrimination in the closing of public neighborhood schools and the expansion of charter schools.

Here, for what it’s worth — which appears to be pretty much nothing — is the report that was taken down: