wpostServer: http://css.washingtonpost.com/wpost

The Post Most: Local

Answer Sheet
Posted at 04:00 AM ET, 09/13/2011

A primer on navigating education claims

This was written by Paul Thomas, an associate professor of education at Furman University in South Carolina. A version of this post was published on the Daily Kos blog.

By Paul Thomas

For all stakeholders in U.S. public education, debates about education and education reform can be taxing, confusing, and ultimately circular, resulting in little that we call productive.

Recently we’ve seen the reform debate being recycled with patterns similar to those seen surrounding the 2010 release of the documentary Waiting for Superman ,” except this time the focus is on Steven Brill’s new book “ Class Warfare .”

This is made clear by Rick Hess, director of education policy studies at the conservative American Enterprise Institute, who,, on his Education Week blog, Straight Up, addresses not only Brill’s claims, but also the debate about the debate:

“In Brill’s telling, the ‘anti-reformers’ and ‘deniers’ are nothing more than a massive obsidian block squatting in the way of his noble (if ill-defined) forces of light. If Brill had been angling to pen anything other than a black-and-white shoot ‘em up, he could’ve told a much more informed, intriguing, and constructive tale, and one which would have made his conclusions feel more like an extension of the narrative and less like an afterthought.”

Here, then, I want to offer some guidelines for navigating the education debate based on my own experience as an educator for nearly three decades (almost two decades as a high school teacher and another decade in higher education/teacher education) and my extensive work as a commentator in print and on-line publications.

When you confront claims about education, and the inevitable counter-claims, what should you be looking for?

• Are the claims and counter-claims framed within the perspective of the person making them?

Claims or suggestions that a commentator or scholar is being objective should be met with deep skepticism. For example, when I make claims and comments about “no excuses” charter schools, I clarify that I will never value any claims of success in those schools since they practice an ideology I reject on ethical grounds; this is not a claim that “no excuses” schools do not have results that are positive, but that I cannot excuse the methods just for the results.

• Are educational claims framed as “miracles”?

This is likely the greatest red flag of all. Despite recurring claims of miracles — the Texas miracle, the Chicago miracle, the Harlem miracle — these claims tend always to fall apart once examined closely (see the miracleschool wiki). I have concluded that claims of miracle are a corrosive side effect of the growing need for advocacy among schools, particularly charter schools.

• Are the claims of educational quality expressed in terms of correlation or causation?

One of the most problematic elements of public discourse about research is the careless conflating of correlation and causation. While making claims that charter school X increased graduation rates may be a statistically accurate claim, suggesting or claiming that the status of the school as a charter school (compared with a public school, for example) caused the higher graduation rates is a tall order — statistically difficult to produce. Most research studies of quality make the distinction and offer caveats about causation that are ignored or hidden in public accounts, particularly when advocates choose the study primarily to promote an agenda instead of genuinely engaging in a discussion.

• Do the claims address student populations being addressed?

Another careless aspect of public debates about education is the knee-jerk suggestion that School X did it so everybody else can too. Many claims about charter schools, for example, fail to reveal the disparity between student populations when compared to public schools.

For example, many public charter schools now serve high-poverty populations that tend to be primarily minority children. Claims of strong outcomes with these students seem to suggest that traditional public schools that don’t produce similar results are failing — until you discover that the charter schools do not serve the special needs or ELL populations that the traditional public schools must serve. This does not necessarily discount the success of the charter schools with the high-poverty students, but these facts do discredit implications or claims about public schools functioning under much more complex and different circumstances. In other words, if the charter school serving the high-poverty minority population had to add special needs and ELL students, the strong results could (and would) likely disappear.

• Do claims of education success by non-public schools address issues of scalability, selection, attrition, stratification/re-segregation of students, and out-of-school factors?

Each time claims are made about reforms or non-public schools producing positive results (high test scores, increased graduation rates, closing the gap among races or classes, etc.), those claims must factor in each of the parameters above, or the comparisons mean little. If Charter A produces results that are not scalable to traditional schools, then those results are credible for that situation only, but not for widescale reform policy. In the general population still, most people believe private schools outperform public schools (they don’t; the truth is complicated), but that misconception is driven by a failure to consider the populations of students served and the impact of out-of-school factors on measurable student outcomes.

• Do counter-claims made about education commentaries start with fair and accurate characterizations of the positions being debated?

Over the past two years, a disturbing pattern in the education reform debate has been the tendency to characterize anyone criticizing Education Secretary Arne Duncan or Bill Gates as one of the following: anti-reform, defender of the status quo, union mouth-piece, or fatalistic (suggesting that somehow by acknowledging the influence of out-of-school factors, the person is essentially saying that children in poverty cannot learn).

Many pundits take this strategy: Mischaracterize the person’s position and then attack your own mischaracterization. This has become a common pattern in the education debate as well (I recommend scanning the comments sections of on-line commentaries about education to see how powerful this technique is).

• What are the experiences and credentials of the person making the claim?

Many people can and even should offer insight on topics such as education, but it is likely that medical doctors know more about medicine than engineers — and it is likely that people having taught in public education for decades, spent their lives both gaining and working in education are more credible in their claims than people without those experiences or education. The pontifications on education of career politicians and entrepreneurs should be taken for what they are: pontifications on education of people who aren’t working in education.

• Are claims supported with evidence — citations, hyperlinks, or both? One of the commitments I have in my commentary/Op-Ed work is to bring the same qualities of scholarship to popular discourse that academia requires. I prefer on-line publications since I can insert hyperlinks and encourage readers to examine the basis for my claims themselves. This lends credibility to my claims, but it also provides me with learning opportunities since many people do as I hope and then provide me nuanced and new understandings about the topics I address.

Just as our public schools appear to be mired in conditions that never change, our public debates about education and education reform suffer from insular and unproductive cycles of monologues.

Our public schools need and our children deserve genuine school reform — reform that is nuanced and complex — and without the same nuance and complexity in an authentic dialogue about education and education reform, we are unlikely to reach the school reform we need.



Follow The Answer Sheet every day by bookmarking http://www.washingtonpost.com/blogs/answer-sheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page. Bookmark it!

By  |  04:00 AM ET, 09/13/2011

Read what others are saying

    © 2011 The Washington Post Company