Another day, another fight in the education world. This one is worth delving into because it is really not about who said what but about fundamental understandings — and misunderstandings — of standardized testing data and how it drives policy.
NAEP, as the test is known, is sometimes referred to as “the nation’s report card” because it is seen as the most consistent measure of U.S. student achievement since the 1990s. It is administered every two years to groups of U.S. students in the fourth and eighth grades, and less frequently to high school students. When Loveless told her that NAEP proficiency scores do not refer to grade level, a social media fight ensued between Campbell and her critics.
In this post, Carol Burris, a former award-winning high school principal who got involved in the Twitter exchange, explains why the substance of this debate matters.
I asked Brown to comment about her statement that two out of three eighth graders cannot read or do math at grade level and why she thinks NAEP proficiency means grade level. She said in an emailed response, which you can see in full below, that “if I were trying to be completely and utterly precise then I would have specified ‘grade-level proficiency,’ instead of ‘grade level’ in the context of NAEP score,” and that “any reasonable person or parent” would understand what she meant.
By Carol Burris
“Two out of three eighth graders in this country cannot read or do math at grade level. We are not preparing our kids for what the future holds.”
The above was the lead message that Slate used to introduce education activist Campbell Brown’s “advice for the next president” video that it posted. The claim is false.
When Tom Loveless, a senior fellow at the Brookings Institution, a nonprofit think tank in Washington D.C., asked Brown to retract her assertion, an angry, and sometimes amusing, series of Twitter posts and blogs began. Loveless is a former teacher and Harvard professor who is an expert on school reform and student achievement.
Brown’s resistance to correct the record and her dismissal of Loveless’ request is a story worth telling. It speaks to the problems that arise when advocacy is demanded due to philanthropic funding of a news website, and it speaks to how rhetoric drives the reform agenda, while dismissing any critique as an attack.
Brown was a journalist for years with NBC News and an anchorwoman for CNN, and for a time, had her own series for CNN, entitled “No Bias, No Bull.” The show was canceled in 2010, and Brown reemerged in 2014 speaking out in favor of charter schools and against teacher tenure.
In July 2015, The Washington’s Post’s Paul Farhi reported that Brown was leading a new education news agency called the Seventy Four. In his description of the new venture, Farhi questioned whether the Seventy Four reported news or advocacy. He used as examples Brown’s statement that she felt no need to cover both sides of every story; the content of the initial reports on the Seventy Four’s site; and the agency’s funders, which include the Dick and Betsy DeVos Family Foundation (Betsy DeVos leads a school choice PAC), the Walton Family Foundation and Bloomberg Philanthropies.
I heard about Campbell’s exaggerated claim of student failure when Loveless emailed me and told me about it. He had asked Brown to justify her claim, and she replied that it was based on eighth-grade “proficiency” rates from the National Assessment of Educational Progress. Loveless, who regularly issues research and policy papers on NAEP, was concerned that Brown had resurrected the myth that NAEP proficiency represents “grade level.” His attempts to get Brown to correct her statement were fruitless.
I joined Loveless in asking Brown via Twitter to correct the record.
I've studied test scores a long time. I know of no data to support the "two thirds of 8th graders" charge.https://t.co/a3xrVJmMPk via @slate
— Tom Loveless (@tomloveless99) May 17, 2016
And one of my Tweets
.@campbell_brown NAEP prof is NOT grade level. Never was meant to be. Set record straight @The74 @tomloveless99 https://t.co/o5Dz7dMPIo
— Dr. Carol Burris (@carolburris) May 17, 2016
And this was Brown’s reply:
@carolburris @The74 @tomloveless99 no. I wont engage w/ you who launch disparaging personal attacks on those who disagree #notworthmytime
There were other tweets as well from both sides, and the online conversation deteriorated. Putting aside the tone of the responses, the fact is that Brown is wrong — and her inaccuracy is not merely a matter of semantics. This is why.
What does NAEP “proficiency” represent?
NAEP tests produce a continuum of scores, much like the SAT. In order to make scores more understandable to the public, the four categories (Below Basic, Basic, Proficient and Advanced), each aligned with performance descriptors, were created and assigned to four ranges of scores by the NAEP governing board (NAGB). In doing so, the NAGB made it clear that “Proficient is not synonymous with grade-level performance.”
Loveless, who has written extensively about NAEP, said the following in his email correspondence with me:
“The cut point on NAEP is much too high [to be considered grade level].
In a 2007 study, researcher Gary Phillips projected where scores on the TIMSS, a series of international math and science given to kids around the world, would land on the NAEP scale. He estimated that 27 percent of Singapore’s 8th graders would fail to meet the NAEP proficient cut score in math. At the time, Singapore was the highest scoring country in the world. Japan — not exactly a weak math country–would see only 57 percent meet proficiency; 43 percent would “fail.” You can read more about that study on pp. 10-13 of the 2007 Brookings Report authored by Loveless that you can find here.
Education historian and activist Diane Ravitch also weighed in and asked Campbell Brown to correct the record. Ravitch served on the NAEP Governing Board for seven years. She also disputed Brown’s claim, and explained what proficiency means here on her blog.
Why is equating NAEP proficiency with “grade level” a problem?
Brown appears to believe that NAEP proficiency is the reasonable and right standard that every child in America should meet or exceed. Unfortunately, she is not alone. Common Core tests are more often than not aligned with NAEP proficiency. That has a backwash effect on curriculum and teaching. But as Loveless and Ravitch point out, NAEP proficiency is neither a reasonable nor attainable standard to expect every child to meet.
If you set an unreasonable cut score on state tests and pressure teachers and kids to meet it, it can work against student learning, especially for students who struggle. Sizeable numbers of kids will learn less than they might if the instructional pace and content were developmentally appropriate and well sequenced. This is especially true in the younger grades. You can always differentiate instruction if you need to for the kids who excel, but if you hold struggling learners to a standard they cannot meet, frustration, not learning, happens. Inappropriately difficult standards also promote drilling for the test, adult cheating and the narrowing of the curriculum as explained in this Politico New York report on the test-driven, Success Academy Charter Schools.
In New York, where Brown lives, the problem is especially acute. New York reformers brag that they have set the test-score bar higher than NAEP, with former schools Chancellor Merryl Tisch pronouncing, without rationale, that proficiency tells parents where their children “need to be.”
That is why New York scores on the Common Core tests are not progressing, the achievement gap is growing, parents are upset with instruction, and so many New Yorkers are opting their kids out of the test.
The above is not an argument against teaching challenging and enriched content at every grade level. There are limits, however, and responsible educators respect them. The NAEP standard of proficiency is inappropriate for making judgments of grade level performance, and even further damage is done when this unreasonable goal is used to create public dissatisfaction in order to justify wrongheaded policies.
I do not doubt that Campbell Brown and her allies want better schools for kids. I am equally passionate in wanting our public schools to be more successful with all students. The difference is not the goal, but the means by which to accomplish it. Brown and her allies embrace privatization and the commercial mindset, while I believe that equity reforms embedded in democratically governed public schools are the most effective and ethical means to improvement.
It is a mirror of the Sweden vs. Finland debate, one that Sam Abrams, the director of the National Center for the Study of Privatization in Education at Teachers College, Columbia University, describes so well in his outstanding book, “Education and the Commercial Mindset,” recently released by Harvard University Press.
In the late 1990s, Sweden embarked on a course of privatization as the driver of school reform. The country embraced choice, corporate reforms, vouchers and privatization. Former Florida governor Jeb Bush and current Louisiana state education Superintendent John White were fans. Rupert Murdoch and Joel Klein, the former chancellor of New York City schools, went to the country to see how Swedish schools put self-paced curricula on computer tablets with minimal instruction provided to students by teachers.
By 2011, however, the model came under fire. The country went into “PISA shock”—Sweden was the only nation in the Organization for Economic Cooperation and Development to see its scores decline every time on that international test since PISA began in 2000. PISA is the Program for International Student Assessment, given every three years to 15-year-olds in dozens of countries and school systems to test their performance on mathematics, science, and reading. Abrams describes scandals and bankruptcies, grade inflation due to school marketing, higher costs, increased segregation, and patterns of clear advantage for the children of savvy parents. The municipal schools were left to educate the neediest children—an unequal system had gotten much worse.
Finland, across the gulf, however, chose equity reforms over commercial reforms. The Finns rejected privatization and chose smaller class sizes, higher teacher pay, no curriculum tracking until Grade 10, establishing the school as a community centerpiece, free hot lunch for all students, strong university-based teacher preparation programs, the elimination of “school inspections” and the limiting of testing to “micro-samples” across all areas of curriculum including music and the arts. Finnish students consistently earn top or near top scores on PISA in reading, math and science. They outscore their Nordic neighbors, who have demographically similar populations.
The reason to oppose the reforms that Brown and her allies espouse is because the inequitable outcomes of Sweden are inevitable if you follow their course of reform. We are already seeing those outcomes emerge. In the education debate, we are Swedes and Finns. And the philosophies are so dramatically different it is difficult, if not impossible, to find middle ground.
Despite those differences, however, if Brown assumes the responsibility of giving counsel to the next president, getting her facts straight is not too much to ask. For better or worse, parents really do want “no bias, no bull,” the former trademark of Campbell Brown.
Here is Brown’s response to questions posed to her about her statement that two out of three eighth graders cannot read or do math at grade level and why she thinks NAEP proficiency means grade level. Here is her full emailed response:
“The histrionic reaction to the distinction between “grade level” and “grade level proficiency” begs the question: is this all you’ve got? You’ve lost the debate on charter schools. You’ve lost the debate on special protections you want for abusive teachers. You’ve lost the debate on tenure. Again, this reaction screams desperation. If I were trying to be completely and utterly precise then I would have specified “grade-level proficiency”, instead of “grade level” in the context of NAEP scores. But any reasonable person or parent can rightly assume that if their child is not reading at grade level, then their child is not proficient. Any reasonable person or parent knows exactly what I meant in that statement. That the people who disagree with my characterization would react by attacking me personally with sexist insults speaks volumes. Those feigning outrage over the difference between “grade level” and “grade level proficiency” are the people who profit off the system’s failure and feel compelled to defend it at all costs. Sadly, in the age of Donald Trump and Diane Ravitch, this is what constitutes discourse.”
Correction: Fixing name of center at Columbia University