(Photo by Sarah L. Voisin/The Washington Post)

The recently released reading and math scores from the 2015 National Assessment of Educational Progress caused some consternation in the education world because they went down for the first time in the history of the NAEP. The exam is often called the nation’s report card because it is the only measure of student achievement given periodically to a sampling of students around the nation. Here is a piece about what the drop in math scores tell us about the Common Core State Standards, which have been implemented in most states for the past few years, and about the Core’s relationship to the NAEP.

[U.S. student performance on NAEP slips]

[What the national drop in NAEP scores means]

It was written by Sarah Lubienski, a professor of mathematics education at the University of Illinois at Urbana-Champaign.  Her research focuses on inequities in students’ mathematics outcomes and the policies and practices that shape those outcomes.  She conducts large-scale studies using national data sets, as well as smaller, classroom-based studies.  Lubienski has chaired the National Assessment of Educational Progress (NAEP) Special Interest Group of the American Educational Research Association and is a member of its Grants Governing Board.  Her previous NAEP analyses have been funded by the National Center for Education Statistics and the Institute of Education Sciences. She is co-author of The Public School Advantage: Why Public Schools Outperform Private Schools.

 

By Sarah Lubienski

The one- to two-point point drops in math and reading scores on the National Assessment of Educational Progress announced last week have caused quite a commotion, with various critics blaming school testing, accountability, choice and other policies prominent over the past decade. Departing Secretary of Education Arne Duncan and other defenders of recent policies have pointed to early implementation issues and demographic/economic shifts as possible reasons for the disappointing scores. The results were described as a “train wreck” by Rick Hess and a “fiasco” by Diane Ravitch, with many critics seeing them as evidence that the education policies of President Obama as well as his predecessor, George W. Bush, have been counterproductive.

If we dig deeper into the data, we can get a clearer sense of recent trends and the likely reasons for them. Although most reports of NAEP mathematics performance focus on overall math scores, NAEP actually tracks subscores for student performance in five strands: Number/Operations, Algebra, Geometry, Measurement and Data Analysis/Probability. In contrast with the “back to basics” era of the 1970s and 80s, which focused on number computation, in 1989 the National Council of Teachers of Mathematics (NCTM) promoted the importance of teaching these five strands throughout Pre-K-12th grades.

This was a major shift for schools, and it makes sense that, after the NAEP test became aligned with the new NCTM standards and schools began to align their curricula and assessments with those standards, we would see a steady increase in NAEP scores. And that is, indeed, what happened. Children do learn what we teach them.

Probability and statistics, in particular, was quite new to the school mathematics curriculum, especially in early grades. Probability is critically important for making smart consumer, health, and political decisions and yet adults are notoriously prone to misconceptions about probability. For example, if you flip a coin and gets five heads in a row, you are NOT more likely to get tails on the next flip – the coin does not remember. Hence, developing students’ understanding of probability from an early age seemed like a good idea when NCTM introduced it back in 1989. On the other hand, emphasizing every math strand every year contributes to the “mile wide, inch deep” problem that the Common Core State Standards tries to address.

In fact, when the Common Core State Standards were released, one of the most striking shifts in the mathematics curriculum was its stance that probability instruction should be delayed until Grade 6. Within the small world of mathematics education experts who analyze NAEP, these new standards raised the question, “What will happen to NAEP scores on the data analysis/probability strand?”

An examination of the NAEP subscores for each of the five strands (via the NAEP Data Explorer) tells us the answer. Figures 1 and 2 (see below) show trends in NAEP scores on each of the five math strands for public school students, as these students are most affected by national policies. As shown in the graphs, Data/Probability scores dropped a highly significant four points at Grade 4 and five points at Grade 8.  In fact, this the only strand to drop more than two points at Grade 8.  In other words, drops in the 2015 Data/Probability scores were major contributors to the declines in math scores at both Grades 4 and 8.

On the one hand, these patterns point to the Common Core standards as a factor in the score drop. On the other hand, those using the small, one- to two-point NAEP drop to condemn Common Core and all recent policies might be missing the point. However, those who try to explain away the drop as due to demographic or economic shifts, are also missing the point.

Instead of blanket condemnation or excuses, we can use the latest scores to diagnose weaknesses in our curricula and ensure that we are assessing the things we currently care about.  For example, if we as a nation do not think that fourth graders need to know probability, then why are we testing it?  There are disconnects between the Common Core and NAEP that merit serious consideration, as highlighted by an AIR report released last week.

On the other hand, the fact that the drop in the Data/Probability strand was particularly large at Grade 8 means that the Common Core philosophy of delaying instruction in this area might be problematic.  However, it might also be too soon to tell, given that current eighth-graders started school before the advent of CCSS (something Arne Duncan should have considered more carefully before crediting the Common Core for the 2013 scores). At any rate, the data suggest that trends in our students’ understanding of probability need to be carefully monitored.

More generally, this brief history of mathematics education reform and NAEP trends raises the question of how much improvement one can expect on any assessment over the long haul. It is not surprising that when we aligned the NAEP test to a new mathematics reform in 1990, we saw strong gains in mathematics (and not reading) as schools across the nation began to implement new instructional materials and methods.

It is also not surprising that we would see a plateau in these gains at some point, as the reform saturates the schools. When the focus in schools shifts to other instructional reforms less aligned with the NAEP test, we should expect to see declines, which can then prompt us to consider the content we want our students to learn and whether the test is, indeed, assessing that content. This question is largely about the outcomes we value, as opposed to about policies “working” or not.

This look from a mathematics education perspective might seem to be in the weeds a bit, but it is a call for a more nuanced discussion of the NAEP data and for making maximal use of our national assessments to improve curriculum standards and other policies that affect our nation’s schools.

 

Figure 1: 4th Grade Public School Achievement by Mathematics Strand, 1990-2015

 


 

Figure 2: 8th Grade Public School Achievement by Mathematics Strand, 1990-2015


 

Data are from the NAEP Data Explorer

¹ Testing accommodations were not permitted for the 1990 and 1992 assessments. A score for the Data strand was not available in 1990.

 

 

 

Data are from the NAEP Data Explorer

¹ Testing accommodations were not permitted for the 1990 and 1992 assessments. A score for the Data strand was not available in 1990.

 

 

Figure 2: 8th Grade Public School Achievement by Mathematics Strand, 1990-2015

 

Data are from the NAEP Data Explorer

¹ Testing accommodations were not permitted for the 1990 and 1992 assessments.