(Correction: Sherman Dorn is now at Arizona State University, having moved from the University of South Florida).
How successful were Bush’s reforms in Florida? He, of course, says they were enormously successful. Others don’t. To understand what really happened, I had an e-mail conversation with professor Sherman Dorn at Arizona State University (formerly of the University of South Florida), who has spent years researching and writing about public education in the Sunshine State. He maintains a blog about public education at www.shermandorn.com.
Here’s our conversation:
Q) Let’s start with the basics. When Jeb Bush became governor of Florida in 1999, how did he proceed in terms of school reform?
A) In his first term, most of Jeb Bush’s efforts in education came in three areas: test-based accountability, private-school vouchers, and support for improved reading instruction. In 1999, Bush signed legislation that required annual testing of all children in grades 3-10, tied test scores to annual “A” through “F” labels assigned to local public and charter schools, and required retention of children in third grade if they did not meet critical scores in the state reading test or provide other evidence of reading skill. In the same year, the Florida legislature created two voucher programs, one tied to the state labeling of local public schools and the other available to children with disabilities. Bush also created the Florida Center for Reading Research in 1999, which used both state and federal funding to support classroom teachers and reading coaches.The real-estate boom in Florida at the time made it relatively easy for the state legislature to add funding in the form of bonuses for teachers in schools labeled A (or in schools with improved labels), and to support the hiring of hundreds of reading coaches in Florida’s elementary schools. Bush left office at the peak of the boom years and never had to face budget crises that are a regular part of state politics now.
Q) Bush frequently talks about how his test-based policies led to higher test scores. I’m not sure if he was referring to NAEP or to FCAT. What happened with the test scores and the achievement gap?
A) Most of the time that Bush or his policy advocates talk about Florida children’s achievement, they refer to NAEP. Attached are some relevant materials taken from the NAEP website, comparing reading and math scores between Florida’s students and the rest of the country for grades 4, 8, and 12 for the years where Florida scores can be separated out and where NAEP officials see the relevance of comparisons (i.e., when accommodations for disabilities were allowed — since 1998 for reading and since 2000 for math).First, on general levels of test scores:Governor Bush and his allies generally point to fourth-grade reading as the most important story, and that is where one can see large increases in average scale scores, not only across cohorts of fourth-grade students but in comparison with the national sample of fourth-grade students. Between 1998 and 2013, Florida’s fourth graders rose from being quite a bit below the national average on the NAEP testing program to being well above the national average. You can quibble with testing samples and comparison issues, but this is an unambiguous good.The picture is less optimistic when you look at reading in eighth grade or math at either fourth or eighth grade. NAEP reading scores for Florida eighth graders slowly converged to the national average, with large bounces up and down across the years. That’s good if less impressive than fourth grade. Fourth-grade math scores went from just about average in 2003 to just about average in 2013, with a few test years above the national average. Eighth-grade math scores tracked well below average at every NAEP administration from 2003 to 2013.Because 12th grade state comparisons only exist for 2009 and 2013, I am providing the images but not commenting on them. You cannot draw any conclusions about Bush’s administration from them.Second, on achievement gaps:I have also attached some spreadsheets on achievement gaps in NAEP scores by eligibility for free and reduced lunch, looking at 1998 and 2013 as the comparison points. While the average NAEP reading score gaps by lunch-program eligibility closed somewhat in both fourth- and eighth-grade reading, NAEP officials say that it is too close to the national trend to say that one can draw any inference that there was a greater change in Florida than the rest of the country (i.e., any change in the gap is not statistically significant in comparison with national changes). The group of states that cluster with Florida in any of those rankings are a motley group: NY, DE, WV, FL, LA, and AZ for fourth-grade reading; MD, TN, CA, GA, FL, DE, and UT for eighth-grade reading. (If you’re interested in math, that group would be WI, NY, NJ, FL, and GA for the 2003-2013 comparison.) It’s hard to draw any conclusions from that, other than the achievement gaps haven’t closed much in the entire country since Jeb Bush became governor, and Florida isn’t worse than the entire country in that mediocre track record.The bottom line: Bush is correct that Florida’s children benefited from his time in office if children graduated high school at the end of fourth grade, and only evidence of general reading skills mattered. For most other independent test-score measures, the picture is less impressive.And that’s even before we get to the question of what is responsible for the rise of fourth-grade reading scores in Florida.
Q) So what was responsible for the fourth-grade rise in reading?
A) The most likely explanation is a combination of reading coaches hired in the boom years in Florida and the creation of the Florida Center for Reading Research (FCRR). FCRR provided technical support for the teaching of reading, and its activities started with the support of primary-grade students. With extra state and local revenues provided by the real-estate boom at the time, elementary schools were able to hire reading coaches. It is important to credit Bush for pushing for the creation of the FCRR. It is a shame that he never mentions that in public today. Bush usually credits all of his other policies, but the other policies affected multiple grades and subjects, all of which pale in comparison with the achievements in primary-grade reading instruction.
Q) The former governor talks about closing the achievement gap, especially with Hispanics. Did that happen? Bush recently said this: …
Or take my beloved home state of Florida. Working with the Florida Legislature we implemented a bold suite of reforms, starting with the A+ Plan for Education when I first became governor in 1999.Florida went from a national failure to a Top 10 state in education.Today, our low-income fourth graders lead their peers in every other state in reading, according to the latest results from the National Assessment of Educational Progress.Our Hispanic fourth graders do better than or equal to the average student in 34 states and D.C.Our African American fourth graders have advanced 2 ½ grade levels in reading since our reforms began.We are a national leader in providing disadvantaged students access to Advanced Placement classes.Those in this room know what works.We have built a nationwide reform movement based on a set of proven principles.
I know his fourth-grade comment about leading their peers everywhere is wrong. Also, it doesn’t actually sound like the achievement gap CLOSED any.
A) I focused on fourth-grade reading, where there is the best evidence for improvement in Florida children’s achievement during and since Bush’s terms. For fourth-grade reading looking at NAEP, there is evidence of gap-closing for children in low-income households and students with disabilities, and reduction of the gap at a faster pace than the nation as a whole. While there is evidence of White-Black and White-Hispanic gap shrinking, it is roughly at the same pace as national changes.
|Florida 1998 gap (mean scale score)||U.S. 1998 gap (mean scale score)||Florida 2013 gap (mean scale score)||U.S. 2013 gap (mean scale score)||Florida gap change, 1998-2013||U.S. gap change, 1998-2013|
|Free-reduced lunch (eligible vs. not eligible)||29||31||24||29||-5||-2|
|Students with disabilities (with IEPs vs. not)||38||41||27||42||-11||1|
How to read the “gap change” column: In Florida between 1998 and 2013, the gap in average reading scale scores shrank by 5 on the NAEP fourth-grade reading scale for the difference between children eligible for free or reduced-price lunches and children not eligible for the subsidized-lunch program. Negative numbers mean a calculated shrinking of the gap. Positive numbers mean an estimated expansion of the gap.
Bush is correct that Hispanic fourth-grade students had higher average reading scores on NAEP in 2013 than the nation; the same was true in 1998, before he became governor, so he is both correct in comparing Hispanic scores in Florida to the country as a whole and also incorrect if he claims that the White-Hispanic gap closed more in Florida than for the country as a whole, at least since 1998 in fourth-grade reading. It is important to keep in mind that there are still significant achievement gaps by measures of poverty, disability, race-ethnicity, and language. (Language does not appear in the table above because of insufficient information for 1998.) Moderate closures of achievement gaps are important and also not enough.
An important caveat: Looking at achievement gaps in NAEP and changes in those gaps is harder than you might think because some category definitions change, the demographics of children change (a higher proportion of children are eligible for free and reduced lunches than in the late 1990s), and once you look at differences in scores (gaps) and changes in those differences, the standard errors of those measures expand from the standard errors in the mean scale scores. The numbers above are far less precise than one might assume; for example, while the changes in achievement gaps by lunch-program eligibility and disability status are meaningful, take the specific numbers with more than a few grains of salt.
You may also be interested in: