When George W. Bush became president in 2001, we began to hear a lot about what was called “the Texas miracle.” It went something like this: His first education secretary, Rod Paige, had led the Houston Independent School District in a program that gave standardized tests to kids in grades 3 through 8 every year; looked at the scores by race, ethnicity and other characteristics of students; and then either honored or punished schools that didn’t do well. Test scores were reported to have gone up, and so did graduation rates, and it became the basis of Bush’s 2002 No Child Left Behind law, which made the system national with some additions. But research into the “miracle” found that it was illusory, including a 2000 Rand report, which found that big test score gains by Black and Hispanic students were the result of intensive drilling, and that a jump in high school dropouts made the scores look better than they were. So much for the miracle.
This post looks at another example of what the author says could be a new questionable “miracle” in Texas schools: the rise in test scores in the Marlin Independent School District. Daniel H. Robinson and Emily Cole of the University of Texas at Arlington do a deep dive into the recent rise in test scores in the district. Robinson is associate dean of research at UT-Arlington and formerly director of research and measurement at the University of Texas at Austin’s Center for Teaching and Learning. Cole is a graduate student in the mind, brain, and education program at UT-Arlington. This post is longer than most on this blog because it contains details required to make the case.
I asked the Texas Education Agency for comment, which you can see in full at the end of the post. Marlin Superintendent Darryl J. Henson and Marlin City Manager Cedric Davis Sr. did not respond to requests for comment.
(Corrections: An earlier version of the following post mischaracterized how Texas students performed on tests of achievement. They regressed. Also clarifying difficulties in providing tutoring during the school day.)
By Daniel H. Robinson and Emily Cole
Marlin Independent School District hosted an “Achievement Parade and Block Party” on Sept. 1, 2022, to celebrate receiving its best academic achievement grade by the Texas Education Agency (TEA) in 10 years. Miraculously, the school district’s TEA score jumped from 56 (F) in 2019 to 86 (B) in 2022 (no ratings were issued in 2020 and 2021 because of the pandemic). It’s the first time in 10 years the school district did not receive an F, Improvement Required, or Academically Unacceptable rating.
The city of Marlin celebrated the hard work done by students, staff, and teachers over the past three years. “The kids are proud of who they are,” said Marlin City Manager Cedric Davis Sr. “The superintendent and his staff put that pride back into them. They know they can excel in the classroom.” Darryl J. Henson, superintendent of the district, said, “Marlin ISD is the place to be, literally be, if you want what’s best for your child. Our last score was a 56. So, to rise all the way to an 86, a 30-point increase, that is to the testament of our teachers and students and community.”
The fact that the TEA score for Marlin ISD increased is not a big surprise. According to TEA, most schools improved from 2019. In fact, 74 percent of the schools and districts in Texas received either A or B grades. More surprising is that over 1,000 campuses went from a B, C, D, or F in 2019 to an A in 2022. How did grades assigned to schools increase considerably in Texas over the pandemic when scores from the ACT exam and the National Assessment of Educational Progress decreased? Graduation rates also declined. How is Texas scheduling parades to celebrate “improved” performances?
According to Henson: “Aug. 3 was our first day of school, but Aug. 15 was the best day of school. Not only did we rewrite the history of the Marlin ISD, we rewrote the history for the state of Texas.”
Such an amazing result is reminiscent of the widely touted “Texas miracle” that occurred over 20 years ago when Rod Paige was superintendent of the Houston Independent School District. During his 2000 presidential campaign, George W. Bush often spoke about an approach to education, especially in Houston, that led to declining dropout rates and increasing test scores. The approach held superintendents and principals accountable for students’ performance. The “success” of this approach in Houston undoubtedly influenced Bush’s decision to choose Paige to be secretary of education. Houston became the model on which the No Child Left Behind (NCLB) educational reform was based (i.e., more high-stakes testing and school leader accountability).
Unfortunately, for both Bush and Paige, and for countless teachers and administrators who were forced to play the accountability game, there were no actual decreases in dropout rates nor increases in student performance. Rather, the impressive numbers were a result of falsifying student records. For example, although Houston reported a dropout rate of 1.5 percent, the true dropout rate was between 25 percent and 50 percent, researchers found.
Similarly, Houston gamed the system by dramatically increasing the numbers of students who were categorized as special education so their test scores would not be included in a school’s overall test scores. As a result, overall test scores increased for many schools. Why were student records falsified? The incentive system for principals was seductive for fraud. Principals were given only one-year contracts. If they met their goals, they received cash bonuses. If not, they were replaced.
Robert Kimball, an assistant principal at a Houston high school, was the whistleblower who first alerted the press about the Texas miracle hoax. He noticed that his school reported zero dropouts when, in fact, 463 students had dropped out. After his school had reprimanded and demoted him, Kimball filed a whistleblower suit against Houston schools and received $90,000.
Is the Marlin ISD “approach” the next “Texas miracle?” In 2014, the district lost its accreditation status. The state of Texas intervened and installed a monitor, then later a board of managers and conservator, to oversee efforts to improve the district. Henson was hired in 2020 to help save the district.
But has unsatisfactory performance been replaced by satisfactory performance? The overall score of 86 is based on scores in three domains: student achievement, school progress, and closing the gaps. Of the first two, only one domain is counted toward the overall TEA score, whichever one is higher, and counts for 70 percent of the score. Let’s take a closer look at each domain.
In 2021-2022, the Marlin school district received scores of 80 in student achievement, 92 in school progress, and 73 in closing the gaps. Thus, the 92 is weighted 70 percent and the 73 is weighted 30 percent, yielding a score of 86. For comparison, in 2019, the district scored 57 on student achievement, 60 on school progress, and 47 on closing the gaps. In a three-year span, the school progress score increased 32 points and the closing the gaps score increased 26 points. What happened to increase scores so much?
The school progress score is the higher score between academic growth and relative performance. Are you confused yet? Academic growth is determined by the change in student scores from the previous year to the current year. A half point is awarded if the student did not meet growth expectations but maintained performance, whereas a full point is awarded if a student meets the Expected or Accelerated STAAR progress measure expectation. For those students whose performance decreased, they do not contribute points in the calculation of academic growth (perhaps they should be negative). STAAR is the acronym for the series of standardized tests formally known as the State of Texas Assessments of Academic Readiness.
According to the TEA website example, if you have 100 students who took the STAAR reading and math test (200 total test scores), imagine that 17 maintained performance and 134 had expected or accelerated growth. Yes, that is only 151 scores; the other 49 students’ scores decreased. You multiply 17 x 0.5 plus (134 x 1) and get 142.5, divided by 200 is 71. For the third step, you convert this “raw” score to a number out of 100 that’s easier to understand. The example’s “easier” score to understand is an 81 out of 100. Of course, the 71 is also out of 100 as it is a percentage.
Relative performance is based on the number of students who are economically disadvantaged (qualifying for free or reduced lunch). Next, you calculate the STAAR (for elementary and middle schools) or STAAR and College, Career, and Military Readiness (CCMR) results averaged (for high schools, K-12, and districts). Finally, you plot these two scores (percent poverty and STAAR or STAAR+CCMR averages) on a graph. The graph shows how other districts with a similar poverty level perform on the tests. If a particular district scores higher on the tests than predicted based on its poverty score, it receives a high relative performance score.
So, did student performance increase dramatically? Certainly not on the STAAR tests. In 2022, the Marlin Independent School District had a scaled score of 56, which is an F. In 2019, it scored 50, which is also an F. Let’s repeat that just in case someone is under the impression that student achievement dramatically increased for the Marlin district. It went from an F to an F. Imagine if a student’s average grade in third grade was an F and the average grade in fourth grade was an F. Would you schedule an achievement celebration?
That is clearly not enough of an improvement in STAAR scores to move a district from an F grade to a B in the School Progress category. But STAAR scores make up only 40 percent of the Student Achievement grade for high schools. The CCMR score is also weighted 40 percent and the graduation rate is weighted 20 percent. In 2022, Marlin ISD had a scaled score of 98 on the CCMR. In 2019, the CCMR scaled score was 65. That is quite an increase. What is going on with the CCMR scores? Remember, only high school students receive a CCMR score. Elementary and middle school students only receive a STAAR score.
For a student to be designated as CCMR or college, career, or military ready, only 1 of 10 criteria needs to be met. Looking closely at the CCMR for the Marlin district in 2022, it appears that a main driver was 49 students earning credit for taking a college prep course. Suffice it to say that the increase in the overall School Progress score for the Marlin district was mostly caused by the unusually high CCMR score (for only the high school students) and not an increase in STAAR scores.
As previously mentioned, the overall School Progress score is the higher of either the Academic Growth score or Relative Performance score. For the Marlin district, Academic Growth was 76 and Relative Performance was 92. How was Relative Performance so high? Remember that the high Student Achievement score was mainly due to a dramatic increase in CCMR score. The STAAR and CCMR results averaged is then compared to other schools with similar poverty levels. Marlin reported a 99.2 percent poverty rate, so they were compared to the very poorest districts. Due to its sky-high CCMR score, the Marlin district was higher than most other similar schools in other districts.
But what about the Closing the Gaps score? To calculate the Closing the Gaps score, you assign all students to groups based on race/ethnicity, special education status, and whether they are continuously enrolled, emergent bilingual or English learners, or economically disadvantaged. The state has performance targets for each group in four different categories: academic achievement (math and reading), graduation rate, English language proficiency, and school quality. School quality, by the way, is a “readiness for college, workforce, or the military”). This sounds a lot like the CCMR. You then determine how many students in each group met the targets divided by the total number of targets.
In 2022, Marlin ISD received a Closing the Gaps score of 39, which was “scaled” to a 73. This included a raw score of 3.3 for graduation rate and 0.0 for English language proficiency. Most of the score of 39 came from the School Quality raw score of 30, which means that 100 percent of the student groups met this target. Again, the enormous increase in CCMR scores surfaces as the reason for such a huge increase in the Closing the Gaps score.
Taken together, the huge increase in the TEA score for Marlin ISD was not the result of higher student achievement on the STAAR test. Like the rest of the country, Texas students regressed on tests of achievement. Instead, the dramatic 30-point TEA score increase for Marlin ISD was mostly caused by much higher School Progress and Closing the Gap scores. Each was heavily influenced by the CCMR score which increased most dramatically from 2019 to 2022.
What is more of a head scratcher is that only the high school students receive CCMR scores. How can the high school so heavily influence a district’s TEA score? It is because TEA basically treats districts like a giant high school, where CCMR + graduation rate counts more than the STAAR. Encouraging more students to take college prep courses might be an easy way to game the system.
Mike Morath, TEA commissioner, was quoted as saying, “These results show our state’s significant investment in the post-pandemic academic recovery of Texas public school students is bearing fruit.” But are the “results” really indicative of success? At minimum, such unbelievable gains at other schools and districts deserve a more thorough investigation.
Is Marlin ISD now “off the hook” in terms of worrying about low-performing students? Not really. The Texas Education Agency knows what these school and district ratings really mean, which is why they created HB4545 in 2021 which requires students who failed a STAAR test to receive 30 hours of supplemental accelerated instruction (tutoring).
So, Marlin Elementary School, which has an overall TEA score of 83, but also earned a "not rated" for school achievement, has a lot of work to do. Its students failed 76 reading tests, 84 math tests, and 39 science tests. That is 199 failed tests at a rate of 30 hours per test for a total of 5,970 hours of required tutoring.
The state statute requires that students not be removed from recess or from the foundation curriculum or enrichment curriculum as defined in TEC, §28.002, the state’s education code. For elementary schools this would require a complete restructuring of the school day to accommodate additional tutoring, or extending the school day which is one suggestion from TEA for schools to meet the requirements of HB4545. So now Marlin Elementary School, like hundreds of other Texas schools, must find a way to provide thousands of hours of additional tutoring. Fulfilling this requirement seems like another impossible task for schools, teachers and students.
Should teachers, administrators, and students at high-poverty schools feel bad if their improved TEA scores turn out to be too good to be true? Certainly not. Assigning grades to schools is ridiculous. Shaming high-poverty schools has never worked in terms of turning them around. The notion of the 90/90/90 (90 percent poverty, 90 percent ethnic minority, and 90 percent passing rate on academic standards) turnaround school is a widely acknowledged myth.
It is a unicorn that some have reported seeing but, upon a closer look, has proved to be an educational Piltdown man (Glass, 1968). The Marlin Independent School District certainly has the first two percentages in terms of poverty and ethnic minority. But its academic performance is still low (an F). Gaming the TEA score by having several high school students take a college prep course does not solve the problem. Rather than celebrating, we should be (1) investigating how many other high-poverty Texas districts have similarly been awarded A and B grades by TEA and (2) looking for solutions.
Here is a comment from the Texas Education Agency:
“Texas’s A-F accountability system includes multiple indicators to measure the performance of districts and campuses, including: STAAR assessments, graduation rates, and measures of college, career, and military readiness like dual credit classes and SAT scores. The A-F system is structured so that ratings are based both on percentages of students achieving a certain level on these indicators, and on the percentage of students who gained a great deal academically even if they didn’t achieve at a high level. In this way, ratings reflect both overall student achievement and the impact of highly effective educators.
“Marlin ISD saw notable jumps in the percentage of students who gained a year academically, up to 68 percent of students in 2022 vs. 56 percent in 2019. While nearly the entire student population of Marlin ISD is economically disadvantaged, 91 percent of its most recent graduating class demonstrated readiness for college, career, or the military. One measure of college readiness required under state law is that students successfully complete a course approved by a college that indicates that students are ready for college-level work, which fully half of Marlin’s graduating seniors successfully did.”