Oklahoma City educator John Thompson, my guest columnist, has some thoughts on the magic of statistics, celebrated in the book and film “Moneyball,” and how they work, and don’t work, in schools.
So-called school reform came of age when numerous professions were under the sway of data-driven decision-making. Since those methods helped produce the financial bubble and the Great Recession, they have lost their luster. Only in education does the simplistic faith in numbers still survive.
The data boom culminated in Michael Lewis’s “Moneyball.” An Atlantic Wire discussion about the effect of sophisticated statistics on baseball can benefit our education debate. These baseball analysts argue that when actually playing the game, “stats only go so far” because they “are much better at illustrating the past than predicting the future.” Numbers cannot measure “infectious optimism,” or a player’s (or a teacher’s) “sheer presence.”
I would add two points regarding the similarities between sports and schools. Perhaps the best way to use numbers in the classroom is to turn them into visual aids. Student performance statistics can be the equivalent of a flashy, high-tech scoreboard. Back when my class discussed Lewis’s “Moneyball,” teachers were being urged to use charts and other “bells and whistles” to transform some types of learning into a game. The students enthusiastically endorsed those efforts. They were less supportive of the data-driven gamesmanship of NCLB that would drive much of the fun of learning from the classroom.
Secondly, the focus of data-driven accountability was on increasing test scores and “Moneyball” was more effective in putting larger numbers on the scoreboard. Bill James, the father of “Moneyball’s” statistics, says that his unsentimental emphasis on “on base percentage” as opposed to traditional tactics such as base-stealing and relying on “clutch hitting,” has held up well. James admits, however, that he missed the importance of defense — of not booting the ball and letting the opponents put numbers on the board.
Metaphorically speaking, school reformers have done the same thing. Accountability devotees have to run up the points that they attribute to their innovations, while ignoring scores that they gave up through hare-brained experiments. Reformers invest in test prep, curriculum narrowing and questionable credit recovery programs to increase the standardized test pass rates and graduation rates. They have ignored the costs of those tactics, however. In other words, reformers want to swing away but not be accountable for strikeouts. Reformers have thus committed a series of unforced errors that have compromised the integrity of the learning process.
Baseball experts agree that statistical analysis has changed baseball’s front offices for the better, but, still, money trumps data. I can’t understand why reform-minded administrators did not take the logs from their eyes by using data to improve central office procedures before addressing the motes in teachers’ eyes. Just as baseball stats were best used to organize teams, the accountability hawks should have used new statistical procedures to help the bureaucracy efficiently support teaching and learning, not micromanaging classroom activities.
The baseball commentators concluded that “the lure of statistics has had a dramatic impact for the worse for fans” who have “never felt or contributed to the rhythm of the game” (or the classroom.) The same applies to policy analysts who just happened upon the field of education and jumped to the conclusion that they could use the methods of their professions to close the achievement gap. Self-styled saviors of education were quick to embrace experimental statistical models, although they knew little about the concrete realities of schools. They were decidedly uncurious about the practical wisdom of practitioners.
The best wisdom to come from baseball analysts is a reminder of the lessons of Social Science 101. Data-driven reformers have largely ignored the definitions of data and numbers. These sports commentators note that, “numbers don’t negate stories. Numbers are stories, a narrative way to process and describe reality.” We need numeric and verbal storytelling to understand the mechanical side of sports, as well as teaching.
When it comes to explaining human behavior, these stories can give us control — or, more accurately, the illusion thereof. I would add another point about the biggest misunderstanding of data in education. The sophisticated use of stats in sports built upon generations of experience in coaching players to improve. School reformers, however, first tried to reward and punish with numbers, without taking the obvious step of using numeric and verbal storytelling for coaching to improve the performance of schools and educators.