Three years ago, a scientific study claimed that storms named Debby are predisposed to kill more people than storms named Don. The study alleged that people don’t take female-named storms as seriously. Numerous analyses have since found that this conclusion has little merit.
When the study came out, I reported on it, and it generated tremendous interest. Because of the Internet and the tendency for interesting articles to continue circulating years later, the 2014 story is still being read by thousands of readers — even though its key results have largely been debunked. It’s past time to set the record straight on this.
The premise of the study was that female-named storms are deadlier because of “implicit sexism” — that is, we make decisions about storms based on the gender of their names without even knowing it. The study produced data showing that the most damaging hurricanes with female names killed substantially more people than storms with male names.
“[Our] model suggests that changing a severe hurricane’s name from Charley … to Eloise … could nearly triple its death toll,” the study said.
But just a day after the study was published by the Proceedings of the National Academy of Sciences, critics began poking holes in it.
For example, Eric Holthaus, a meteorologist writing for Slate at the time, found that simply removing Hurricane Sandy from the storms in the study group flipped the results. “Singlehandedly, Hurricane Sandy switches the authors’ entire premise on its head,” he wrote. “Ignoring Sandy’s outlier nature, male-named hurricanes now cause more deaths than female ones.”
In subsequent months, several study rebuttals were submitted to and published by the same journal that had published it originally, from academics and others:
- Björn Christensen and Sören Christensen, statisticians from Germany, showed that the study’s conclusion was “based on biased presentation and invalid statistics.”
- Laura A. Bakkensen and William Larson, from the University of Arizona
and the U.S. Department of Commerce, respectively, concluded that the results were “of questionable robustness.”
- Steve Maley, a Gulf Coast resident, noted that after outlier storms in the study group were accounted for, “the statistical difference between the male and female subsets becomes negligible.”
- Daniel Malter found that the results were “not robust” in his analysis.
In addition to the formal rebuttals published by the journal, Gary Smith, a professor of economics at Pomona College, wrote a critique on his blog, in which he described “several compelling reasons” to be skeptical about the study results. He concluded: “The study’s assertion that female-named storms are deadlier than male-named storms is not robust, evidently because it relied on the questionable statistical analysis of a narrowly defined set of data, including data from years when all hurricanes had female names.” The blog critique was subsequently expanded in an article published in the journal Weather and Climate Extremes in June 2016.
Smith said in his article’s news release: “It is implausible that an imperiled public’s response to a potential storm of the century — with catastrophic warnings broadcast by news media that feed on sensationalized reporting — depends on whether the name Sandy is perceived to be a feminine or masculine.”
The publication of this study and the hype it generated (and continues to generate) show the potential pitfalls of coming to unqualified conclusions from a single study. It also reveals the importance of the peer review process, which encourages critical responses to new findings so that work that cannot withstand scrutiny does not endure.
I am updating the original story with a new headline and link to this post at the top of the article so that readers encountering it understand that the initial findings are generally considered flawed.
For an interesting perspective on the journalistic challenges in covering this story, read this illuminating essay: Why Female Hurricanes Made Getting the Story Right, Hard