The premise of the study was that female-named storms are deadlier because of “implicit sexism” — that is, we make decisions about storms based on the gender of their names without even knowing it. The study produced data showing that the most damaging hurricanes with female names killed substantially more people than storms with male names.
“[Our] model suggests that changing a severe hurricane’s name from Charley … to Eloise … could nearly triple its death toll,” the study said.
But just a day after the study was published by the Proceedings of the National Academy of Sciences, critics began poking holes in it.
For example, Eric Holthaus, a meteorologist writing for Slate at the time, found that simply removing Hurricane Sandy from the storms in the study group flipped the results. “Singlehandedly, Hurricane Sandy switches the authors’ entire premise on its head,” he wrote. “Ignoring Sandy’s outlier nature, male-named hurricanes now cause more deaths than female ones.”
In subsequent months, several study rebuttals were submitted to and published by the same journal that had published it originally, from academics and others:
- Björn Christensen and Sören Christensen, statisticians from Germany, showed that the study’s conclusion was “based on biased presentation and invalid statistics.”
- Laura A. Bakkensen and William Larson, from the University of Arizona
and the U.S. Department of Commerce, respectively, concluded that the results were “of questionable robustness.”
- Steve Maley, a Gulf Coast resident, noted that after outlier storms in the study group were accounted for, “the statistical difference between the male and female subsets becomes negligible.”
- Daniel Malter found that the results were “not robust” in his analysis.
In addition to the formal rebuttals published by the journal, Gary Smith, a professor of economics at Pomona College, wrote a critique on his blog, in which he described “several compelling reasons” to be skeptical about the study results. He concluded: “The study’s assertion that female-named storms are deadlier than male-named storms is not robust, evidently because it relied on the questionable statistical analysis of a narrowly defined set of data, including data from years when all hurricanes had female names.” The blog critique was subsequently expanded in an article published in the journal Weather and Climate Extremes in June 2016.
Smith said in his article’s news release: “It is implausible that an imperiled public’s response to a potential storm of the century — with catastrophic warnings broadcast by news media that feed on sensationalized reporting — depends on whether the name Sandy is perceived to be a feminine or masculine.”
The publication of this study and the hype it generated (and continues to generate) show the potential pitfalls of coming to unqualified conclusions from a single study. It also reveals the importance of the peer review process, which encourages critical responses to new findings so that work that cannot withstand scrutiny does not endure.
I am updating the original story with a new headline and link to this post at the top of the article so that readers encountering it understand that the initial findings are generally considered flawed.
For an interesting perspective on the journalistic challenges in covering this story, read this illuminating essay: Why Female Hurricanes Made Getting the Story Right, Hard