1) Disbelief. This study isn’t for real. It’s from The Onion, right?
“If you had not told me that it was a real paper I would have assumed it was satire,” Robert J. Meyer, co-director of the Wharton Risk Management and Decision Processes Center at the University of Pennsylvania, told Mashable.
Yet the study is legitimate, published in the prestigious Proceedings of the National Academy of Sciences (PNAS). Over the years, some commentators have questioned the rigor of the journal’s review process. Science 2.0 wonders if the study’s residence in PNAS is “going to lead to people wondering if it’s another one of their studies where a friend of the senior author hand-walked this past peer review, like PNAS has controversially done in the past”. For its part, PNAS is transparent about its review procedures and a debate over its merits would be interesting, but a diversion here.
2) Shock. How can humans be so stupid to let the gender of storm names influence how they respond to storms?
If we really think Hurricane Alexander is going to be worse than Alexandra based on name alone, it’s tempting to conclude our collective elevators aren’t going all the way to the top.
“Why would any intelligent person think that a hurricane with a female name could not be as intense as a hurricane with a male name?,” Max Mayfield, past director of the National Hurricane Center, wrote to me in an email. “I can’t help anyone who thinks like that.”
But one of the study’s pivotal findings is that sexism lies beneath the surface – it’s ingrained in our thought processes.
We just tend to associate women as less violent and dangerous than men.
“Such gender biases are pervasive and implicit,” said Madhu Viswanathan, professor of marketing at Illinois and co-author of the study. “We found that people were affected by the gender of hurricane names regardless of whether they explicitly endorsed the idea that women and men have different traits. This appears to be a widespread phenomenon.”
3) Skepticism. The study is methodologically flawed.
In assessing technical concerns about this study, it’s best to think about its methodology in two parts: 1) The analysis of historical death rates according to the gender of the hurricane name and 2) The psychological experiments testing how hurricane gender influences how people judge storm intensity and their responses (i.e. will they evacuate?).
Generally, criticisms about the first part of the study have been more serious than for the second part of the study.
Criticism published in blogs and in press accounts online have (see Mashable, USA Today, New York Magazine, National Geographic, and Slate) stated the dataset used in the first part of the study was too small, failed to control for certain influences, and/or lacked statistical robustness.
* Eric Holthaus, at Slate, says removing Hurricane Sandy from the study’s dataset would reverse its findings: “Singlehandedly, Hurricane Sandy switches the authors’ entire premise on its head. Ignoring Sandy’s outlier nature, male-named hurricanes now cause more deaths than female ones”
* The improvement of storm warnings over time may have impacted results. From 1950 to 1978, at a time when forecasts weren’t as good, hurricanes only were assigned female names. Male names were introduced in 1979 after which forecasts dramatically improved.
“It could be that more people die in female-named hurricanes, simply because more people died in hurricanes on average before they started getting male names,” said Jeff Lazo at the National Center for Atmospheric Research.
In response to Lazo’s remarks, published at National Geographic, the authors posted an online comment stating how long ago the storm occurred did not predict its death toll in their analysis.
* William Larson, an economist at the Bureau of Economic Analysis, on his own time analyzed the study’s data and on his blog concluded: “I would suggest there is no direct empirical evidence that the gender name of a hurricane affects the number of deaths, contrary to the author’s experiment-based results.”
Before fully accepting these criticisms, it’s important to recognize they themselves have not been subject to review. While it may be tempting to discredit the study based on the points Holthaus, Lazo and Larson have raised, due process in science requires iteration and give and take through peer review channels, not press accounts and the blogosphere.
As our piece discussed yesterday, researchers seem to more or less accept the results of the psychological experiments in the second part of the study – that the gender of the storm name plays a role in our perception of storms. The question is how large a role compared to all of the other factors that influence people’s decisionmaking during hurricanes that were not tested in the study.
“The authors have identified a key factor in the communication process that is worthy of more consideration,” said Gina Eosco of Cornell University’s risk communication group. “This is not something to dismiss, but I would strongly encourage more research that places the study of communication into the larger social process of making hurricane evacuation decisions.”
Overall, I think this study has brought about a welcome discussion about the influence of names and other non-weather factor in how people react to hurricanes. Regardless of the merits of the study, bringing to attention to the psychological aspects of our responses to hurricanes is critical. As I’ve written repeatedly in the past, a perfect forecast is worthless if people do not respond effectively to a looming weather hazard.
(An interesting aside: in reading press accounts and critiques of this study, I’ve anecdotally found criticism from men about this study much harsher than from women. Gender bias on a study about gender bias?)
Update, 9 p.m. Tuesday: The authors requested I post this comment regarding the Larson analysis described above which found “no evidence” the gender name of a hurricane affects the number of deaths:
We noted a blog post by an economist, who improperly modeled the data using basic ordinary least squares (OLS) regression. However, the data in our article were modeled using sophisticated count models. Appropriate adjustments were made to standard errors for extra-dispersion in the data, and goodness-of-fit tests were comparatively applied to the models. What this told us is that there appears to be a statistically meaningful relationship between hurricane damage and femininity of name (for severe storms). Of course any model, significant or not, allows for the possibility of being mistaken. It is a probabilistic relationship. However, it is critical to apply the correct modeling technique when modeling count data such as fatalities. OLS regression is not an appropriate method. This was explained in detail in our paper.
Further, the authors posted responses to a number of other criticisms found in online coverage of their study: Responses to media misunderstandings of study’s statistical approach.