But in recent years, the agency has come under scrutiny for how it communicates severe thunderstorm risk, with many meteorologists arguing the long-standing risk scale is as confusing as it is helpful. Some feel the system needs to be scrapped and overhauled, while others think changing things could be even more counterproductive. Now, emerging social science research suggests that there are bona fide issues — but the path toward remedying them remains unclear.
The topic of debate? Whether the words assigned to risk categories to describe severe thunderstorm potential — marginal, slight, enhanced, moderate or high — are adequately descriptive or inherently confusing. It’s a classification scheme that’s been around in some capacity since at least the 1970s, expanded to include the addition of the “marginal” and “enhanced” categories in 2014. As one moves up the scale, the severe weather risk understandably increases. High risks are reserved for the most dire tornadoes and damaging thunderstorm outbreaks.
It’s been shown time and time again, however, that the general public doesn’t understand the categories. Is a slight risk or a marginal risk more significant? And why is moderate risk not in the middle? What’s an enhanced risk? Meteorologists know the system like the back of their hand. But the public does not.
That’s according to Alex Forbes, an undergraduate at Mississippi State University. He worked on research for his senior capstone project aimed at probing the effectiveness of the SPC’s categories. His findings reveal the ongoing struggle.
Public confusion revealed
As part of his project, Forbes asked nearly 4,000 respondents to rank the presumed order of the SPC’s five categories based on their implied severities. On the whole, respondents got 4 out of the 5 categories wrong.
On average, participants felt that a “marginal risk” was worse than a “slight,” and that “moderate” sounded less threatening than “enhanced.” In fact, only one category — the “high risk” — was found to be readily understood by the majority those surveyed; otherwise, respondents correctly ordered all the other categories less than 40 percent of the time. Forbes was blown away by the results.
“Most of the people who took it at least see weather information on a regular basis,” said Forbes, who enlisted the help of more than 400 broadcast meteorologists in sharing the link to his survey on social media.
“I asked people to give me their Zip code; I had over 2,000 participants from Alabama, 1,000 from Arkansas, 419 in Tennessee, and more than 300 in Georgia.”
A widely recognized problem
Forbes’s results were surprisingly similar to those of Sean Ernst, who is tackling the problem as part of his master’s thesis at the University of Oklahoma. He also found that most people associate a “moderate” risk as a middle-of-the-line threat, and noted that most also felt that a marginal risk is more significant than a slight.
“These things came from military and government uses, and have since proliferated to the general public,” Ernst said. “They were never designed necessarily with the general public in mind.”
“The SPC outlook was never meant for public consumption,” he said. “It still isn’t to this day. The only reason it’s published is because [the SPC is] required to by federal law as a federal agency.”
A product identity crisis — who is the forecast for?
Before the advent of modern-day Internet, the Storm Prediction Center’s forecasts were primarily reserved for use by regional National Weather Service offices and emergency management partners. Only in recent years have their outlooks and discussion migrated to the mainstream. Nowadays, they’re also commonly broadcast by media outlets.
“Initially the products were geared towards internal government use,” Patrick Marsh, chief of science support at NOAA’s Storm Prediction Center, said in an interview. “They even had a logo on them that said ‘for internal government use.’”
Marsh explained that the outlooks are drawn heavily upon by emergency management and community officials for planning purposes. W. Craig Fugate, former administrator of the Federal Emergency Management Agency and previously the director of Florida’s Emergency Management Decision, played a role in strengthening this link after prescient forecasts by the SPC leading up to the deadly Feb. 22-23, 1998, Florida tornado outbreak.
“When he became administrator in 2008 with the Obama administration, he pushed FEMA to be proactive and really be engaged based on the forecast of the [meteorological] community,” Marsh said. “That really set the stage with the National Weather Service and SPC, and for being more proactive than reactive.”
Marsh explained that emergency managers over time had become used to the SPC’s way of doing things. It was a system that, for the most part, worked well.
“Emergency managers on the Plains and [in] the Southeast have a high propensity to deal with severe weather, and have become accustomed to SPC forecasts that we were putting out,” Marsh said. “They became familiar with the terminology of the slight, moderate, and high risks.”
Marsh said that, once emergency managers experienced enough slight, moderate, and high risk days, they were largely able to anticipate the coverage and magnitude of impacts heading into an event. The system worked as intended.
Eventually, the SPC found that one of their categories — slight risk — was too broad, encompassing lower-end and more widespread events. After reviewing feedback from emergency management, they broke it into three categories: marginal, slight, and enhanced. That’s when public confusion really ramped up.
“There is no real word that fits between ‘slight’ and ‘moderate’ that works,” Marsh said. “‘Enhanced’ showed a little bit more promise. But SPC knew there was going to be a problem there. The words aren’t perfect.”
Expansion into the public realm raises new challenges
That confusion about the three middle tiers may lead the public to conflate, under-prepare, or overestimate a risk — the latter of which can result in taking future events less seriously.
But it raises questions surrounding the role of the SPC’s outlooks, which were originally intended for use by primarily meteorologists and emergency managers. If the status quo works for emergency managers — but not the public — should changes even be made, considering where the product was aimed in the first place?
Some feel that the outlooks should be designed with the public in mind. Others suggest that the onus lies on broadcast meteorologists, whose job they argue it should be to translate SPC’s outlooks into something consumable by the public.
Who’s job is it to communicate a technical forecast?
“You can have a perfectly accurate forecast,” Ernst said, “but a forecast has no value unless a user can make an educated decision based on it.”
Making a forecast — and clearly communicating it — are two sides of the same coin. But who is responsible for “translating” those well-serialized outlooks into layman’s terms?
Susan Jasko, a senior research scientist at the Center for Advanced Public Safety at the University of Alabama, explained the challenges of producing something that fully encapsulates the needed precision and science, but is also accessible to those without extensive user background or scientific aptitude.
“Think of any technical profession where there’s an inside loop of language and understanding and training and experience,” Jasko said. “You take any piece of that and put it into a larger knowledge kind of area that isn’t so technically focused. … It’s going to take on a life of its own.”
Jasko said that, no matter how accurate a piece of information is, its effectiveness is limited if the consumer doesn’t have the expertise to interpret it.
“It’s going to be understood in the context it’s received, not in how it’s produced,” Jasko said. “Things get lifted and then they get misunderstood. The interpretation is always in the eyes of the receiver. ”
Jasko argues that the solution is not necessarily in overhauling the product for public use, but instead in finding ways to more effectively communicate the information it contains — whether that be internally at the SPC or through the media.
“I think it’s an internal, powerful tool. People like [emergency managers] rely on this kind of information … to help them make the kind of judgments they have, and influence other people’s decision-making,” Jasko said. “It’s an incredible chain. Whatever we do we, need to be exceedingly careful to not damage that.”
She says that, in many ways, the communication could best start with broadcasters.
Enlisting the help of broadcasters
Turning to broadcasters to lead on this matter has been a common theme. Ernst referred to broadcasters as the “conduit through which this information reaches the public.”
Forbes said he’s seen a lot of television stations experimenting on their own in trying to find best ways to communicate Storm Prediction Center outlooks, with varying degrees of success.
“To broadcasters, I would say drop the [category] names,” Forbes said. “Unequivocally, don’t use them.”
Forbes instead recommended on-air meteorologists convey categorical risk with the 1-to-5 scale that corresponds to the outlook tiers, comparing that ease of communicability to that of hurricanes.
“I’ve never had a person ask me if a Category 5 is worse than a Category 1,” he said. “It’s straightforward; the higher the number, the higher the threat.”
“The colors look pretty good,” said Ernst, whose research project included probing alternate methods of communicating risk. “People are actually doing pretty well with the colors. Usually the colors are making a bull's eye, but even alone people interpret the colors pretty well.”
Presenting the risk information effectively may vary from one television station to another says Eric Snitil, chief meteorologist at WROC in Rochester, N.Y. He says it boils down to understanding the needs of local viewers.
The broadcast meteorologist is the “best resource for understanding exactly what his/her audience is going to digest best,” Snitil wrote.
SPC’s near-term response
For its part, the Storm Prediction Center has been making a concerted effort in the nearer term to cater to their growing, public-facing audience — even if public consumption had never been the agency’s original intention.
“It’s something that’s been on our list, to develop a more public facing and public friendly,” said Bill Bunting, the SPC’s chief of forecast operations. “You see that more incrementally with the publicly focused summary.”
In the past several years, the SPC has begin offering parallel products with simplified language, including public severe weather outlooks, during higher-end events. The graphics have also been amended to include a colorized 1-through-5 scale.
“You might notice our forecasters, or at least some of them, [are] starting to say more like ‘a moderate level 4 risk,’ ” Bunting said. “We’re seeing signs the anchoring [in numbers] is taking place as well.”
But the SPC knows it’s a temporary fix to a permanent problem, aware of the growing need to make their information accessible to the public.
“We’re all about trying to make the best possible product and communicate the threat,” Marsh said.
Down the road
Eventually, the SPC hopes to adopt a more permanent solution — one that will be arrived at after considerable partnership with social scientists. Right now, it’s not ruling anything out — but it knows it has one chance to get it right.
“We only want to do [a] big change once,” Marsh said. “We’re actively engaged in trying to solve this problem."
He said that the Storm Prediction Center “has been hesitant to engage in the idea of changing words” like marginal and enhanced but is exploring all options at this point.
Jasko says that any large changes would likely require an education campaign to get the public up to speed.
Whatever the solution, meteorologists and the public alike unanimously recognize the importance of severe weather forecasting, and the strides the science has made in recent years. Bunting hopes that momentum will continue.
“Our system has been pretty successful,” he said. “We want to build on that past success and make it even better.”