Nate Silver mistakes wild weather for unpredictable weather

( <a href="http://fivethirtyeight.com/features/which-city-has-the-most-unpredictable-weather/">Nate Silver and Reuben Fischer-Baum, fivethirtyeight.com</a> )

Rapid City, South Dakota has the most unpredictable weather in the nation, claims Nate Silver at fivethirtyeight.com. But what his analysis actually uncovered is that Rapid City has the nation’s most volatile or erratic weather, not necessarily its most unpredictable.

Silver attempts to evaluate unpredictability through a painstaking review of historical temperature, precipitation and severe weather statistics for 120 U.S. cities across the 50 states. For example, he determines how “predictable” temperatures are by examining how much they differ on a day-to-day basis from historical averages. Cities, like Phoenix, in which day-to-day temperatures don’t deviate much from historical averages (i.e. when the average high is 90, the actual high is typically close to 90) are considered predictable, but cities, like Denver, in which averages are seldom a reflection of reality (i.e. when the average high is 50, the actual high could almost as easily be 30 or 70) are considered unpredictable.

Such an analysis of weather variability will give you a sense of the size of a city’s day-to-day weather swings or how erratic the weather is, but it’s not a crystal clear indicator of “unpredictability.”

If Silver wanted to analyze how “unpredictable” the weather was, he would have needed to evaluate which cities have the worst forecasting. For example, he could’ve examined forecasts from the different agencies and companies (like the National Weather Service, AccuWeather, and The Weather Channel) averaged them, and reported on which cities show the biggest errors. That would be an interesting analysis, which Silver or some other enterprising weather analyst could take on.

It’s totally possible that many of the cities which have the most volatile weather are the among the least predictable. Intuitively, for example, you would expect a city where the weather doesn’t change much, like San Diego or San Francisco, would be easy to forecast whereas cities where the temperature can drop 50 degrees in a few hours, like Denver, would be more challenging.

But volatility isn’t always the best indicator of unpredictability. Big fronts which cause big swings in the weather are actually pretty easy to forecast whereas more subtle variations can be much more difficult to detect, while the consequences are enormous. For example, in Washington, D.C., we frequently see cases where a couple degrees represents the difference between 32 and a paralyzing snowstorm, and 34 and rain.

As long as you recognize Silver is evaluating how wild the weather is, not unpredictable, his analysis shows some interesting but not surprising things (see image at top of blog post). The Northern Rockies and Northern Plains, which frequently experience big weather swings and/or severe weather, are the nation’s hot spots for weather volatility. The West Coast, displaced from the battle ground between hot, unstable air in from the Gulf of Mexico and frigid Arctic air from interior Canada, has generally less erratic weather.

To close, Silver did some nice analysis and developed some eye-catching visuals, but would’ve been on sounder ground had he framed his presentation around weather volatility rather than predictability.

Update, 12:40 p.m.: To my point that Silver should’ve analyzed forecasts for this analysis, Silver has a footnote in his blog post stating he considered doing that but that data wasn’t available. Footnote text: “One could also compare daily weather conditions against short-term weather forecasts issued by organizations like Accuweather. Our focus here is on climatology because weather forecast data is not well preserved in the public record. However, Reuben Fischer-Baum and Dennis Mersereau’s analysis of weather forecast data for 2013, which relied on data compiled by the website ForecastAdvisor, revealed that parts of the country where weather is poorly predicted from climatology also have inaccurate short-term forecasts.”