Review of Dark Sky’s new forecast tool, Forecast.io

A new online weather forecasting application, with an elegant look and silky smooth performance, is generating quite the buzz among web developers. But is the information provided by this application – called Forecast.io – credible and useful from a meteorologist’s perspective?

Early returns are mixed for this app, developed by the creators of DarkSky (a tool that simulates when it will rain or snow).  The biggest flaw of Forecast.io, in my view, is that it provides no percentages or forecast confidence information.

But to start with its strengths: the app looks great and functions seamlessly. Tech Crunch calls it “The Web App That Has Web App Developers Drooling”

forecast-io

The front page of the app automatically loads your location and provides nice snapshot information while allowing you to drill down for more detail.  Did I say the app was beautiful?

But its forecast will either be right or dead wrong, with no in-between. That’s a serious drawback if you’re trying to  understand the range of forecast possibilities and make a decision.  The app is keeping information from you.

Let’s take the forecast for this coming Easter Sunday in Washington, D.C..  It predicts it will be partly cloudy until 3 p.m. and then rain will start.  What’s the chance of rain? Does the rain start at exactly at 3 p.m. or is there actually a much broader window when rain might begin? (It’s the latter) What’s the forecast confidence? This app fails to offer this information.

(Forecast.io)

(Forecast.io)

It’s unfortunate that the likelihood and confidence information is omitted since it’s available to the developers.  On the back end, the data feed driving the forecast comes from a grouping of forecast models. This is a legitimate approach, but the tool would be immensely more valuable if it somehow communicated the differences (or spread) in the models to better convey uncertainty.  Given the developers’ obvious skill in presenting complex information, they could take this next step and provide a state of the art weather information tool.

(Forecast.io)

On its raw data page, the Forecast.io app shows the various model inputs for its forecasts (Forecast.io)

The lack of uncertainty information in the Forecast.io app is just one shortcoming.

Ryan Maue, a meteorologist at WeatherBell.com, says processing model data requires challenging statistical adjustments to account for local variations in weather and he’s unconvinced Forecast.io has broken new ground. (2:30 p.m. update:  Forecast.io describes how they address local weather variations in a clarification we’ve added at the bottom of this post).

“[Doing this statistical work] is hard in [the] U.S. and even harder globally,” says Maue. “Until a detailed verification is presented for this new system, it has not distinguished itself as at all novel — just another API [application programming interface] undercutting the big guys.”

But do these technical issues and gaps in information matter from the standpoint of an end-user? Can a slick app like this take the place of human forecasters?

“I and other meteorologists bristle at this [possibility] because we know the limitations of those models and the accurate predictability of the atmosphere at long time scales, and we believe we need humans in the process to improve upon that model data for every forecast point for every day,” says Nate Johnson, a broadcast meteorologist for WRAL in Raleigh, North Carolina.

The reality says Kevin Selle, chief meteorologist for Texas Cable News, is that human forecasters cannot provide details for every location. Moreover, this kind of simple app meets the needs of many users, he says.

“Weather is a competitive space and I’ve long said there is no competitive advantage to automated data, but the truth is, on most days, Partly Cloudy, 72, is enough for a lot of people,” Selle says. “[Local meteorologists] will lose a few users to Forecast.io, and a few more to The Weather Channel (now doing very short term text forecasts on Android) and a few more to the next thing, and the audience will get ever smaller.”

A post from Capital Weather Gang (CWG) reader Joshua Davis on Facebook reinforces Selle’s point:

I’m still going to check out CWG for in-depth forecasts, but this is a really cool and beautiful web app for a quick weather forecast

A few more reactions from meteorologists on Twitter

 

 

Clarification, 2:30 p.m.:  Adam Grossman of forecast.io emailed me to further explain how its predictions are calculated: “We don’t do a simple average of the models. We first correct for geographically related bias in each model individually (e.g., GFS temperature seems to be annoyingly low in some places, higher in others). And then we monitor the accuracy of each model, geographically, to calculate a standard error which we use to weight each model on-the-fly whenever a request comes in for a forecast. Just because our app is “pretty” doesn’t mean we don’t know what we’re doing. ;-)

Correction, 3:20 p.m.: An earlier version of this post said Ryan Maue had stated Forecast.io “simply averaged models”; that was an inaccurate paraphrase of his comments.

Also on Capital Weather Gang

D.C. area forecast: Cold and breezy, then warmer in time for weekend