Eventually, the GFS model forecast shifted to the correct solution, but the European model had the correct forecast about 24 hours before the GFS, emerging victorious.
Some in the weather community, as well as interested enthusiasts, are making much out of the European model’s win. Its triumph in this significant forecast is not meaningless, but it’s also neither a huge deal nor unsurprising. And it does not detract from other important advances being made by the National Weather Service in hurricane forecasting.
In reviewing what happened with these model forecasts, I’ll present five issues:
1. It’s not irrelevant that the European model beat the U.S. model in a consequential forecast.
When big storms threaten the East Coast, forecast accuracy at long lead times has important implications. Based on the National Hurricane Center’s official forecast on Wednesday, which directed Joaquin straight into the Mid-Atlantic over the weekend – largely based on the GFS and other models – weekend events in the D.C. area were altered (for example, the Taste of Bethesda was cancelled and Maryland vs. Michigan football game kickoff time was moved).
Some attempted to downplay the European model’s success. The National Hurricane Center tweeted the GFS model’s accuracy in forecasting tropical systems has matched the European’s, on average, during the past three years.
But others in the weather community correctly pointed out that what really matters is how well models perform when the stakes are high.
Joaquin marked the second straight high profile win for the European model for a hurricane threatening the eastern United States. In 2012, the European model gained fame for correctly predicting Sandy would strike the Northeast coast at long lead times while the U.S. model hooked the storm out to sea.
2. It shouldn’t surprise anyone that the European model performed better: It is better.
Statistics show the European model is the most accurate model – on average – for forecasting global weather systems. The European model has a more powerful computer, better model physics and a superior system for bringing in data. All of this profoundly affects how well the model simulates what’s happening in the atmosphere.
The GFS model has shown a tendency to experience performance “drop-outs” in complicated situations, according to Ryan Maue, a meteorologist for WeatherBell Analytics.
In forecasting Joaquin, the National Hurricane Center forecasters considered the European model’s well-known accuracy advantage. But it could only give the European model so much weight since it was an outlier among a large number of models, including the GFS model, which predicted Joaquin would turn to the coast.
3. The European model is not always the best model. The significance of one model is often overblown.
I often hear the question: If the European model is the gold standard, why pay attention to the other models? I stress: The best model, on average, is not the best model in each place and each situation.
“[A]s we’ve seen in the past, no single model is correct 100 percent of the time,” said William Lapenta, director of the National Weather Service’s National Centers for Environmental Prediction in a statement. “That’s why track forecasts by the National Hurricane Center are based on an ensemble of numerical guidance information and forecaster experience.”
There have been notable recent cases in which the GFS model provided a better forecast. For example, in 2012, the GFS model offered a more accurate track forecast for Hurricane Isaac (in short-term forecasts, the European model was better at longer ranges) that tracked through the Gulf of Mexico.
The GFS model also provided a better forecast in January when New York City was supposed to be buried under more than two feet of snow, according to the European model. In reality, only eight inches fell, closer to the GFS model forecast.
“I recall having dinner with the director of ECMWF Alan Thorpe in Reading, U.K., a few years ago, and he chuckled at the vigor of how citizens here debate the models,” wrote Marshall Shepherd, professor of atmospheric sciences at the University of Georgia, in a recent column at Forbes.com. “He alluded to the fact that GFS beats the European model often.”
4. The United States is working feverishly to improve the GFS model but might never surpass the European model.
As we’ve reported previously, the National Weather Service is in the early stages of increasing the capacity of the supercomputer powering the GFS model by a factor of 10.
It is attempting to narrow the performance gap with the European model, but it’s not clear how successful it will be. The ECMWF is simultaneously pumping resources into the European model to maintain its edge.
Ultimately, even if the ECMWF model stays ahead in the weather computer arms race, if both models become more powerful and skillful, it’s a win-win for weather forecasts everywhere.
5. The United States might already have the best model for forecasting hurricane intensity.
While the ECMWF model devotes most of its resources into a single model, the National Weather Service supports a conglomerate of models, including a hurricane-specific model known as the Hurricane Weather Research and Forecast system or HWRF. While the European model featured the best forecast for the track of Joaquin, the HWRF forecast for Joaquin’s intensity was far superior to any other model (though its track forecast erred similarly to the GFS).
“HWRF intensity forecasts for Joaquin have been the best … and have shown pretty impressive rapid intensification forecasts as the storm was near Bahamas,” said Chris Vaccaro, a spokesperson for the National Weather Service.
The HWRF model is run off the same supercomputer as the GFS, which has been upgraded since last winter.
Although the HWRF model is “now considered the model of choice” for intensity guidance, according to the NWS, funding for the program supporting the model was significantly reduced last fall.
###
* Note: While I say the European model consistently and correctly tracked Joaquin out to sea, it did bring the storm ashore in one errant simulation.