A leading forecasting model used by the White House to chart the coronavirus pandemic predicted Monday that the United States may need fewer hospital beds, ventilators and other equipment than previously projected and that some states may reach their peak of covid-19 deaths sooner than expected.
By Wednesday morning, the model had been revised even more dramatically downward. It now predicts a total of 60,400 U.S. deaths by August and forecasts the peak of those deaths arriving in just four days on April 12, instead of April 16 as previously projected. Experts, however, have noted that this particular model’s numbers and projections — while used widely — have been consistently lower than those of other models.
Experts and state leaders, however, continued to steel themselves for grim weeks ahead, noting that the revised model created by the Institute for Health Metrics and Evaluation (IHME) at the University of Washington conflicts with many other models showing higher equipment shortages, deaths and projected peaks.
Some state leaders have also grown increasingly concerned about how the federal government is using IHME’s lower estimates to deny states’ increasingly desperate requests for equipment and help in preparations. The stark differences between the IHME model and dozens of others being created by states exposes the glaring lack of national models provided publicly by the White House or agencies such as the Centers for Disease Control and Prevention for local leaders to use in planning or preparation.
“It’s unclear exactly what the White House is doing on this front,” said Dylan George, who helped the Obama White House develop models to guide its Ebola response in 2014. “As a result, you have every state trying to create their own models to anticipate their needs. And you have one model like IHME being adopted as the national guide.”
The danger of relying so heavily on one model is that model could be wrong or overly optimistic.
“When you plan, you want to plan for the worst-case, not for the average or best-case,” said Natalie Dean, an assistant professor of biostatistics at University of Florida. “Because the risk is not proportional.”
This is how starkly models can differ.
Local leaders in the District said on Friday that their model estimates the outbreak in the nation’s capital will peak June 28. The IHME model predicts the peak is coming in just days, on April 16. The District’s model predicts hospitals will need 1,453 ventilators at the peak. IHME predicts a need for only 107. The District is using the IHME model as a best-case scenario and the more dire model to prepare for a likely surge.
“While we hope that our experience will follow a curve closer to the IHME model, we cannot use a single model for our preparation and risk being underprepared. We continue to refine our models and assumptions and are tailoring them to the DC population and context,” spokeswoman Alison Reeves said in an email.
In states more populous than the District, that vast gap in planning and modeling could mean a life-or-death difference for tens of thousands of people.
LaQuandra S. Nesbitt, director of the District’s Department of Health, explained how the city’s leaders chose their model. It’s called CHIME and was created by researchers at the University of Pennsylvania.
“We felt that a model that determined the District would have essentially no medical surge needs was not indicative of what we anticipated would be our reality in the District and thought that a model that did not overestimate the impact of social distancing in the United States” was the right one, Nesbitt said.
In the two weeks since IHME’s model was originally released — the researchers announced revisions Monday — it has been criticized by some experts as overly optimistic. But even critics are quick to note that in the absence of any tool offered by the federal government and with no other model offering nationwide state-by-state estimates, IHME could be a lifesaver.
To coordinate their response, some states with few modeling resources or home-state experts have used the IHME forecast that projects peak deaths and the resources needed. The White House relied on it in part to generate its estimate last week that the epidemic would kill 100,000 to 240,000 people nationwide.
Most epidemiological models look at different populations that interact in an outbreak — people susceptible to infection, those who are infectious and those already infected who go on to die or recover.
Funded by the Bill & Melinda Gates Foundation, the IHME model embraces an entirely different statistical approach, taking the trending curve of deaths from China, and “fitting” that curve to emerging death data from U.S. cities and counties to predict what might come next.
For that reason, many experts saw IHME as overly optimistic when it was launched March 26. Few U.S. states or cities are taking action as drastic as what was adopted in Wuhan, China — the birthplace of the coronavirus pandemic — or even Northern Italy in locking down residents.
Another big difference between IHME and other models is a fundamental assumption about how effective social distancing can be. The creator of IHME’s model, Christopher Murray, said many state models assume that social distancing will only slow or reduce transmission to some degree. The IHME model, drawing from the example of Wuhan, assumes policies such as social distancing and stay-at-home orders, can effectively reduce transmission to the point where an epidemic — at least in its first wave — is actually brought under control by authorities.
At the White House Coronavirus Task Force briefing Monday, health officials said they thought it was possible to have fewer deaths than have been projected by models, because of the extreme social distancing efforts being undertaken by Americans.
“Models are good, they help us to make projections. But as you get data in, you modify your model,” said Anthony S. Fauci, director of the National Institute of Allergy and Infectious Diseases. “I don’t accept everyday we’re going to have to have 100,000 to 200,000 deaths. I think we can really bring that down."
More models, better prediction
When it comes to predictions, statisticians have a favorite maxim: “All models are wrong, but some are useful.”
By drawing on the multiple models, experts are often able to better triangulate their predictions and assumptions. This is why weather experts often draw on several models rather than one in forecasting storms, using an “ensemble” or “suite” of models. Such ensembles are also what generate the cone of uncertainty for hurricanes.
In Illinois, state leaders are using four models — a version of the CHIME model and models from the University of Chicago, Northwestern University and the University of Illinois at Urbana-Champaign. New York state officials have said they are drawing on at least four different models, including IHME’s.
In North Carolina, state leaders are using a “weather forecasting” approach that combines several models — and instead of focusing on specific dates or numbers of beds or ventilators, they have decided to predict the likelihood that the medical system is overwhelmed.
“What we’ve been focusing on is less about ‘what is the exact timing of the peak?’ and ‘what is the exact height of the peak?’ and more about how likely is it that the demand for health care is going to outstrip the supply,” said Kimberly Powers, an epidemiologist at the University of North Carolina at Chapel Hill. “Are we going to need more than we have?”
Their composite models predicts a peak in mid- to late-May. As long as social distancing continues, they predict only a 1 in 4 chance of exceeding the capacity of acute care hospital beds. But if those orders were lifted after April, the chances of overwhelming hospital capacity doubled.
One concern from some experts is that the IHME model is being used too much like a crystal ball with undue weight given to its predicted needs for ventilators and hospital beds and staffing.
Leaders in one state said Trump administration officials have used IHME’s numbers to push back and in some cases deny their requests for equipment and help. Officials in that state cited emails and documents in which federal officials highlight IHME projections as evidence the state needs thousands fewer ventilators and beds than the state’s models project. The state officials spoke on the condition of anonymity because they said they fear retaliation by the Trump administration that could result in even fewer ventilators and less federal assistance.
“If the federal government is really making these kinds of life-or-death decisions on a single model and on only on the lower end of that model, that’s scary,” said one state official.
Murray said he is well aware of the criticism of his model
But in the absence of any other state-by-state planning tool, he noted in a Monday briefing with reporters, his model is providing a much-needed public service — a point even critics of the IHME model are often quick to point out.
Murray and his team have worked around-the-clock since they first released their model to feed newly emerging data and sharpen it its projections.
On Monday, they announced their biggest revisions to date — driven by a large amount of new domestic and international data.
While their original model relied only on Wuhan’s curve, the updated model now incorporates curves from seven regions from Italy and Spain where epidemics have also peaked.
The newer version also found that deaths in some states — such as Florida, Virginia, Louisiana and West Virginia — could peak earlier than previous projections. But the deaths nationally were still projected to peak April 16. The newer model suggests the number of acute care hospital beds needed at the peak could be cut almost in half and the number of ICU beds needed at the peak of the surge could drop from 40,000 to 29,000. The model also suggested the total number of deaths would be lower, with an estimated 82,000 deaths from the first wave of infection, although the number could range from 49,000 to 136,000.
Murray agreed with the critique of others that multiple models should be used.
“I could not agree more,” Murray said. “What we’ve learned from 30 years of weather forecasting, even Netflix predictions for movies … you make better predictions when using multiple models.”