washingtonpost.com
In health debate, those numbers are just numbers
The CBO's price tags are educated guesses, but guesses nonetheless

By Lori Montgomery
Monday, October 19, 2009

Phil Ellis may be the most powerful guy you've never heard of in the health-care debate. A senior analyst with the Congressional Budget Office, Ellis is the man who has to decide what it would cost to rebuild the health insurance system. He has essentially condemned two legislative proposals by slapping them with trillion-dollar price tags. A third plan rocketed to prominence after he said it would cost much less.

In the coming weeks, Ellis's judgments will shape the fate of President Obama's reform effort. But Ellis, an amiable father of three who hasn't had a day off in about six months, is the first to admit that his painstaking numbers are almost certainly wrong.

"We're always putting out these estimates: This is going to cost $1.042 trillion exactly," he said. "But you sort of want to add, you know, 'Your mileage may vary.' "

As Democrats embark on a plan to reorder one-sixth of the U.S. economy, the CBO is the umpire, charged by Congress with assessing the effect on the federal budget and the potentially profound impact on American lives. The Senate majority leader has vowed to hold no vote on a health plan until the CBO passes judgment. But the agency, while almost universally praised for honest and impartial analyses, does not have a crystal ball.

"Everyone in the process -- especially the CBO -- knows that it is very, very difficult to make these estimates and that they're no more than very educated guesses," said Alice Rivlin, who served as the CBO's founding director in 1975. "But if you didn't have this process, we know that the consequence is that everyone would want to spend money and not pay for it."

Inside the estimates

Much of what the CBO does is akin to trying to forecast your grocery bill in 10 years. First, it establishes a baseline: Your history of spending $200 a week at Safeway projected into the future with adjustments for inflation and expected demographic trends (i.e., more children, larger pets). Then it factors in proposed policy changes: Say you want to eat only organic and enroll your husband in Jenny Craig. Costs for meat, produce and dairy would go up, but spending on toothpaste and Saran Wrap would be unaffected. Meanwhile, the extra $70 a week for diet food would be partially offset by lower spending on Cheetos and frozen pizza.

The CBO has plenty of data to help make such calculations, including projections for inflation and the price of organics. But it would have to make some judgment calls: Is it reasonable to assume that Krispy Kremes are off the table? Or is it safer to budget for a dozen doughnuts once a quarter? And even the most careful estimate can be blindsided: What if the baby you projected to arrive in 2012 turns out to be twins in 2010?

Sometimes the projections are straightforward. If lawmakers want to impose a $6 billion tax on health insurance companies, the CBO -- and its revenue-estimating partner, the Joint Committee on Taxation -- assumes that it will generate $6 billion a year. Other provisions are more complex, and that is Phil Ellis's world.

Ellis heads the CBO's health insurance modeling unit, which examines plans to expand coverage to people who don't have it. The team's primary job is figuring what it would cost the government. But to do that, Ellis and his colleagues have to predict how people and businesses would react to a bewildering array of new economic incentives.

To help with this task, the CBO has constructed a computer model built on a government survey of 70,000 people, who were asked about family structure, health status and insurance options. The data are layered with scholarly research that suggests how those people might respond to various policies. The CBO also has assigned each worker to a "synthetic firm," whose virtual executives also have decisions to make, such as whether to offer insurance to their workers or pay a penalty to the government.

Ellis compares the model to SimCity, a computer game that reacts as players construct a virtual metropolis. Ideas go in (What happens if there's a $750 fine for not having insurance?), and a forecast comes out (Some people would choose to pay it, generating about $1 billion a year for the government.). Each legislative package contains dozens of such provisions, and they interact in infinitely variable ways.

Based on experience

Most of the model's assumptions are based on data from actual experience. The Massachusetts experiment with mandated coverage has been particularly helpful, Ellis said. But where there is no real-world precedent, Ellis and his colleagues just have to noodle it out. "In some cases, we're just going on economic logic and what's in people's self-interest," Ellis said, putting the range of uncertainty at as much as 20 percent.

Running policy through the insurance model is just the beginning. Results are checked with the CBO's Medicaid team and congressional tax analysts, who have their own computer models. Each 10-year cost estimate takes days to produce. With requests from lawmakers pouring in, Ellis and his team have been working practically around the clock to score three dozen coverage packages since April.

The latest numbers were delivered to House leaders last week in a weary late-night e-mail: "Talk to you tomorrow morning -- or is that today," said the missive, which, along with the new estimates, was obtained by The Washington Post (though not from the CBO, the agency wants you to know).

Despite the inherent uncertainty, the CBO takes its numbers very seriously. "With such consequential decisions on the table, we feel the weight of responsibility to provide the best information that we possibly can," said Doug Elmendorf, the agency's director.

Could it do better? Lately, the CBO has come under fire from liberal bloggers and champions of supposedly money-saving ideas, such as disease-prevention programs. Harvard economist David Cutler, a member of the CBO's health advisory panel, has complained that the agency is too cautious in estimating potential savings from changing the way doctors are organized and paid.

That charge is supported by research from Jon Gabel, a senior fellow at the National Opinion Research Center, who found that the CBO "substantially underestimated" savings from major health-reform legislation in the past three decades. The example most often cited by CBO critics is the 2003 creation of a Medicare prescription drug benefit, one of the first bills Ellis scored for the CBO. The agency predicted that spending would top $200 billion in the first three years. But, by the CBO's own estimates, Gabel said, actual spending turned out to be 40 percent less because the program attracted fewer people who bought more generics at lower prices than the CBO projected.

The CBO's defenders note, however, that Ellis called it closer than anybody else. At the time, in fact, the CBO was accused of low-balling the drug plan -- particularly after it was revealed that Bush administration actuaries had attached a much higher price to the president's proposal.

MIT economist Jonathan Gruber, who has advised Obama and the CBO on health care, said the agency should be more transparent about its assumptions so independent analysts can get inside the numbers that "literally are going to make or break this debate." Still, Gruber praised the CBO and Ellis -- one of his best PhD students ever, Gruber said -- for making the best of "an unbelievably hard job."

"Is there something obviously wrong with what they're doing? I don't actually think so," Gruber said. "There are 500 things in that model. They're doing as well as they can on 493 of them, and I might quibble on seven."

View all comments that have been posted about this article.

© 2009 The Washington Post Company