“We believe we’re a better country than a country where we allow, every day, 14,000 Americans to lose their health coverage; or where every year, tens of thousands of Americans died because they didn’t have health care; or where out-of-pocket costs drove millions of citizens into poverty in the wealthiest nation on Earth.”

–President Obama, remarks on the Affordable Care Act, Dec. 3, 2013

This column has been updated

This was quite a series of statistics reeled off by the president as he defended the law popularly known as Obamacare.

The first one is an old favorite of the president that had been deemed largely correct by PolitiFact in 2009, so we will leave that aside and concentrate on the other factoids—which, on the face, appear rather astonishing.  It turns out there are layers of complexity behind each of these numbers.

The Facts

Let’s first concentrate on the claim that millions of citizens are driven into “poverty” by out-of-pocket costs.

The White House said this fact came from the Census Bureau’s Supplemental Poverty Measure, which is a long-term effort to update the current official poverty statistic. Critics have said the current measure is out of date because it does not reflect either the effect of government policies, such as food stamps, that alleviate poverty or the impact of expenses, such as medical costs or transportation costs to work, that reduce income.

So, this is a different way to measure poverty, and it results in about 6 percent more people deemed as being in poverty than the official measure. But what the White House has done is pluck one figure out of the survey—out-of-pocket medical costs—to say all those people (10.6 million) have been thrown into poverty. But a) it is not the official poverty measure and b) it is just one element of this alternative measure.  Here’s how it breaks down:

You can see what the White House did. Officials simply focused on the 10.6 million figure.

Update: A careful reader convinced us that we had misinterpreted what these figures show. So we are striking out the incorrect material. Essentially, these are not overlapping elements, and so The Fact Checker earns some Pinocchios for fuzzy-headed thinking. But under Obama’s accounting, you’d also have to say that work expenses drove 5.9 million people into poverty—or that Social Security lifted 26.6 million out of poverty. That kind of cherry-picking of the data does not make a lot of sense, because presumably a good chunk of the people thrown into poverty through medical expenses are then lifted back out of poverty by Social Security, refundable tax credits, food stamps and so forth.

The ACA’s expansion of Medicaid–which not all states have accepted–in theory would eliminate many out-of-pocket expenses. Obama could have had made a stronger point by saying that the ACA potentially would lift millions out of poverty by eliminating medical expenses, because that would be adding a number to the positive part of the ledger.

All things being equal, the data indicate that fewer than 700,000 people end up in poverty because of medical out-of-pocket expenses under the supplemental measure, after accounting for the positive effect of government programs. That’s much less than “millions.”

The other factoid—“tens of thousands of Americans died because they didn’t have health care”—also has an interesting provenance.

The White House originally gave as its source a 2002 study by the Institute of Medicine, which was updated in 2008 by the Urban Institute. The IOM study estimated that 18,000 Americans died in 2000 because they were uninsured; the updated study concluded that 22,000 people died because of a lack of health insurance in 2006. This important study built on previous work, and has been the basis for other, later work that is often cited.

But IOM’s findings were debunked by Richard Kronick of the University of California-San Diego, who wrote a paper in 2009 concluding that IOM’s “conclusion is almost certainly incorrect.”

The IOM study was based on interviews in two studies, published in 1993 and 1994, with relatively small sample sizes. “Both studies estimated that lack of insurance was associated with approximately a 25 percent increased risk of mortality during the follow-up period,” Kronick wrote. “The apparent similarity of the results emboldened the IOM committee to conclude that lack of insurance was associated with a 25 percent increase in risk.”

(Side note: the data in these studies are relatively old. The 1994 study is based on Census data from 1982 to 1985 while the 1993 study used data collected between 1971 and 1987. Readers can decide how much relevance data more than four decades old have to today’s medical treatments and the demography of the uninsured.)

Kronick showed that the underlying data were different in each, with one controlling for health status and smoking behavior, while the other did not. If similar control data had been used, he concluded, the IOM study “would almost certainly have found no difference in survival between the uninsured and similar insured respondents.”  As part of his study, Kronick took a sample of more than 600,000 people, interviewed from 1986 to 2002, with follow-ups, and then controlled for a variety of factors, including how long they went without insurance. He consistently found no difference in the outcome.

One interesting theoretical problem is that only a percentage of people chronically lack health insurance, whereas as most people lack insurance for relatively short spells. That can affect care at the time a person is uninsured (Exhibit 13), but it makes it harder to track the impact of insurance on death rates over time. (The ratios may be changing as a result of the economic crisis. A new study just published by the Department of Health and Human Services’ Agency for Healthcare Research and Quality found that about 20 million people, or nearly 8 percent of the population under age 65, were uninsured for the four-year period from 2008 through 2011.)

Kronick could not definitely determine why there appeared to be little difference in health outcomes, but he wrote that “part of the answer may be that the safety net catches some uninsured people before illness and restricted access to medical care lead to premature death.” He said that “the results of this work strongly suggest that arguments in favor of universal coverage should not focus on the beneficial effects of that policy on the life expectancy of the currently uninsured.” (In other words, the president’s use of the data to advocate for the Affordable Care Act is potentially misplaced.)

Kronick is a respected researcher and not a partisan. In fact, since 2010 he has been a senior official at the Department of Health and Human Services, where he now heads the Agency for Healthcare Research and Quality, a type of in-house think tank. In other words, one of the the administration’s senior health-care officials has criticized a key study used by the president to make his claim.

In an interview, Kronick pointed to a more recent study, by Benjamin D. Sommers and others, published in 2012 by the New England Journal of Medicine, as a significant advance in understanding this issue.

Sommers found that expanding Medicaid coverage to poor adults was associated, after controlling for multiple variables, with a 6.1 percent drop in total deaths among non-elderly adults. The study compared states with Medicaid expansion, such as New York, with neighboring states that did not, such as Pennsylvania.

“Reductions were greatest among nonwhites and older adults, with smaller but significant reductions among whites and no effect among persons under the age of 35 years,” the report said. “Counties with higher poverty rates had larger mortality reductions.” (Note: Sommers worked for the Obama administration, at HHS, in 2011-2012 but the study was drafted before he worked for the government.)

“The evidence is stronger now than it was four years ago, and the president’s remarks are a reasonable estimate in the face of the uncertainty,” Kronick said.

The White House pointed to a number of other studies in support of Obama’s statement.  Yet there continue to be doubters, such as June E. O’Neill, a former director of the Congressional Budget Office, who also concluded in 2009 that there is “little discernible difference in mortality based on insurance status.”

At the very least, even if researchers agree that a lack of insurance has some impact on mortality, there appears to be  uncertainty about the precise magnitude of the effect.

The Pinocchio Test

The president’s use of the supplemental Census data to say “millions” of Americans fall into poverty because of medical expenses relies on a single variable, to the exclusion of other variables that would help keep people out of poverty. But his use of the phrase “millions” is appropriate in this circumstance.

Meanwhile, regarding people who died because they lacked health insurance, the president should speak with less certitude about “tens of thousands” of people who have died without health insurance, given the conflicting studies on this important issue. We wavered between One and Two Pinocchios, given the compelling critique in Kronick’s 2009 study, but his on-the-record comment that the evidence has shifted was a mitigating factor.

One Pinocchio

NEW: Send us facts to check by filling out this form

Follow The Fact Checker on Twitter and friend us on Facebook