“Tell the haters: Newly insured Americans are happy, relieved, and confident.”
–tweet by Organizing for Action, citing a graphic created by Enroll America
We first spotted this as a tweet by the Barack Obama-affiliated Organizing for Action. But it turns out it is really a graphic unveiled this week by Enroll America, which seeks to help people get enrolled for health insurance.
The numbers are based on a survey of 671 newly enrolled Americans by PerryUndem Research, with a margin of error of plus or minus 5.2 percent. The interviews were conducted between April 10 and April 28, just as many people were first experiencing health care under the Affordable Care Act.
We have no issue with the survey methodology, which was based on online panel that was recruited offline through probability methods, in that every non-institutionalized American had some chance of being selected. (Our colleague, polling analyst Scott Clement, vetted the survey for The Fact Checker.) The survey itself is very interesting, providing a ground level look at how people feel about getting health insurance.
But we do have issues with how Enroll America described the findings in its survey.
The key claim is that “newly insured Americans are happy, relieved, and confident.” Then the info graphic breaks it down as thus, going clockwise:
- “4x more likely to feel happy about their new coverage than unhappy.”
- “Over 7 out of 10 of newly enrolled people are confident they can afford their monthly premiums.”
- “The word that came to mind the most for newly enrolled people to say how they feel? Relieved”
- “Almost 6 in 10 newly enrolled people believe their plan has enough doctors and providers to choose from.”
Let’s look at these one by one.
The 4x number is misleading, in that, in question 53, just 41 percent who enrolled are either very happy (20 percent) or somewhat happy (21 percent) with their coverage. The 4x statistic is derived by combining somewhat unhappy (7 percent) and very unhappy (4 percent) and then comparing 11 percent against 41 percent.
But 16 percent are neither happy nor unhappy and a whopping 31 percent say it is “too soon to tell.” So how did enrollees simply get labeled as happy?
“As we note in the survey report, it’s true that 31 percent said it was too soon to tell whether or not they were happy with their coverage; that means the numbers may change over time, but the data we have is easily robust enough to indicate how people currently feel about their coverage,” said Justin Nisly, Enroll America’s spokesman. “We couldn’t include every bit of data in the infographic, but what we did include is entirely accurate, and the full context is readily available.”
The statistic that 7 out of 10 are confident they can make their payment appears okay. In question 45, 74 percent said they were very confident or somewhat confident they could make their payments. But one of four were not confident.
The phrasing of the next element, “relieved,” also veers into misleading. Calling it “the word that came to mind the most” implies respondents were asked to come up with their own word. Instead, “relieved” was one among a choice of four words in question 54, including “confused,” financially stressed,” “in control,” and “it doesn’t really affect me.”
In other words, “relieved” was one of four words that people could choose from, but it was not “the word that came to mind the most.” In fact, relieved was chosen by 47 percent, so it was not even the majority choice.
“When offered a range of positive, negative, and neutral phrases to describe their feelings about having health insurance, 47 percent chose ‘relieved’ – the next most chosen phrase had just 17 percent, so I think it is entirely accurate to say that the word that best described the way consumers feel about their coverage is ‘relieved,’” Nisley said. “I guess you could quibble with using ‘came to mind’ to describe a question that offered various options, but I don’t think that changes the basic point that a majority of respondents chose positive words to describe ‘what it feels like to have health insurance.’”
Finally, question 49 did ask enrollees whether their plan had enough doctors, and 56 percent said yes. But the very next question asks whether a person had tried to see a doctor yet, and 61 percent said no. And 63 percent said they had not tried to get prescriptions yet. And nearly half—49 percent—said they had not yet tried to use their insurance yet.
In other words, it seems a bit early in the game to start declaring that people are “happy, relieved, and confident.”
“We released a full report on the survey a week before this infographic, and it covers all of the data in detail, both positive and negative (for instance, reasons people chose not to enroll),” Nisly said. “The infographic was a way to draw attention to a particular section of the survey and encourage folks to read the whole thing, not a substitute for the survey itself, which we’ve shared quite freely.”
He added: “’Happy, Relieved, Confident,’” is just the title of the infographic, but it is backed up by the supporting detail from the survey.”
The Pinocchio Test
We realize an infographic has to condense information, but there is a high burden to meet for one designed to be tweeted. In this case, Enroll America grabbed the statistics that cast the Affordable Care Act in the best possible light, even though a huge percentage of the respondents had barely a chance to experience the system and form opinions about it. We highly doubt many people actually read the report before they retweeted the graphic.
Send us facts to check by filling out this form