Poli-Sci Perspective is a weekly Wonkblog feature in which Georgetown University’s Dan Hopkins and George Washington University’s Danny Hayes and John Sides offer an empirical perspective on the issues dominating Washington. In this edition, Hopkins examines how slogans and messaging affect opinions of health reform. For past posts in the series, head here.

Health-care reform is back in the headlines, with the recent decision by the Obama administration to delay the employer mandate and with the new public relations efforts planned on both sides of the law.  But on its own, new rhetoric isn’t likely to move public opinion.  Prior rhetoric certainly hasn’t.

Not long after the 2010 midterms, New York Times reporters Jennifer Steinhauer and Robert Pear summarized the conventional wisdom when they wrote that the Obama administration and its allies “largely lost the health care message war in the raucous legislative process.”  It’s easy to see where that idea came from.  Public support for health-care reform declined markedly during the first year or so of Obama’s presidency, at the same time that Republicans were ramping up their attacks on the legislation.  Perhaps the most memorable rhetorical salvo came in August 2009, when Gov. Sarah Palin denounced a bill that could put her son before Obama’s “death panel.”  Commentators left and right later pointed to that phrase as an example of effective anti-reform messaging.

But as I show in a working paper, the evidence on health care messaging and public opinion tells another story.  On ‘death panels’ specifically, if the phrase was so powerful, why was it not used by a single person when a Pew Research Center survey asked Americans to explain their health care views a few months later?  But I am getting ahead of myself.

To consider the role of messaging systematically, I used an algorithm developed by computer scientists to identify clusters of words in almost 1,500 Senate press releases about health care from 2009 and 2010.  The words have been stemmed—that is, similar words like “propose” and “proposed” have been combined into a single word stem.

Below, we see one such cluster of word stems, a cluster that is characterized by words related to the cost of health care reform: “trillion,” “dollar,” “spend,” and “cut” define it.  Put differently, those words are likely to appear together in press releases, suggesting that they are part of the same topic.  The figure illustrates the proportion of senators’ press releases that fell into that topic over the course of the health care debate.  (As a reminder, the legislation passed the Senate in December 2009, and the Senate bill was then passed by the House of Representatives in March 2010.)  The solid blue line shows the proportion for Democratic press releases, while the dashed red line shows it for GOP press releases.

As the figure makes clear, during 2009 and 2010, Republican senators were talking about the costs of health care reform a lot more than their Democratic counterparts—no surprise there.  But notice, too, that Republican rhetoric shifted in late 2009, just as the Senate was taking up the bill.  Suddenly, the GOP was talking a lot about legislative procedure, as the figure below shows.  Republicans’ rhetoric changed to emphasize amendments, procedural issues, and the deals Senate Democrats were striking to win passage for the bill.  And in January 2010, the GOP turned to yet another line of argument, emphasizing how the legislation would affect Medicare.

The  GOP wasn’t deploying a single argument throughout the debate.  Instead, it was trying any argument that might give it leverage, from Medicare to the “Cornhusker Kickback.”  And that conclusion isn’t just true for press releases.  In the accompanying research paper, I also look at the language used on Sunday political talk shows and find the same pattern.

Now let’s consider the words used by the American public during the same period.  If the parties’ messagingwas influential, we might expect Americans to draw on the more persuasive messages when explaining their views.  This method should offer a more subtle test than looking at the overall share of Americans who support or oppose health-care reform—analyzing the words Americans use when justifying their opinions allows us to detect changes in reasoning even when overall assessments remain unchanged.

Using open-ended survey questions asked by the Pew Research Center and the Kaiser Family Foundation, I then estimated which words were typically used by the Affordable Care Act’s supporters and opponents.  Consider the figures below, which plot each word’s frequency on the x-axis and the differential between supporters’ and opponents’ use of the word on the y-axis.  Words that are higher are those commonly used by health-care reform’s supporters, while the words that are lower are used primarily by its opponents.  And words that are further to the right are more frequently used in general.

The figures cover the eventful 15 months from the end of President Obama’s honeymoon in July 2009 to the eve of the overwhelming GOP victory in November 2010.  Keep in mind that in July 2009, the tea party was in its infancy, and no one was talking about “death panels.”  But despite all the arguments by both sides that followed, despite a September 2009 prime-time presidential address, despite heavy television advertising, and despite months of legislative wrangling, the words that Americans used to defend their positions were strikingly stable over that period.

First, take a look at July 2009.  Like Senate Republicans, Americans opposed to the bill were worried about its cost: “pay,” “money,” and “cost” were three of the defining words they used.  And in October 2010, those same words remained among the most frequent words used by the law’s opponents.  Sure, pundits might have starting talking in terms of “death panels” over that period—but there is not much evidence that the public did.  For opponents, the bill’s cost and its increased role for government were core reasons for opposing it even before the rhetorical war heated up.

Similarly, supporters of the legislation in July 2009 talked about helping people afford health care and about expanding access.  In the fall of 2010, those same words were among those most frequently used by supporters.  A lot changed between July of 2009 and October of 2010.  The rhetoric and tactics of the parties certainly did.  But by and large, the way that Americans talked about health-care reform did not.

Might the efforts of Oprah Winfrey and others encourage Americans to sign up for health insurance?  Sure—and a separate vein of research in political science provides evidence that Americans’ actual experiences with social policy programs can have a pronounced influence on their views.  Signing up people, not shifting public opinion, is the aim of the Obama administration’s latest efforts.  But after five years of debate, it is clear that public opinion on the Affordable Care Act is unmoved by catchy slogans alone.