Some Words of Caution on Media Polls
By Richard Morin
Washington Post Polling Director
Monday, December 7, 1998
Too many critics of media polls are merely annoying scolds; Arianna Huffington comes immediately to mind. Some warrant a listen, while a precious few deserve much more. When these wise men and women speak, media pollsters and the editors and reporters they serve should take notes.
Philip Meyer is one such critic. He's walked the walk as a journalist, pollster and academic. Over the course of his career make that careers he has covered a beat as a reporter for Knight Ridder Newspapers when that chain was unequaled in its commitment to quality journalism. He attended Harvard for a year on a journalism fellowship and wrote "Precision Journalism," the first and best book describing how reporters could use the tools of social science to report the news. He is a former president of the American Association for Public Opinion Research, and currently is the Knight Professor of Journalism at the University of North Carolina. He's also been a friend and mentor to me.
Recently, Meyer was part of a group of media pollsters, academics and journalists who spoke at a forum organized by The Freedom Forum and the American Association for Public Opinion Research.
In his presentation, Meyer spoke directly to editors, offering a collection of do's and don'ts of reporting on public opinion polls. "If I were an editor, I would post the following guidelines on poll reporting in my newsrooms," he said, and he's right. His provocative, sometimes contrarian guidelines include:
Don't use polls to decide what issues to emphasize in election coverage. In newsrooms across the country, surveys are being used to identify issues important to voters, and then craft election coverage around this poll-driven issues agenda. It's called civic journalism, and it's all the rage. It's also "an abdication of our leadership responsibility," Meyer argues in a printed version of his remarks. "This behavior parallels the case of politicians who follow the obedient representative model to a fault and watch the polls to make up their own minds. Whether it's done by media or by government officials, it sucks the creativity out of the process."
Don't use polls to ask the public for its expert opinion in matters where it is not expert. Meyer cautions that public opinion is just that: perceptions, not reality. Asking Americans to evaluate the state of race relations, for example, tells you only what people think is happening. "To take public perception as reality, which journalistic interpretation of such polls sometimes do, is nonsense." In a similar vein, he urges media pollsters to sort informed from uninformed opinions. He suggests adopting Daniel Yankelovich's idea of asking a battery of questions to create a "mushiness index" to rate just how firmly survey respondents adhere to the opinions they are offering poll-takers. "Those of us who were skeptical about the public's ability to compartmentalize its thinking about Clinton the person and Clinton the leader might have learned from it," Meyer says.
Don't report polls without context. In real estate, it's location, location, location. In polling, Meyer argues, it's trend, trend, trend. "A polling number is generally not useful unless it is compared with another number whether from another place, another time, or another subset of the population," he writes. Meyer says the Watergate polls were most useful not as a referendum on impeachment, but "because they showed a clear declining trend in public support for the president." Conversely, polling on Clinton showed no change in public attitudes toward Clinton the leader (though they showed sharply declining views of Clinton's personal honesty and integrity.)
In addition to the don'ts, Meyer offered several "do's" things "that should be done but are not now being done."
Do use polls to help subgroups in the electorate understand one another. Here, he suggests a parallel between polls and old-fashioned town meetings, with survey results offering citizens the opportunity to find out what others are thinking. "Democracy is about compromise, and we can and should adjust our thinking to make a better fit with the community," he writes. "Polls aimed at helping groups in conflict understand one another would be an excellent application toward the goals of civic journalism."
Do use polls to enhance your leadership, not substitute for it. Remember: Polls are a sturdy tool for understanding public opinion, but not the only tool. And surveys should not substitute for the reasoned judgment of politicians as well as editors. "Feedback from polling can show both kinds of leaders how successful they are at getting important messages across," he writes. He paraphrases Edmund Burke's admonition that leaders in a representative form of government owe citizens their judgment, not merely the promise to follow public opinion.
"I think we undermine that when we take polls to find what to put in and what to keep out of the paper and when we treat a poll as though it's the final answer to a policy question," he concludes. "A poll should never be treated as the final answer to a policy question. It is just data to be considered in the context of many sources."
Richard Morin is director of polling for The Washington Post. "What Americans Think" appears Mondays in The Washington Post National Weekly Edition. Morin can be reached at email@example.com .
© Copyright 1998 The Washington Post Company
Back to the top