Most Read: Opinions

direct signup

Today’s Opinions poll

Would you use an app that tells you the partisan affiliation of products you're considering buying?

Submit
Next
Review your answers and share

Join a Discussion

Weekly schedule, past shows

Erik Wemple
On Twitter E-mail |  On Twitter Follow |  On Facebook Fan |  RSS RSS Feed
Posted at 05:48 PM ET, 05/17/2012

What’s wrong with the CBS/New York Times poll?

Visit msnbc.com for breaking news, world news, and news about the economy

Say all you want about the prolifically sexist Rush Limbaugh, but the guy knows a news story when it comes across his desk. On his Tuesday show, Limbaugh pounced on some fresh numbers from a CBS News/New York Times poll. Limbaugh noted:

Romney is leading Obama among women [46-44], after the contrived War on Women, after the contrived Sandra Fluke thing, after all of these efforts that have been expended to make Obama look like the first female president, the first gay president, the first Jewish president and they’re doing all of that, by the way.

Gender gap closed! Previous polls have found Obama leading Romney by large numbers among women, a crucial consideration for his reelection prospects. Any dent in that lead is a big story, and one that the radio host blasted CBS News and the New York Times for not trumpeting. “They’re burying it,” said Limbaugh. Correctomundo, Rushbo.

In its write-up on the poll, the New York Times appears to hype everything except the most booming finding in the poll. It goes deep on Obama’s recently announced position on same-sex marriage, revealing that 67 percent of respondents had concluded that the switch was motivated by political considerations and 24 percent attributed it to principle.

The story included this bit on the issue’s political dynamics:

The results reinforce the concerns of White House aides and Democratic strategists who worried that the sequence of events leading up to the announcement last week made it look calculated rather than principled.

Yeah, as if it’s all a matter of perception and spin. Could it be that the decision — or at least its timing — was indeed a political maneuver and the people are on to the president?

CBS News also declined to tout the gender-gap findings of its poll. The headliners related to same-sex marriage, the economy and Romney’s overall lead in the poll. In a separate analysis of the poll, CBS did mention that Romney had taken a lead among women voters, but that was at the bottom of the piece. Buried, as Limbaugh said. (A CBS News source indicates that there was no directive to marginalize the gender findings when writing up the summaries).

Odd behavior from two storied U.S. media outlets: Publish explosive poll results and barely comment on them. How to explain?

That’s where Paul J. Lavrakas, Ph.D, comes into the picture. Lavrakas is an election survey consultant for the Associated Press and in a few days ascends to the presidency of the American Association for Public Opinion Research (AAPOR). After looking at the materials relating to the New York Times’s handling of the research, Lavrakas offers some speculation on what’s behind the pooh-poohing of the alleged gender-gap reversal: “They [the New York Times] didn’t want to make anything out of it because they simply weren’t confident enough that it actually had indeed reversed,” says Lavrakas. “They didn’t feel it was strong enough evidence.”

Nor did Obama reelection aide Stephanie Cutter, who took aim at the poll in an MSNBC appearance on Tuesday (see video at top of post):

We can’t put the methodology of that poll aside. Because the methodology was significantly biased. It is a biased sample.

Cutter was hesitant to get into the nitty gritty of polling, though this scenario requires such tedium. Instead of conducting a fresh, national poll, CBS News and the New York Times did a “call-back” poll, in which they returned to the folks they’d queried in an April survey. There were 615 respondents.

The poll, as it turns out, entails a complicated exercise known as “weighting,” which the New York Times explains as follows:

Overall results have been weighted to adjust for variation in the sample relating to geographic area, sex, race, Hispanic origin, age, education, marital status and number of adults in the household. Respondents in the landline sample were also weighted to take into account the number of telephone lines at their residence.

There’s nothing new or scandalous about weighting; it goes on all the time. Yet Lavrakas insists that the people who’ll respond to a call-back poll are most likely the type who are fired up about the topic at hand — they’re often at extreme ends of the opinion spectrum. So when the CBS News/New York Times callers started reaching out to respondents from its April survey, it likely got cooperation from the people who are passionate about gay marriage and the like, according to Lavrakas. Using demographic weighting considerations to project from that group to a national picture of public opinion is a reach, he contends.

The “cumulative” effect of the methodological issues in the poll, notes Lavrakas, “would make someone reasonably concerned about the reliability of this [sample] projected to the national population.” That’s an important consideration, too, given that the poll’s findings on the gender gap are “strikingly different” from what other organizations have found, he says.

Note that the New York Times doesn’t mention the gender findings when it issued its official defense of the poll, via spokeswoman Eileen Murphy:

We stand by our latest poll with CBS News on same-sex marriage and its use of a call-back survey of respondents from our most recent poll in mid-April. While The Times and CBS regularly conduct fresh random sample surveys on a variety of issues, we occasionally employ a short-term callback method of previous respondents when we are interested in analyzing changes in public opinion on timely subjects, as we did, for example, upon the death of Osama bin Laden. As we pointed out when we published the article, results on subgroups from call back surveys have a larger sampling error.

More directness here would help. If the reference to sampling error on subgroups means that the Times has little faith in the gender-gap findings, why not just say so? I pinged Murphy in search of an explanation. “We’re not going to comment beyond our statement on this.” That’s factual: I failed in my attempts to get several other Times people to comment. Peter Baker, the lead reporter on the polling summary piece, responded this way when I reached him by phone: “You’ll have to talk with someone else.”

CBS News lockstepped with the Times. I sent an e-mail to one of the polling byliners listed on the CBS News report. I got a prompt response — from the PR department, reminding me that press inquiries go straight to them. That reminder came just before this: “CBS News declines to comment.” Message received — CBS News and the New York Times want this story to beat a swift and silent retreat into the archives.

The CBS News/New York Times poll had one moment of glory. For a short stretch there, it was rounded up in the Real Clear Politics poll aggregation machine. But then the Real Clear Politics people pulled it when they realized it was a call-back poll, which isn’t allowed to mingle with fresh national polling. John McIntyre, a founder of Real Clear Politics, says, “Some of the write-ups didn’t make that as clear as they might have.”

By  |  05:48 PM ET, 05/17/2012

Tags:  new york times, cbs news, poll, president obama, same-sex marriage, stephanie cutter, paul j. lavrakas

 
Read what others are saying
     

    © 2011 The Washington Post Company