Yesterday's NY Times-CBS poll that showed Mitt Romney gaining strength and ahead among women for the first time, and that President Obama's gay marriage decision is being viewed by the public with a jaundiced eye, got a fair amount of buzz among political insiders. Republicans, who have often criticized those news organizations broadly and their polling specifically, were delighted by the results.
But were the results accurate? Stephanie Cutter, a senior Obama campaign strategist, immediately raised questions about the survey, which interested me because she is too much of a pro to whine about polls. So what's up?
It seems the two news organizations used an accepted, but fairly uncommon, polling technique in which respondents to a previous survey are called back, after some period of time, to get their responses to a change in a market or political landscape. There are some real flaws in this methodology: 1.You almost never get the same size survey on round two as respondents drop out; indeed, the sample size was much smaller on yesterday's poll than the original; 2. Once people are surveyed and told they will be again on the same or similar subject, they become sensitized, and therefore biased, in the minds of many experts.
So why do this kind of polling? It's cheaper and faster. In order to get enough respondents to constitute a poll with an acceptable margin of error, you have to call as many as five or six phone numbers to generate one interview. That takes time and money. In a callback, you only call the people you surveyed before.
This perhaps tedious dissertation on polling is to make this point: Reputable news organizations have an obligation to spend the time and money to get it right, or why bother? There are more than enough lousy polls already.