“This was an idea that was supported by a strong majority of the American people — including nearly half of Republicans. The majority of millionaires supported it. And Senate Republicans didn’t listen.”
— President Obama, Lorain County Community College in Ohio, April 18, 2012
Many surveys, by pollsters such as Gallup and Washington Post-ABC News, show strong public support for the president’s proposal — killed in the Senate this week — to impose a tax surcharge on people with adjusted gross income of more than $1 million, known as the Buffett Rule.
But we were struck by the president’s assertion that even a majority of millionaires support such a tax. The claim that two-thirds of millionaires back the plan is also promoted on the Obama campaign Web site, under the headline: “Millionaires stand in support of the Buffett Rule.”
The White House directed us to an article that appeared in the Wall Street Journal, which was headlined “Millionaires Support Warren Buffett’s Tax on Rich.” The article cited a study by Spectrem Group, which claimed that “68 percent of millionaires (those with investments of $1 million or more) support raising taxes on those with $1 million or more in income.”
First of all, note that the article referred to people with investments (ie, net worth) of $1 million or more, which is different from people with $1 million in income. People with net worth of $1 million includes many people who make less than $1 million a year, so at least some, if not many, of the people who supported higher taxes actually would not be affected by the Buffett Rule.
So Obama — and the Wall Street Journal — are mixing up two different types of millionaires.
The other problem is that the article gave no detail on how the survey was conducted, the sample size and so forth. We found a brief online description on the Spectrem Web site and we spoke to George Walper, Spectrem’s president.
Walper said the questions were asked once last fall as part of a regular sample of high net worth individuals via an online survey. He said about 500 people in the survey have a net worth of $100,000 to $1 million, 333 have a net worth of $1-$5 million and 165 have a net worth of $5-$25 million. The statistics cited by the Journal were just from the last two categories, and he said the margin of error was plus or minus 4 percent.
The Washington Post has strict standards about the types of polls that we quote. We are especially wary of online polls, and a standard practice before quoting a poll in a news story is to make sure it is vetted by The Post’s polling unit. So we asked the Post pollsters to assess the Spectrem survey.
Here is what Polling Analyst Scott Clement reported:
George Walper had me talk to Tom Wynn, director of affluent research for Spectrem Group. He says the survey results are from an opt-in online panel. That's a big weakness as far as polling is concerned. We don't report these surveys per our methodology standards at the Post, and the American Association for Public Opinion Research's 2010 task force cautions researchers to avoid such panels when trying to estimate population values — i.e. what percentage of millionaires support the Buffet rule. Wynn also refused to supply the topline results of the survey, a standard item of survey disclosure for publicly released polls.
Do such details really make a difference? Well, another Spectrem poll, from a month earlier, has been cited by CNN Money as evidence that millionaires do not support the Buffett Rule. That survey found that only 24 percent of millionaires said that higher taxes on higher income was the fairest tax system.
A White House official countered that The Washington Post, from time to time, has cited Spectrem’s research, including even The Fact Checker in 1999, for an article on 401k investments. Not everyone at The Post gets the message to be careful with polls, but the eight articles cited by the White House did not mention the poll in question.
“We're not suggesting a blanket dismissal of the group's work, only that this poll doesn’t merit publication,” said Post polling director Jon Cohen. “Some of the [articles noted by the White House] is market info — who knows where from. Some is survey data, but none of it vetted by us.”
Walper said he was “very confident” that his poll was representative of all millionaires.
“In his speech, the President cited a survey that has been cited by, among others, the Wall Street Journal and Time magazine,” said White House spokesman Jay Carney. “The survey itself was done by an organization whose work the Washington Post has deemed credible in many of its own articles. By the tortured reasoning set forth in this ‘fact-check,’ much of the content that appears in the Post on any given day would merit multiple Pinocchios.”
The Pinocchio Test
The explosion of polls has made it essential to be careful about understanding how they are conducted and what they really mean. The president has on occasion cited “a survey” as reporting this claim about millionaires and we suppose he could have been even more specific by citing a “survey reported by the Wall Street Journal.” But this week, and on his campaign Web site, all caveats were dropped and the assertion was stated as fact.
Reputable surveys already show the Buffett Rule is broadly popular, so there is little reason to hype the claims even more by citing a survey with questionable provenance. You can’t just accept any poll, no matter how much it fits with your convictions. The president, in particular, should have higher standards than that.
UPDATE: This column was focused on the president’s failure to insist on compliance with accepted public opinion research standards, but Spectrem Group wanted to reiterate it has no control over how people use its data. The firm sent the following additional comment on this column: “We are a market research firm, not a political polling firm. Spectrem’s market research methodologies--which include opt-in online surveys targeting a small sub-segment of the US population at large-- necessarily differ from the random sampling that the Post requires for polls that it cites.”
Check out our candidate Pinocchio Tracker
Track each presidential candidate's campaign ads