The Washington PostDemocracy Dies in Darkness

Here are some red flags to fishy polling, on carbon taxes in particular

(REUTERS/Jim Urquhart)
Placeholder while article actions load

“Look, you have got an overwhelming majority of the public that wants this done. The latest Fox poll on this issue, 68 percent of Americans want the Keystone pipeline built.”

–Sen. John Hoeven (R-N.D.), Fox News Sunday, Jan. 11, 2015

“Senator Hoeven keeps talking about what 70 percent of Americans want. Seventy percent of Americans in a recent national poll also said they want a carbon tax or they want the EPA to be able to regulate carbon dioxide. What a majority of Americans want is clean energy jobs, improving our environment and growing good construction jobs.”

–Sen. Chris Coons (D-Del.), Fox News Sunday, Jan. 11, 2015

This column has been updated

Don’t you love it when politicians battle over policy by citing polls that purport to show the American public is on their side?

The Fact Checker has often warned readers against relying on opt-in Internet surveys that do not accurately estimate population values — and this was another example of that. We want to use this exchange over the Keystone XL pipeline as an opportunity to help readers — and politicians tempted to cite them — develop a more discerning eye when quoting polls. In particular, the first step is: always check the methodology.

Did the senators correctly portray two-thirds of Americans’ views on the Keystone pipeline and the regulation and taxation of carbon dioxide emissions? Let’s examine the evidence.

The Facts

Keystone pipeline

Polls have consistently shown the public is supportive of Keystone. Hoeven cited the December 2014 Fox News poll, which appears to be one of the most recent ones conducted, based on a random national sample of registered voters. The result — 68 percent said the government should build the pipeline, 26 said it should not — is consistent with the public opinion expressed in numerous polls.

Various polls have shown public support for the pipeline ranged from 59 percent to 70 percent. The support for Keystone is almost universal, as our colleagues at The Fix previously reported. According to the Pew Research Center, the most liberal of Democrats were the only group opposed to the project.

The Washington Post-ABC News poll in March 2014 also found similar results. The public overwhelmingly (85 percent) believed the pipeline would create a significant number of jobs. (As The Fact Checker previously reported, the vast majority of them would be temporary or indirect jobs.) Just under half believed the pipeline would pose a significant risk to the environment.

While the December Fox News poll cited the higher end of the percentage of Americans who expressed support, Hoeven correctly portrayed the public opinion on the project.

(Side note: Hoeven mentioned in this Fox News Sunday interview that a budget amendment he sponsored in March 2013 was an amendment to approve Keystone. “That was to move forward and approve the pipeline,” Hoeven said about his amendment. That is not entirely correct. It was a technical, non-binding amendment made to the Senate’s budget resolution. Hoeven’s spokesperson, Don Canton, said Hoeven was “clarifying that the amendment was directly related to the approval of the Keystone pipeline and not just a budget amendment. … It was widely understood that this was a vote in support of the Keystone pipeline.”)

Carbon tax

Coons said 70 percent of Americans “want a carbon tax” or “want the EPA to be able to regulate carbon dioxide.” His figures are from a December 2014 “Beyond the Beltway” report by the Benenson Strategy Group and SKDKnickerbocker. Both groups have ties to the Democratic Party (Benenson Strategy Group provided polling for the Obama campaign in 2012), but did not conduct the report for a client.

The report contained little detail about its methodology, other than that it was a result of 993 interviews conducted online with a sample of likely 2016 voters. That immediately raised red flags.

After speaking with the researchers, The Fact Checker learned it was an opt-in online survey of voters. Researchers compiled a representative sample of respondents based on Census data, then screened them based on the target population of the poll, using quotas on age, race, region and gender. Then, respondents were further screened based on their vote history in the last two major elections (2014 and 2012), and whether the respondent was likely to vote in 2016.

As we have previously reported, such opt-in online panels do not meet the methodology standards at The Washington Post, and has led to cautions by the American Association for Public Opinion Research. The full questionnaire is not available to the public, so we are unable to examine the battery of questions and how the questions about carbon tax and regulation were framed. (This is another red flag; a more transparent poll would provide the precise questions and the answer options.) [Update: The firms provided The Fact Checker later in the night the precise questions and answers relating to climate change, upon our request. We have embedded the questions at the bottom of this post. However, it is the Post’s standard that the entirety of the poll be made public.]

The survey further gives a false sense of precision when it claims a margin of error of 3.11 percent. This should not really be done with an opt-in survey, as the results simply reflect the people who have been surveyed.

Nonetheless, Coons correctly cited the study. In response to a series of questions about the issues Congress should act on to reduce the threat of global climate change, 75 percent of respondents agreed Congress should “have the Environmental Protection Agency regulate emissions of carbon dioxide as a pollutant.” In addition, 75 percent of respondents agreed that Congress should “enact a carbon tax on pollutants and use the revenues to pay for new, cleaner technologies for the future.”

Note that Coons cited 70 percent instead of 75 percent, and attributed the opinion to “Americans” instead of “likely voters.” It is inaccurate to attribute the statistic to a statistical sampling of the American population. But as this was an on-air interview on live television, we will give him a pass for the slip-ups.

This finding is generally in line with other polls that show the public supports the federal government limiting carbon emissions. But it is important to note the caveats in a variety of polls and surveys that accompany public support for a carbon tax.

Public support for carbon tax is sensitive to the potential size of the tax. The June 2014 Post-ABC poll found 70 percent believed the federal government should limit the release of greenhouse gases from existing power plants in an effort to reduce global warming. The same percentage believed the government should require states to limit the amount of greenhouses gases produced within their borders. When asked both questions under the scenario that it would raise their monthly energy expenses by $20 a month, the percentage decreased. At least six in 10 supported each, under the scenario of a $20 monthly bill.

An April 2014 Yale/George Mason survey also found 55 percent in support of “requiring fossil fuel companies to pay a carbon tax and use money to pay down the national debt.” But when the same question was asked in a follow-up version with a price — an average $180 a year for the average household — only 36 percent of registered voters supported it, and 62 percent opposed it.

The public’s enthusiasm for the carbon tax also was sensitive to what the tax revenues would potentially fund. For example, a July 2014 survey by the University of Michigan’s Center for Local, State and Urban Policy  found that most Americans oppose a carbon tax when there is no specified use for the tax revenue. A revenue-neutral carbon tax, in which the revenues are returned to the public as a rebate check, received 56 percent support. Sixty percent supported a tax with revenues used to fund research and development for renewable energy programs.

While Coons correctly cited the study that he used, there is an important caveat that public support of carbon tax is sensitive to factors that are specifically outlined in various polls and surveys. It is also the speaker’s responsibility to vet the data in a poll or a survey before citing it.

“Benenson Strategy Group is one of the most respected and trusted polling outlets in the country, and the point of the Beyond the Beltway Initiative isn’t to persuade, but to educate. We trust their counsel and we trust their data,” Coons’s spokesman Ian Koski said.

Coons is not the only one to cite this report; The Washington Post also reported some of its findings in a news article.

(Hat tip to our colleague, polling analyst Scott Clement, for vetting the surveys and polls cited in this article and pointing us to some of the sources included here.)

The Pinocchio Test

Polling for Keystone has consistently shown that a majority of Americans support the project. The public overwhelmingly views the pipeline as a significant job creator (though, as The Fact Checker has documented, that is a misperception), and far fewer believe it would pose a risk to the environment. Hoeven was correct in citing the figure, and we will not award him Pinocchios.

While the report that Coons cited generally reflects public sentiments on carbon tax and emissions regulations, public support for the carbon tax carries some caveats. The figure on carbon tax that Coons cited lacks context, and how sensitive public support for the carbon tax can be. We award Two Pinocchios to Coons — and to media outlets that have cited the poll — for relying on a survey with a narrow respondent panel to portray American public opinion. As we have noted before, it is up to the politician to properly vet the quality of polls before citing them.

Two Pinocchios

(About our rating scale)

Send us facts to check by filling out this form

Follow The Fact Checker on Twitter and friend us on Facebook

Environmental questions from survey

Questions related to environment by GlennKesslerWP