On Monday, President Trump tweeted about a new survey finding released earlier in the day. He said that a poll found that “50% of Americans AGREE that Robert Mueller’s investigation is a Witch Hunt.” That claim is in line with the USA Today report. But a close look at the survey and other recent polling suggests it may overstate the public’s skepticism of the Mueller probe.
Here’s how the question was asked: “President Trump has called the Special Counsel’s investigation a ‘witch hunt’ and said he’s been subjected to more investigations than previous presidents because of politics. Do you agree?”
There are a lot of ways to ask a question such as this, and it’s useful to measure whether Americans share Trump’s skepticism of the Mueller investigation. But this question and its interpretation skirt a couple of best practices for opinion-poll question wording, as highlighted by Monmouth University Polling Institute director Patrick Murray in several tweets.
First, the question is double-barreled. It tells respondents a) that Trump claims the special counsel’s investigation is a “witch hunt” and b) that Trump claims he is being investigated more than previous presidents for political reasons, then asks whether respondents agree with Trump. It’s possible that a respondent could think the Mueller investigation is a witch hunt but not that Trump has been investigated more than past presidents for political reasons, or vice versa. Yet the question asks respondents to react to both at the same time, and as a result we don’t know which specific claim they are agreeing or disagreeing with.
The placement of the question in the survey also may have led respondents to think about issues beyond the Mueller investigation. Just before asking about Trump’s “witch hunt” claim and excessive investigations in general, the Suffolk-USA Today survey stated that House Democrats had launched “a series of investigations” into Trump and asked whether they were going too far or doing the right thing. Having this as a lead-in question may have primed respondents to think of an entire “series of investigations” by Democrats rather than Mueller’s investigation when answering the subsequent query in question.
A second issue is that the question is unbalanced — it describes Trump’s viewpoint and asks, “Do you agree?” without an accompanying option “or disagree.” Balancing the response options gives a respondent two ways to respond to a question and not just one, signaling to the respondent that it’s acceptable to say the opposite. There’s also a broader phenomenon in which respondents are more likely to “agree” when asked an agree-disagree question than other formats, a reason many polls avoid such questions.
Lastly, while the question articulates Trump’s criticism of investigations, it does not offer an alternative view that counters Trump’s opinion — that the investigation is not a witch hunt or that the president has not been subject to more investigations than past presidents. Doing so would provide respondents with a more balanced choice between high-profile arguments. And in a similar way to lacking a “disagree” option, unbalanced arguments in a question make it far easier for respondents to side with the only response option provided.
These needling survey-nerd issues withstanding, there are a handful of non-double-barreled, balanced questions about this issue that other polling firms have asked, and they tend to find less skepticism toward the Mueller probe.
A February CNN poll found significantly more Americans who said the investigation was a “serious matter that should be fully investigated” (58 percent) than “mainly an effort to discredit” Trump’s presidency (37 percent). CNN has asked that question numerous times dating to August 2017 and has found between 55 percent and 64 percent saying the matter should be fully investigated.
A Washington Post-Schar School poll that same month also found more saying that Mueller was “mainly interested in finding out the truth” (57 percent) than “hurting Trump politically” (36 percent).
A January CBS News poll also found slightly more Americans who said the investigation into dealings between Trump associates and Russia was “justified” (50 percent) than that it was “politically motivated” (45 percent). But those results are not far from the Suffolk-USA Today findings, and a November CBS poll found 51 percent saying the investigation was politically motivated.
Meanwhile, a November Quinnipiac poll found that more registered voters said the investigation was “legitimate” (50 percent) than a “political witch hunt” (44 percent). Quinnipiac has asked the question over a year and has found between 48 percent and 54 percent saying it is legitimate.
These four surveys suggest that the public is more positive than negative toward the special counsel’s motives, with between 50 percent and 59 percent rating them positively, while between 36 percent and 45 percent see its motives in a negative light. A significant share of the public will probably look at Mueller’s final report with a skeptical eye, but probably not quite as many as suggested by the Suffolk-USA Today poll.