In October 2014, scholars at Dartmouth and Stanford conducted a research project in two Montana judicial elections. It caused considerable controversy. As part of the project, flyers that placed Montana Supreme Court candidates on an ideological scale with Obama and Romney as reference points were sent to 102,780 voters, approximately 15 percent of the state’s electorate.

In particular, the researchers targeted Republican and Democratic precincts and voters likely to be Republican or neutral in Republican precincts and Democratic or neutral in Democratic precincts. The researchers assumed that turnout would be higher among Republican voters so the flyers were sent to 38,515 voters in Republican precincts and 64,265 voters in Democratic precincts in an effort to achieve balance.

According to Dartmouth College, “The focus of the study was to measure whether providing the information in the Flyer had an identifiable impact on voter participation, as measured by voter turnout and voter rolloff in precincts that did and did not receive the Flyer.”

Now, the report of the Commissioner of Political Practices of the State of Montana, Jonathan Motl, is available.  I was contracted to evaluate the study as part of this report.  Here is what Motl found.

The Commissioner determined that the project violated Montana law. He found the flyers were “express advocacy” rather than “issue advocacy.” In the 1976 case Buckley v. Valeo, the court ruled that there was a difference between discussion of issues and express advocacy supporting candidates.

In a footnote to the decision, the Court gave examples of “magic words” that constitute express advocacy. In subsequent decades this concept has undergone refining and now there is substantial precedent for a test of “functional equivalent of express advocacy.” For instance, in the 2007 Supreme Court case FEC v. Wisconsin Right to Life, Chief Justice Roberts employed this test.

The Commissioner determined that the Montana flyer study met this standard for “express advocacy.” This is because the flyer exhorted voters to go to the polls and vote for candidates and because its intent was to secure additional Republican votes in Republican precincts and additional Democratic votes in Democratic precincts. Thus, the flyer was an appeal to vote for candidates. Montana law does not have exceptions for academic studies.

The decision has been referred to the County Attorney for Lewis and Clark County in Montana. Normally the Attorney waives the right to prosecute and the Commissioner decides upon legal action. Normally most cases are resolved by the payment of a fine. The Commissioner also asked the Lewis and Clark County Attorney to review the unauthorized use of the Great Seal of Montana in the flyer and the possibility of violating the impersonation statute.

Of course, universities and colleges have a mechanism to ensure that research conforms to ethical standards. Researchers must submit studies using human subjects for review to the Internal Review Board, or IRB. Here is where several problems arose.

One is that the Montana study was never submitted to any IRB for approval. A study about “information and extremism in U.S. Primary elections” involving the 2014 primary in the First Congressional District of New Hampshire was submitted to the Dartmouth IRB, but not the Stanford IRB.

However, its purpose, according to Dartmouth, was to determine whether giving information about a candidate’s ideology would “have an effect among voters, particularly among moderate, independent or unaffiliated voters.” The Montana flyer study was much different.

But the Montana project also reveals that the IRB process itself is fundamentally flawed. In my report to the Commissioner, I concluded that even if the researchers had scrupulously followed all procedures, the Dartmouth IRB would likely have still approved the research project. The IRB raised no objection to the design of a study in which campaign communication was intended to influence people’s votes, or to the risk that such a study could change the outcome of an election.

This is because the IRB process is narrowly tailored to protect the individual subject from research malpractice rather than individuals as an aggregate. And this in turn is because many colleges and universities treat the IRB process as “one size fits all” for all disciplines and shoehorn political science research into a review process designed for biomedical and health research. However, as Macartan Humphreys has noted, the standards that apply to biomedical and health research are often less relevant to political research.

In this research, the “subject” most vulnerable is not the individual but the community, the candidates, and political institutions. By only focusing on the individual subject, the researchers and the IRB concluded that the potential harm to individuals receiving a flyer in the mail was minimal. The protection of the community was not considered.

But to the community — the citizens and candidates in Montana — the outcome of the Montana Supreme Court election was very important. So the harm is not from receiving the flyer in the mail but the possibility that the flyer would change the outcome of an election.

It is possible, as Chris Blattman argues, that we sometimes need to allow research to intervene in the real world. But in the case of the Montana experiment, the “real world” arguably complicated the ethics of the experiment.  Outside conservative groups had poured what turned out to be well over $1 million in the state and Montanans were inundated with advertising connecting an incumbent Supreme Court Justice with President Obama, an unpopular figure in Montana. The Supreme Court race was no ho-hum affair.

We simply need to change the vetting processes for political research. The presidents of Dartmouth College and Stanford University wrote a letter of apology to Montana voters stating, “We recognize that the purpose of elections is to enable our democratic systems to operate, and that no research study should risk disrupting an election.”

Nonetheless, both institutions seem to backpedal from that position in their subsequent responses to the Commissioner. Dartmouth College has now put in place requirements that contact information for the academic investigator must be supplied in a research project. However, the more fundamental issue of approving a study that could potentially change the outcome of an election is not addressed and such a study, it seems, may be able to still secure approval.

In its response to the Commissioner, Stanford stated, “the use of the seal was a mistake, and was the primary reason why Stanford agreed with your office to mail the apology letter.” Stanford no longer lists the disruption of the election as a major error.

Academics need to do more to engage the ethical issues surrounding the impact of research on elections. Scott Desposato is correct that we cannot outsource ethical judgments to review boards. The whole IRB process backfires if researchers view them as irrelevant irritants, which is too often the case. To learn from the controversy surrounding the Montana study, we need nothing less than a re-structuring of the IRB process for political research.

Jeremy Johnson is Associate Professor of Political Science at Carroll College in Helena, Montana.