Facebook said Thursday that it will permanently ban from its platform an Arizona-based marketing firm running what experts described as a domestic "troll farm” following an investigation of the deceptive behavior prompted by a Washington Post article last month.
The fake accounts, some with either cartoonlike Bitmoji profiles or images generated by artificial intelligence, complemented the real accounts of users involved in the effort, which largely entailed leaving comments sympathetic to President Trump and other conservative causes across social media.
Facebook stopped short of penalizing Turning Point USA or its president, Charlie Kirk, 26, saying it could not determine the extent to which the group’s leaders were aware of the specific violations carried out on their behalf, such as the use of fake accounts. Twitter also acted against the operation on Thursday, suspending 262 accounts involved in “platform manipulation and spam" — in addition to the several hundred accounts already removed last month following questions from The Post — but similarly did not boot Turning Point USA, a tax-exempt nonprofit group founded in 2012, or its affiliates from the online platform.
The decision not to punish the group financing the activity drew criticism from experts in disinformation, who said the limited action would signal to well-heeled organizations that they can get away with social media manipulation so long as they farm the operation out to vendors.
“If, once exposed, there are no consequences, others will try it, too,” said Philip N. Howard, director of the Oxford Internet Institute. “Long term, the industry shoots itself in the foot because limited action diminishes our trust in the authenticity of public life on the platforms. There’s been worry about white nationalists or other extremists using Russia’s tactics; now, it’s also the teenager around the corner who’s on the payroll of a troll operation.”
But Nathaniel Gleicher, Facebook’s head of cybersecurity policy, said the company needs to anchor its enforcement action in “evidence we can observe on the platform,” calling on regulators to set clearer standards about “what is acceptable advocacy and what is deception in politics.”
The accounts and pages Facebook removed had developed a following of nearly 400,000 accounts across Facebook and Instagram. They had purchased nearly $1 million in advertising.
In addition to domestic politics, the network Facebook removed also focused on trophy hunting, the company said. The accounts and pages assailed regulations on blood sports on behalf of a pro-hunting nonprofit called the Inclusive Conservation Group. A phone call placed to a number listed for the organization in tax filings went unanswered on Thursday.
Much of the activity drew on the spam-like behavior of bots and trolls, with the same or similar language posted repeatedly across social media, primarily in response to mainstream news articles.
The messages, parceled out in precise increments, glorified Trump and denounced his Democratic opponent, former vice president Joe Biden. They played down the threat from the novel coronavirus, falsely suggesting that case counts were being intentionally inflated, and cast doubt on the integrity of mail-in ballots, amplifying the erroneous claim that 28 million of them went missing in the past four elections.
Creating the appearance of a social media groundswell in support of Trump, the effort was carried out by teenagers, some of them minors, paid to participate in the campaign. The users set up and wielded fake personas in addition to using their personal accounts, Facebook said. The teenagers drew the posts from a shared online document and scored bonuses if their activity spurred higher engagement, said two people familiar with the effort who spoke on the condition of anonymity to protect the privacy of their underage nephew.
By enlisting human users in the effort rather than relying on computer scripts — and by modifying the posts slightly to avoid automated detection — the campaign offered a case study in the evolution of deceptive online tactics, Gleicher said, noting that many of the fake accounts had been caught by Facebook’s automated systems. The secretive operation, which began in 2018 and kicked into high gear ahead of the 2020 election, appeared designed to evade the guardrails Silicon Valley put in place to limit disinformation of the sort Russia used during the 2016 presidential campaign.
Likening the arrangement to a troll farm, analysts and technology industry experts said the Phoenix-based activity resembled efforts by Russian actors to disrupt American political discourse — just arising domestically.
“The activity gets at the heart of what’s happening in 2020 that’s distinct from 2016,” said Claire Wardle, co-founder and director of First Draft, a nonprofit organization that combats misinformation. “Because of the focus on Russia, we have failed to see how domestic operatives have used similar tactics.”
Turning Point Action, the Turning Point USA affiliate financing the effort, called any comparison to a troll farm a “gross mischaracterization.”
“This is sincere political activism conducted by real people who passionately hold the beliefs they describe online, not an anonymous troll farm in Russia,” the group’s field director, Austin Smith, said in a statement.
The Stanford Internet Observatory, which prepared a report on the network, concluded that most of the content “consists of fairly standard political and issue-based advocacy work.” What distinguished the activity from standard campaigning, the analysis determined, was the “extensive inauthenticity in the form of fake accounts,” as well as “secret coordination or unearned amplification.”
Jake Hoffman, Rally Forge’s president and chief executive, did not immediately respond to a request for comment on Thursday.
Hoffman, 35, is a city council member in Queen Creek, Ariz., and a Republican candidate for the Arizona House of Representatives. He describes Rally Forge as “one of the nation’s top conservative digital and communications agencies.”
As part of Facebook’s enforcement action, which includes a cease-and-desist letter and removal of all Rally Forge assets, Hoffman will lose his personal account on the platform, less than a month before the Nov. 3 election. Twitter is also permanently suspending him.
Kirk’s associates were at pains to present the effort as a project of Turning Point Action, the 501(c)(4) with more leeway to undertake political advocacy, rather than the main 501(c)(3), which is barred from campaign activity. Smith, the field director, said the operation reflected an attempt by Turning Point Action to maintain its advocacy despite the challenges presented by the coronavirus pandemic, which has curtailed many traditional political events.
“Turning Point Action works hard to operate within social platforms’ terms of service on all of its projects and communications and we hope to work closely with Facebook to rectify any misunderstanding,” Smith said.
Turning Point representatives also stressed Rally Forge’s oversight, directing specific questions about the activity to the vendor, which has received $221,349 from Turning Point Action this year, according to Federal Election Commission Data compiled by the Center for Responsive Politics.
Rally Forge and Turning Point USA are closely intertwined. Hoffman, in an email this week, described the people participating in the campaign as “Students For Trump digital activists,” referring to a project sponsored by Turning Point Action, which debuted last year.
Multiple teenagers involved in the effort viewed themselves as working for Turning Point USA or its affiliates rather than for Rally Forge, according to four separate family members interviewed by The Post. One teenager included a link to Turning Point USA’s website in his Twitter profile until The Post began asking questions about the activity this summer. Otherwise, it was not disclosed that the activity was part of a centralized operation by Turning Point or Rally Forge.
Until at least February 2019, Hoffman was listed in a staff directory on Turning Point USA’s website. Described as a “media and communications consultant,” he was featured alongside just three others: Kirk; William Montgomery, a co-founder of the group who died in July after contracting the coronavirus; and Tyler Bowyer, the organization’s chief operational officer. A review of LinkedIn profiles and other public documents suggests the two groups still have overlapping personnel.
Facebook said the deceptive election-focused activity began in 2018, in the run-up to the midterm elections. Two years ago, some of the accounts posed as left-leaning people, whereas the recent activity all reflected a conservative viewpoint.
The network was largely dormant until June 2020, according to Facebook. When it sprang back to life, much of the activity involved what the company called “thinly veiled personas” — using duplicate accounts with slight variations of the real names of the people behind them. Gleicher said the shift owed to Facebook’s increasing skill in rooting out fake accounts, forcing deceptive actors to “work harder” to avoid detection, sometimes using tactics that expose them in other ways.
Rally Forge has carried out additional ad campaigns for Turning Point USA and its affiliates, including a “Navajos for Trump” effort that drew the attention of the Navajo Times in August because, as the newspaper observed, actual Navajo Republicans were “not exactly front and center” in the marketing effort. Turning Point USA has paid Rally Forge hundreds of thousands of dollars in recent years for advertising and marketing, financial filings show.
Only a handful of other clients are apparent from financial disclosures and media reports. Rally Forge received nearly $120,000 in 2016 from a political action committee supporting Kelli Ward, who was unsuccessful in her primary challenge that year to Sen. John McCain (R-Ariz.), and an additional $50,000 in 2018 when she ran again in the Republican primary. She now chairs the Arizona Republican Party.
In 2016, more than $300,000 directed to the firm from Hoffman’s political action committee, RallyPAC, was used to finance a pro-Trump meme campaign on a Facebook page called, “I Love My Country,” according to CNBC. A page with that title was taken down on Thursday — part of Facebook’s broader removal of Rally Forge content that showed signs of inauthentic behavior, not because of the thematic messages hosted on the page.
Leading up to Facebook’s action, the page was still posting memes that bear resemblance to Turning Point USA content. A recent post asserts, “No one is tougher on Russia than President Trump."