The fallout from Friday’s attack shows how the cogs of the Internet’s complex machinery can grind together to make evil go viral. The story starts in the fringe forums of radicalization that right-wingers frequent, such as 8chan. These communities run on irony: The line between provocative “trolling” and incitement to violence is strategically blurred. The curious are lured into what can feel just like another edgy Internet forum — until someone gets killed. The radicalizers maintain plausible deniability.
When the killing does happen, social media sites unintentionally help the hate spread. The alleged shooter live-streamed video of him carrying out the massacre on Facebook. Hours later, versions of the footage were still available on the Internet’s most trafficked sites. The material violated all the top platforms’ terms of service. But their algorithms cannot catch violating content as it streams live, and even after the fact, they often struggle. Sites such as YouTube “fingerprint” offending posts so that they can automatically prevent them from appearing again, but those techniques can also fail. They did this week.
Then there is the alleged shooter’s 74-page manifesto, posted to Twitter and 8chan ahead of the attack. It is littered with memes and cultural in-jokes that inflame and inspire the communities that understand the language. Those not in the know are invited down the rabbit hole, and reporting that attempts to decode the document can help lead them there. The celebrities the manifesto mentions have also been goaded into denouncing it to their followings — giving the author more notice and more notoriety.
White-supremacist sensationalism was impossible to stifle this week because multiple elements of the Internet worked together to accelerate it. There’s no simple cure for this. We want forums to stop hate as it festers but still respect free speech. News outlets must try to avoid amplifying manipulators even as they keep the public informed. Facebook, YouTube and Twitter must be faster, and smarter, in response to footage like Friday’s, but it’s not clear how they could prevent such an incident altogether. The Internet is home to the best and the worst of humanity at once.
This much is simpler: Governments, platforms and individuals alike must condemn and combat the vicious ideology that is, more than any one platform or any one post, responsible for the lives lost in New Zealand.