In the wake of the deadly attack against two mosques in New Zealand, police officers sit in their vehicle out side the Al Aqsa Islamic Society mosque in Philadelphia, ahead of prayers Friday, March 15, 2019. (AP Photo/Matt Rourke)
Matt Rourke/AP
In the wake of the deadly attack against two mosques in New Zealand, police officers sit in their vehicle out side the Al Aqsa Islamic Society mosque in Philadelphia, ahead of prayers Friday, March 15, 2019. (AP Photo/Matt Rourke)
Now playing
02:20
Suspect's manifesto part of a disturbing pattern
Police stand outside a mosque in Linwood, Christchurch, New Zealand, Friday, March 15, 2019. Multiple people were killed during shootings at two mosques full of people attending Friday prayers. (AP Photo/Mark Baker)
Mark Baker/AP
Police stand outside a mosque in Linwood, Christchurch, New Zealand, Friday, March 15, 2019. Multiple people were killed during shootings at two mosques full of people attending Friday prayers. (AP Photo/Mark Baker)
Now playing
02:26
Scenes from one of New Zealand's 'darkest days'
Now playing
02:44
Some far-right politicians' language encourages hate
Human chain forming around Wellington mosque before Friday prayer CREDIT Veronika Meduna
Veronika Meduna
Human chain forming around Wellington mosque before Friday prayer CREDIT Veronika Meduna
Now playing
02:43
New Zealanders form human chain to back Muslim community
CNN
Now playing
01:42
Man describes confronting New Zealand shooting suspect
Tributes laid at the police line close to the Al Noor mosque in Christchurch where 41 people died.
CNN/Hilary Whiteman
Tributes laid at the police line close to the Al Noor mosque in Christchurch where 41 people died.
Now playing
01:51
Terror attack death toll rises to 50
Facebook/Wasseim Alsati
Now playing
00:55
New Zealand attack survivor shares message from hospital
A police officer secures the area in front of the Masjid al Noor mosque after a shooting incident in Christchurch on March 15, 2019. - Attacks on two Christchurch mosques left at least 49 dead on March 15, with one gunman -- identified as an Australian extremist -- apparently livestreaming the assault that triggered the lockdown of the New Zealand city. (Photo by Tessa BURROWS / AFP)        (Photo credit should read TESSA BURROWS/AFP/Getty Images)
Tessa BurrowsAFP/Getty Images
A police officer secures the area in front of the Masjid al Noor mosque after a shooting incident in Christchurch on March 15, 2019. - Attacks on two Christchurch mosques left at least 49 dead on March 15, with one gunman -- identified as an Australian extremist -- apparently livestreaming the assault that triggered the lockdown of the New Zealand city. (Photo by Tessa BURROWS / AFP) (Photo credit should read TESSA BURROWS/AFP/Getty Images)
Now playing
03:18
Online platforms scramble to remove terror attack videos
Rosemary Omar worried mom son missing
Matthew Mckew
Rosemary Omar worried mom son missing
Now playing
01:57
New Zealand mom worries: Waiting to see if son is all right
New Zealand mosque analysis
Levi West
New Zealand mosque analysis
Now playing
01:09
Expert: Catching videos on social media harder than you think
TVNZ
Now playing
02:19
New Zealand Prime Minister: This was a terrorist attack
Matthew Mckew
Now playing
00:41
Survivor: I prayed the shooter would run out of bullets
TVNZ
Now playing
01:28
Police commissioner: Vehicles we stopped had IEDs
TVNZ
Now playing
02:06
Prime Minister: One of New Zealand's darkest days
TVNZ
Now playing
02:01
Witness: Shooting lasted for 10-15 minutes
CNN —  

Social media networks are facing a reckoning for their role in spreading far-right terrorist propaganda, after a deadly attack on a New Zealand mosque was live-streamed on Facebook. 

Recognizing that the gunman’s gruesome stunt was designed to go viral, governments and business leaders have called on Facebook, Twitter and Google to do more to rid their platforms of hate speech that could encourage violence.

Less than two weeks after the terror attack, Facebook announced that it would ban all “praise, support and representation of white nationalism and separatism” on its flagship platform and on Instagram.

Most social platforms have taken aggressive action to tackle another extremist group in recent years: ISIS. But their response to white supremacists has been slower.

Experts say that’s because far-right extremism online isn’t as easy to detect, citing the movement’s innate political nature and culture of ambiguous in-jokes. The messages spread online by white supremacists are commonly compared by researchers and counterterrorism experts to those propagated by the Islamic extremist terror group, for their shared tactic of online radicalization.

“I don’t think (social media companies) have the capabilities, even at a basic level, when white supremacist content is flagged to act on it,” Joshua Fisher-Birch, a content review specialist at the Counter Extremism Project, a US-based nonprofit, told CNN, adding that he doubted those companies were taking white supremacist content seriously enough.

Fisher-Birch, who monitors the proliferation of extremist content online, said that while social platforms and video streaming services have been vigilant in suspending accounts that share ISIS content, white supremacist messaging has gone unfettered.

White nationalism online is outpacing ISIS propaganda

New Zealand Prime Minister Jacinda Ardern has said that her government will investigate the role social media played in the deadly attack.
Mark Tantrum/Getty Images
New Zealand Prime Minister Jacinda Ardern has said that her government will investigate the role social media played in the deadly attack.

Amid a social network crack down on ISIS propaganda, US white nationalist movements have thrived, with followers growing by more than 600% on Twitter since 2012, according to a 2016 study by the Program on Extremism at George Washington University. In fact, growth in white nationalist and Nazi accounts on Twitter outpaced ISIS by almost every metric, in part because they faced less pressure from suspensions.

J.M. Berger, the study’s author, said that the findings “suggest that the battle against ISIS on social media is only the first of many challenges to mainstream, normative values,” adding that “other extremist groups” have been able to learn from ISIS’ successes and mistakes to develop their own digital strategies.

That strategy was on full display in the staging of the New Zealand attack. On Twitter and 8chan, an online message board that allows unconstrained speech including racist and extremist posts, the shooter teased his imminent attack with a call to spread his message through memes and “shitposting,” slang for pushing out a huge volume of low-quality, ironic content to illicit a reaction. 

The response, according to Google, was unprecedented: In the first 24 hours after the video was streamed live on its platform, Facebook said it removed 1.5 million videos of the attack globally, of which over 1.2 million were blocked at upload. 

The flurry came after Facebook’s systems failed to catch the initial video, which was viewed live fewer than 200 times before being unleashed across the internet. The company said it took it down only after being alerted by New Zealand police.

In the wake of criticism over how it polices its platform, Facebook said that while it had long prohibited hateful treatment of people based on race, it hadn’t applied the same rationale to white nationalism and white separatism. It will soon start directing people who search for terms associated with white supremacy to organizations that help people leave hate groups.

Facebook’s failure to spot the initial livestream comes amid repeated pledges to step up moderation of its platforms. The company has recently hired tens of thousands of content moderators. Google’s video streaming service YouTube has also been investing in human moderators to clean up offensive videos on its service.

Google said that the volume of related videos shared on YouTube in the same period was, at times, as fast as “a new upload every second.” Google said that it removed tens of thousands of videos and terminated hundreds of accounts.

Twitter said it was “continuously monitoring and removing any content that depicts the tragedy.”

The New Zealand attack appeared to be one of the first times that far-right terrorism was treated by social networks with the same wholesale erasure as pro-ISIS material. But the speed at which copies of the video spread online reflects the scale of the challenge facing social networks.   

Why is white supremacist content so hard to stop?

A demonstrator hangs banners from multi-faith group 'Turn to Love' during a vigil at New Zealand House in London.
Kirsty Wigglesworth/AP
A demonstrator hangs banners from multi-faith group 'Turn to Love' during a vigil at New Zealand House in London.

In 2016, under pressure from lawmakers in the United Kingdom, Europe and the United States, YouTube, Facebook, Microsoft and Twitter launched a shared industry database of “hashes” – digital fingerprints of extremist imagery. Its aim is to flag ISIS propaganda for removal and, in some cases, block it from ever being posted.

Since its peak in 2014, when ISIS’ media apparatus spammed social media with grisly videos, the group’s online presence has shrunk dramatically, the George Washington University report points out. 

After the New Zealand attack, Facebook said it had shared hashes of more than 800 visually-distinct related videos via its collective database. 

But Julia Ebner, a fellow at the London-based Institute for Strategic Dialogue, who specializes in Islamist and far-right extremism, tells CNN that this strategy has not been deployed as aggressively by Facebook and other Big Tech firms to counter far-right extremist content in the past. 

That’s partly because white nationalist messaging is not easily recognizable, said Ebner. It is often veiled in memes, allusions and insider jokes.

“Alt-right extremist networks are very good at circumventing existing legislation (policies and practices on platforms). They use irony and sarcasm to spread hateful ideas and hide them behind memes,” said Ebner.

“The alt-right has tapped into this in a way that no jihadist movement has.”

In contrast, ISIS content is easier to identify because it tends to repeat words and phrases that don’t appear in other content, said Pedro Domingos, a professor of computer science at the University of Washington and author of The Master Algorithm.

The formulaic way that ISIS disseminated propaganda – using hashtags in different languages, coded speech, watermarked videos and iconography – also allowed review teams to easily create hash copies for blacklist.

Domingos said that digital platforms don’t yet have the technical ability to combat white supremacist accounts in the same way they’ve tackled ISIS.

“The main problem is that the (far-right extremist) content is too variable and multifarious to be reliably distinguished from acceptable content by the filtering algorithms that tech companies use, even state-of-the-art ones,” Domingos said in an email to CNN.

As part of the current debate, technology experts are calling for platforms like Facebook to make the inner workings of their algorithms transparent, giving users more clarity around the content they see and why they see it. Without that transparency, it’s easy for users to think that algorithms are optimized for truth, rather than providing a pipeline of related content to keep people on platforms.

But algorithmic transparency could also backfire, said Domingos, making it easier for white supremacists to game them. 

It’s not just a technological problem

Protesters in London rally against Islamophobia, anti-Semitism and racism after the New Zealand attack.
Tayfun Salci/Anadolu Agency/Getty Images
Protesters in London rally against Islamophobia, anti-Semitism and racism after the New Zealand attack.

Even if a more advanced algorithmic capability existed – one that would stop the spread of nuanced hate speech – tech companies would ultimately need to be committed to deploying it. This requires them to accept more responsibility as publishers: moderating and removing content long before it leads to violence or bullying.

It wasn’t until a 2017 white supremacist rally turned deadly in Charlottesville, Virginia, that most major social media companies, domain registrars and payment platforms looked at addressing domestic extremism.

Several white nationalist accounts were banned on various platforms, including one of the most influential neo-Nazi sites online, the Daily Stormer.

Asked how Facebook is tackling far-right extremist content shared on its platform, a spokesperson told CNN that it is investing in security teams and technical tools to “proactively detect hate speech” and is working with partners to better understand hate organizations as they evolve.

“We ban these organizations and individuals from our platforms and also remove all praise and support when we become aware of it,” the spokesperson said.

A YouTube spokesperson told CNN that it had “heavily invested in human review teams and smart technology” in order to “quickly detect, review, and remove” extremist content. “Hate speech and content that promotes violence have no place on YouTube,” the spokesperson said.

A spokesperson for Twitter said that the company’s policy against hate speech “prohibits behavior that targets individuals based on protected categories including race, ethnicity, national origin or religious affiliation.”

“Where we identify content that breaks these rules, we take aggressive enforcement action,” the spokesperson told CNN.

Should social networks police political rhetoric?

Joan Donovan, director of the Technology and Social Change Research Project at Harvard University’s Shorenstein Center, points out another reason why US tech companies have been slower to react to white supremacist content than to ISIS-related content: Politicians around the world are using the same rhetoric.

“The culture of Silicon Valley itself isn’t designed to look inward,” Donovan told CNN. “So when we’re talking about white supremacy, people on platforms remark that they don’t know how to spot it because it’s sometimes ambiguous, sometimes coded, or sometimes reflects politicians’ language.”

The overlap between individuals who promote white supremacist ideology online and populist political parties around the globe poses a problem for platforms, in a way that combating ISIS did not. 

In 2015, during the US presidential campaign, Donald Trump himself was accused of hate speech after he posted a statement from his website on Facebook calling for a blanket Muslim travel ban, which was later removed. He has also been criticized for posting a racially charged campaign ad and for seeming to allude to anti-Semitic conspiracy theories in his tweets. Trump has said he is not a racist and has condemned neo-Nazis. 

White nationalist Twitter accounts reference Trump and Trump-related hashtags like #makeamericagreatagain more than almost any other topic except for #whitegenocide, according to the same George Washington University study, which underscores the obstacles in routing out offending accounts.

Tackling such polarizing rhetoric by influential figures needs to be part of any long-term digital strategy, said Ebner.

“One of the most alarming aspects of all of this is the mainstreaming of dehumanizing language and black and white narratives by politicians like Trump, (Italian Deputy Prime Minister Matteo) Salvini, or the Australian senator,” said Ebner.

Shortly after the New Zealand mass shooting, Australian Senator Fraser Anning blamed the violence on Muslim immigration in a post on Twitter. 

Asked whether attacks like the one in New Zealand could happen in Italy because of his aggressive, anti-immigrant rhetoric, Salvini said: “The only extremism that deserves attention is the Islamic one.” 

According to Ebner, to confront divisive rhetoric like this we must focus on building strong civil society efforts and promote education around digital citizenship. “We don’t want to clamp down on freedom of speech, but we need to challenge this kind of rhetoric … and that’s a longer term investment.”