Before two 18-year-old men killed 31 people in separate shootings over the past two weeks, they took to a variety of social media apps to share disturbing private messages.
The two men — one killed by police in Uvalde, Texas, and the other charged in the Buffalo shooting — used a combination of disappearing video app Snapchat, Instagram direct messages, Discord chat app and Yubo social app to meet people and share their violent plans. with knowledge. In Buffalo, the suspect also used the Twitch video streaming platform to publicize his deadly attack.
These apps – many of which have been embraced by Gen Z as teens and young people seek out more private corners of the internet – are ill-equipped to police this content. They are fundamentally designed to keep communications private, presenting different challenges than Facebook, YouTube and Twitter, where screeds and violent videos have been algorithmically amplified for millions of viewers.
The way this generation uses social media more generally could render years of work to spot and identify public signs of violence to come obsolete, social media experts warn.
“There’s this shift to more private spaces, more ephemeral content,” said Evelyn Douek, senior fellow at Columbia University’s Knight First Amendment Institute. “The content moderation tools that the platforms have built that we’ve been discussing are sort of dated or about the last war.”
Texas Governor Greg Abbott (right) said Wednesday that the Texas shooter, whom authorities identified as Salvador Rolando Ramos, wrote on social media that “I’m going to shoot my grandma” and “I’m going to shoot a primary school”. shortly before the attack. Facebook confirmed the messages were sent privately but declined to say which of its social networks were used.
Stephen Garcia, who considered himself Ramos’ best friend in eighth grade, previously told the Washington Post that Ramos used the Yubo app, a platform where users can swipe on each other’s profile, a la from Tinder, or hang out in live and virtual broadcast rooms. “meet” other users by playing games and chatting.
Yubo spokeswoman Amy Williams said in an email that the company was unable to release any information outside of direct requests from law enforcement, but the company was investigating an account. who was banned from his platform.
“We are deeply saddened by this indescribable loss and are cooperating fully with law enforcement in their investigation,” she said.
In the case of the Buffalo shooting, the alleged shooter, Payton Gendron, sent an invitation to an online chat room on the Discord instant messaging platform which was accepted by 15 users, allowing them to scroll months of voluminous writing and racist screeds from Gendron, The Washington Post reported. Users accessing the room could also view an online video stream, where footage of the Buffalo attack was shown. This attack was also broadcast on Twitch, a live streaming service popular among video game users.
Discord and Twitch did not immediately respond to requests for comment.
Twitch was able to remove the stream within two minutes after the shooter began shooting, said Angela Hession, the company’s head of trust and safety. The site has a continuous escalation system in place to deal with urgent reports, such as live-streamed violence.
Discord has since said the posts were only viewable by the suspect until he shared them with others on the day of the attack.
In the wake of high-profile mass shootings in recent years, communities, school districts and tech companies have made significant investments in security systems aimed at eliminating violent screeds in hopes of preventing a forward attack. that it does not happen. The Uvalde Consolidated Independent School District previously used an artificial intelligence-backed program to scan social media posts for potential threats years before the attack, though it’s unclear whether it was in use at the time of the shooting.
But these tools are ill-equipped to deal with the growing popularity of live video streaming and private or dying messages, which are increasingly being used by young adults and teens. These messages are then closed to outsiders, who might be able to spot the warning signs that a troubled individual might be about to inflict harm on themselves and others.
These newer social networks also have far less of a history of violent content, and they’re less likely to have policies and staff in place to respond to incitement to violence on their services, experts said.
“For smaller sites or newer sites, they’re going through the same times as larger services like Facebook and YouTube in 2015 and 2016,” said Emma Llansó, director of the Free Expression Project at the Center for Democracy and Technology. , a non-profit organization. backed by big tech companies.
Shooters’ adoption of these upstart apps reflects a larger generational shift in social media usage. Gen Z, teens and young adults born after 1996 have flocked to apps that emphasize private messaging, live streaming, or allow their users to post content that disappears from public profiles after a certain amount of time. time.
They have largely eschewed older social media apps like Facebook, Twitter, and YouTube, which grew in popularity by providing public, open spaces to communicate with the world.
The new apps’ role in the shooting caught the attention of state attorneys general in New York and New Jersey, who, following the Buffalo shooting, launched investigations into Discord and Twitch.
“Time and time again, we have seen the real-world devastation caused by these dangerous and hateful platforms,” New York Attorney General Letitia James (D) said in a statement announcing the investigation after the shooting. from Buffalo. “We are doing everything in our power to shine the spotlight on this alarming behavior and take action to ensure it never happens again.”
Social media has played a prominent role in many mass shootings, and there have been high-profile cases of gunmen posting their plans online for all to see and not being arrested.
Republican lawmakers, who have long resisted moves to expand background checks or limit access to guns, aimed to shine a light on social media’s role in the Texas shooting on Thursday. “The common theme running through nearly all of these mass shootings is the social alienation of sick young men, often fueled by social media,” tweeted Sen. Dan Sullivan, R-Alaska. He did not mention gun access in the post.
Tech industry officials pushed back, warning that such tweets could distract from broader political gun control issues.
“Some people will try to get Facebook to talk about it not being about guns,” tweeted Brian Fishman, former director of counterterrorism, dangerous organizations and content policy at Facebook. . “Don’t let them.”
The tech giants have also been caught in a years-long power struggle as they seek to balance privacy with the police content of their sites and the demands of law enforcement.
Facebook and other companies have opted for strong encryption, a technology that scrambles the content of a message so that only the sender and recipient can see it. WhatsApp and Apple iMessage use it, along with messaging apps like Signal. And Facebook has said it wants to introduce encrypted messaging as the default setting on Instagram and Facebook Messenger, prompting backlash from politicians and law enforcement officials who have warned that the adoption generalization of this technology could leave them in the dark and make them more difficult. for them to investigate the violence.
Some big tech companies scan messages for harmful content, such as child sexual abuse or spam. But experts warn that monitoring more private communication spaces is a delicate balance.
“There are so many incredibly legitimate reasons why people want to use private communications,” Llansó said. “It’s not something that should be sacrificed for everyone because some people want to use private communications for excruciating reasons.”
Social media users tend to be younger, but the generational gaps among the user base among private messaging apps like Snapchat are larger than they are for more traditional public sites like Facebook.
When Snapchat users send private messages to each other, they disappear once the recipient has read them. The app also pioneered the concept of “stories” – public posts that only last for a day – which was later copied by Facebook.
Snap said on Wednesday it had suspended an account that may have been connected to Ramos and was also working with law enforcement.
Meanwhile, Facebook is struggling to keep pace with the rapidly changing social habits of teenage users.
Facebook’s own internal research indicates that young adults are “less engaged” than older adults, posing a significant risk to the company’s business, according to a trove of internal company documents known as from Facebook Files. The company’s research found that young adults prefer sharing life updates over text messages, rather than broadcasting to a wide range of Facebook friends. The researchers suggested that the company respond by relying on groups and more private forms of sharing.
“It will always be a game of cat and mouse,” Douek said. “These are just kind of unsolvable problems. But that doesn’t mean we can’t improve or that we should let the platforms off the hook.
Rachel Lerman of The Washington Post contributed to this report.