AI-generated spam is starting to flood social media. Here's why: NPR

Casey Morris, an attorney in Northern Virginia, recently started checking Facebook again after a long hiatus. Among the posts from friends and family, she noticed a strange trend.

“The caption will say, 'Close your eyes 70 percent and see the magic.' And without looking at it at all, it's obviously like a picture of Jesus, but it's going to be made of vegetables and a tractor and a little girl that's kind of distorted,” he said.

That wasn't the only oddity in Morris' feed. Similar images are repeated with identical captions. So are different, more emotionally exploitative posts that depict disabled mothers and children in mud or smiling amputated children, with captions asking for birthday wishes.

“It made Facebook a very strange place for me,” Morris said.

Between their subject matter, stylistic cues and odd mistakes, it soon became clear to Morris that the images were fakes—the products of artificial intelligence.

They are not being posted by people she knows or follows. Instead, Facebook is suggesting she might be interested in them – and it seems they're really popular.

“They're getting thousands of reactions and thousands of comments. [from] People who seem to be real, so wishing them a happy birthday or saying something religious in the comments,” she said.

Morris isn't the only Facebook user whose feed has started filling up with AI-generated spam. Reporters at the tech website 404 Media tracked an increase in apparently AI-generated posts on Facebook, which is owned by Meta, in recent months. Such AI-generated images are also starting to appear on other social media sites, including Threads, which is also owned by Meta and LinkedIn.

Spam and scams

On Facebook, in many cases, it appears that the platform's own algorithm is boosting AI posts.

When researchers at Georgetown and Stanford universities investigated more than 100 Facebook pages that routinely post AI content — sometimes dozens of times a day — they found that many engaged in scams and spam.

“We've seen AI-generated images of everything you can imagine, from log cabins to grandmothers with birthday cakes,” said co-author Josh Goldstein, a research fellow at Georgetown University. to wonderful children's paintings that just can't be real.” of the preprint study, which has not yet been peer-reviewed.

Goldstein and her co-authors also found that Facebook is actively recommending some of this AI content into users' feeds — potentially creating a cycle where posts get more engagement, Hence they are recommended to even more users. Some of the individual posts on the pages they analyzed garnered hundreds of thousands and even millions of interactions.

“These weren't sporadic images here or there that only a few people were interacting with. They were really getting a ton of traction,” Goldstein said.

Their analysis revealed that some of these pages are classic spam, posting links to websites where they can collect ad revenue. Others are scammers, advertising AI-generated products that don't actually exist.

But many pages don't have a clear financial motivation, Goldstein said. It seems that they are simply gathering an audience for unknown purposes.

“It could be that these were malicious pages that were trying to build an audience and then try to sell products or link to websites full of ads,” Goldstein said. turn the themes of the into something completely political.” “But I'm more likely, there were many creators of these pages who realized it was a useful tactic to get audience engagement.”

Clickbait has always been on social media. But over the past few years, Facebook has doubled the number of recommended posts for its users, as it tries to keep up with the changes in social media offered by TikTok. On a recent earnings call, Meta CEO Mark Zuckerberg told analysts that recommended posts now account for about 30 percent of users' feeds.

A shift from realistic images to the paranormal

At the same time, AI-generated content is now easier than ever for anyone to create. Together, these dynamics are creating the recipe for weird renderings of Jesus, disturbing birthday posts and impossible architecture and crafts to go viral.

“It's copying, like, all the elements that make something go viral. But they're putting out the weirdest images I've ever seen,” said Brian Penny, a freelancer. Lance writer who is almost tracking AI on Facebook. two years. He is part of a group dedicated to sharing and debunking AI images.

Penny has seen a shift from images that have some basis in reality — like the AI-generated photo of Pope Francis in a puffy coat that went viral last year — to something far more unusual.

“We work to reduce the spread of content that is spam or sensational because we want users to have a good experience, which is why we offer them control over what they see in their feed. are,” a Meta spokesperson told NPR in a statement.

The company plans to soon begin labeling AI-generated content created with some industry-leading tools. Last week, TikTok began adding similar labels to some AI-generated posts on its platform.

Meanwhile, the rise of AI spam is turning many off.

Katrina McVay, who lives in Grand Rapids, Mich., says she has had to dissuade her mother from buying woodwork and other home decor she sees on Facebook — which are clearly fake.

“She'd be like, 'Wouldn't that be great for your daughter?'” McVay said. “And I'm like, 'It's not real though.'”

Some Facebook users are considering leaving the platform altogether due to frustration with the spammy AI recommended images.

“Am I going to see all this and see that my cousin has just gone to the Sahara desert?” asked Los Angeles furniture designer Boris Razonka. “It just wasn't worth it to me anymore.”

In addition to finding AI spam on Facebook annoying, many people who spoke with NPR are worried about the high stakes of artificial images appearing everywhere.

“It just reinforces people's disbelief and … makes it harder to see what's what,” said Hobbyford, a puppeteer from North Carolina who runs Facebook groups devoted to science. I've seen images of AI pop up, and claim they reflect new discoveries. .

“And I think that's dangerous in our world right now,” he said.