Inside the Booming 'AI Pimping' Industry

Hey there,

We’ve spent months looking through more than 1,000 AI-generated Instagram Influencers, reading multiple guides from people who teach others on how to create them, and monitoring Discord channels where they share information to report this story. We learned what many of you who have followed our reporting over the last year already know: Instagram doesn’t care that its platform is being overrun by AI-generated content, and is probably benefiting from it because it’s inflating its numbers. What sets this story apart from other forms of AI slop is that the people who make these AI-generated influencers are blatantly stealing from real humans who are increasingly struggling to promote their work on Instagram and make a living.

-Emanuel and Jason

This article was produced with support from WIRED.

Instagram is flooded with hundreds of AI-generated influencers who are stealing videos from real models and adult content creators, giving them AI-generated faces, and monetizing their bodies with links to dating sites, Patreon, OnlyFans competitors, and various AI apps. 

The practice, first reported by 404 Media in April, has since exploded in popularity, showing Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living. 

According to our review of more than 1,000 AI-generated Instagram accounts, Discord channels where the people who make this content share tips and discuss strategy, and several guides that explain how to make money by “AI pimping,” it is now trivially easy to make these accounts and monetize them using an assortment of off-the-shelf AI tools and apps. Some of these apps are hosted on the Apple App and Google Play Stores. Our investigation shows that what was once a niche problem on the platform has industrialized in scale, and shows what social media may become in the near future: a space where AI-generated content eclipses that of humans.

This segment is a paid ad. If you’re interested in advertising, let's talk.

The Electronic Frontier Foundation is the leading nonprofit organization defending civil liberties in the digital world. Member-supported since 1990, EFF champions user privacy, free expression, and innovation. EFF fights to ensure that technology supports freedom, justice, and innovation for all people of the world. Show your support with shirts, hats, totes, socks, cards, and more. Exclusive for 404 Media readers, we’re offering 20% off all items in our shop using coupon code 404MEDIA at checkout.

Elaina St James, an adult content creator who promotes her work on Instagram, said she and other adult content creators are now directly competing with these AI rip-off accounts, many of which use photographs and videos stolen from adult content creators and Instagram models. She said that while there may be other changes to Instagram’s algorithm that could have contributed to this, since the explosion of AI-generated influencer accounts on Instagram her “reach went down tremendously,” from an average of one to 5 million views a month to not cracking a million in the last 10 months, and sometimes coming in under 500,000 views. 

“This is probably one of the reasons my views are going down,” St James told us in an interview. “It's because I'm competing with something that's unnatural.”

Alexios Mantzarlis, the director of the security, trust, and safety initiative at Cornell Tech and formerly principal of Trust & Safety Intelligence at Google, compiled a list of around 900 accounts 404 Media reviewed in its investigation. Mantzarlis, who stumbled on one of these accounts while casually using Instagram, said he started researching the AI-generated influencer accounts because it might show us where AI-generated content is taking social media and the internet more broadly, where he sees a “a rising blended unreality.” Mantzarlis believes he could have easily found 900 more accounts, and that the only reason he didn’t get more is that Instagram restricted the account he used to scrape the platform. 

“It felt like a possible sign of what social media is going to look like in five years,” Mantzarlis said in an interview. “Because this may be coming to other parts of the internet, not just the attractive people niche on Instagram. This is probably a sign that it's going to be pretty bad.”

THE ACCOUNTS

Out of more than 1,000 AI-generated Instagram influencer accounts we reviewed, 100 included at least some deepfake content which took existing videos, usually from models and adult entertainment performers, and replaced their face with an AI-generated face to make those videos seem like new, original content consistent with the other AI-generated images and videos shared by the AI-generated influencer. The other 900 accounts shared images that in some cases were trained on real photographs and in some cases made to look like celebrities, but were entirely AI-generated, not edited photographs or videos.

Out of those 100 accounts that shared deepfake or face swapped videos, 60 self-identify as being AI-generated, stating in their bios that they are a “Virtual Model & Influencer,” or that “All photos crafted with AI and Apps.” The other 40 do not include any disclaimer stating that they are AI-generated.

One of the bigger accounts in the latter category is “Chloe Johnson,” who has a verified account on Instagram and 171,000 followers. This account was deleted by Meta within the last few weeks.

Using Google Lens, which finds similar images on the web, a 404 Media reader was able to find the original source videos that nine of Johnson’s Instagram posts were based on. That showed the person running Johnson’s account is swapping the face of the AI influencer onto the bodies of real women, including the models Tana Rain, Skyler Simpson, and Kyla Yesenosky. Other videos were stolen from TikTok and Instagram users who are not famous, and who have a small number of followers (Ulia Nova, Annabella Sinclair). We’ve also seen face swapping accounts source their videos from swimwear runway shows and Getty’s stock image and video site iStock. 

Left: A still image from a video of model Kara Del Toro at the Poema Swim 2019 fashion show. Right: A faceswapped version of that video posted by the AI-generated Instagram influencer Aubrey Fosterr.

Johnson’s Instagram links to an account on Fanvue, a site very much like OnlyFans where fans can pay to access a creator’s account. Here, Johnson says people can buy “pay-per view full nudes and fucking videos.” Many other AI influencer accounts we reviewed also monetize their content on Fanvue. Johnson’s Instagram also links to another website where nude photos and hardcore porn videos can be bought individually for between $3 and $22.

Like other content hustles, the industry is full of people who are trying to get rich in part by following guides and instructional courses being sold by people who have run successful AI influencers. 404 Media purchased two different guides—one PDF instruction manual called “Instagram Mastery” by an AI influencer agency called “Digital Divas,” and one called “AI Influencer Accelerator” made by someone who calls themselves Professor EP, who says they operate the Emily Pellegrini AI influencer Instagram account, which has 253,000 followers.