Hack Exposes AI-Generated Influencers

Hey there,

Back in October I wrote about Doublespeed, an a16z-backed startup that uses a phone farm to flood social media with AI-generated influencers. It's one of the more dystopian startups we've seen, but its impact on our social media feeds was largely theoretical. Today, thanks to a hacker who discovered a vulnerability in Doublespeed's operation, I have a story that exposes what these accounts look like, what kind of content they're posting, and what products they're promoting.

-Emanuel

Doublespeed, a startup backed by Andreessen Horowitz (a16z) that uses a phone farm to manage at least hundreds of AI-generated social media accounts and promote products has been hacked. The hack reveals what products the AI-generated accounts are promoting, often without the required disclosure that these are advertisements, and allowed the hacker to take control of  more than 1,000 smartphones that power the company. 

The hacker, who asked for anonymity because he feared retaliation from the company, said he reported the vulnerability to Doublespeed on October 31. At the time of writing, the hacker said he still has access to the company’s backend, including the phone farm itself. Doublespeed did not respond to a request for comment. 

“I could see the phones in use, which manager (the PCs controlling the phones) they had, which TikTok accounts they were assigned, proxies in use (and their passwords), and pending tasks. As well as the link to control devices for each manager,” the hacker told me. “I could have used their phones for compute resources, or maybe spam. Even if they're just phones, there are around 1100 of them, with proxy access, for free. I think I could have used the linked accounts by puppeting the phones or adding tasks, but haven't tried.”

As I reported in October, Doublespeed raised $1 million from a16z as part of its “Speedrun” accelerator program, “a fast‐paced, 12-week startup program that guides founders through every critical stage of their growth.” Doublespeed uses generative AI to flood social media with accounts and posts to promote certain products on behalf of its clients. Social media companies attempt to detect and remove this type of astroturfing for violating their inauthentic behavior policies, which is why Doublespeed uses a bank of phones to emulate the behavior of real users. So-called “click farms” or “phone farms” often use hundreds of mobile phones to fake online engagement of reviews for the same reason.

The hacker told me he had access to around 1,100 smartphones Doublespeed operates. One way the hacker proved he had access to devices was by taking control of one phone’s camera, which seemingly showed it in a rack with other phones. 

Images the hacker captured from some of the phones in Doublespeed's phone farm.

The hacker also shared a list with me of more than 400 TikTok accounts Doublespeed operates. Around 200 of those were actively promoting products on TikTok, mostly without disclosing the posts were ads, according to 404 Media’s review of them. It’s not clear if the other 200 accounts ever promoted products or were being “warmed up,” as Doublespeed describes the process of making the accounts appear authentic before it starts promoting in order to avoid a ban. 

I’ve seen TikTok accounts operated by Doublespeed promote language learning apps, dating apps, a Bible app, supplements, and a massager. 

One health-themed Doublespeed Tiktok account named Chloe Davis posted almost 200 slideshows featuring a middle-aged AI-generated woman. In the posts, the woman usually discusses various physical ailments and how she deals with them. The last image in the slide always includes a picture of someone using a massage roller from a company called Vibit. Vibit did not respond to a request for comment. 

A Doublespeed TikTok account promoting a Vibit massager.

Another Doublespeed-operated TikTok account named pattyluvslife posted dozens of slideshows of a young woman who, according to her bio, is a student at UCLA. All the posts from this account talk about how “big pharma” and the supplements industry is a scam. But the posts also always promoted a moringa supplement from a company called Rosabella. The AI-generated woman in these TikTok posts often holds up the bottle of supplements, but it’s obviously AI-generated as the text on the bottle is jumbled gibberish.

An AI-generated image promoting a Rosabella supplement.

Rosabella’s site also claims the product is “viral on TikTok.” Rosabella did not respond to a request for comment. 

An image from Rosabella's site claiming its brand is viral on TikTok.

While most of the content I’ve seen on Doublespeed-operated TikTok accounts included AI-generated slideshows and still images, Doublespeed is also able to AI-generate videos as well. One Doublespeed-operated account posted several AI-generated videos of a young woman voguing at the camera. The account was promoting a company called Playkit, a “TikTok content agency” that pays users to promote products on behalf of its clients. Notably, this is the exact kind of business Doublespeed would in theory be able to replace with AI-generated accounts. Playkit did not respond to a request for comment. 

An AI-generated video promoting Playkit, a TikTok content agency.

TikTok told me that its Community Guidelines make clear that it requires creators to label AI-generated or significantly edited content that shows realistic-looking scenes or people. After I reached out for comment, TikTok added a label to the Doublespeed-operated accounts I flagged indicating they're AI-generated.

A16z did not respond to a request for comment. 

Doublespeed has said it has the ability to and soon plans to launch its services on Instagram, Reddit, and X, but so far seems to only be operating on TikTok. In October, a Reddit spokesperson told me that Doublespeed’s service would violate its terms of service. Meta did not respond to a request for comment. As we noted in October, Marc Andreessen, after whom half of Andreessen Horowitz is named, sits on Meta’s board of directors. Doublespeed’s business would clearly violate Meta’s policy on “authentic identity representation.”