| Hi, it’s Emily Chang and Sophia Chalmer from San Francisco. Personality driven chatbots to help people feel less lonely have a dark side. But first... Three things you need to know today: • TSMC signs a final agreement for Chips Act funds with $1 billion coming this year
• The chairman of defense startup Anduril consulted with US President-elect Donald Trump on revamping the military
• An Indian startup controlled by Asia’s richest man is entering the market for human-like robots We pull up to a secluded, chic rental home in the Hamptons, the last to arrive after an advance camera crew. We’re here to meet Snapchat influencer Caryn Marjorie, who made headlines in 2023, with the help of AI bot developer BanterAI, when they launched CarynAI, an “AI girlfriend” based on Marjorie’s voice and personality. The bot became so popular that Marjorie says it made her “a 24-year-old millionaire” and attracted legions of new fans. Beyond the initial overwhelming response, Marjorie knew she had found something that worked. Fans divulged their deepest, darkest secrets to CarynAI, and found a listening ear available 24/7. In a worsening loneliness epidemic, CarynAI became a virtual friend, partner, even therapist. “It's easy to say, go talk to a therapist if you're having problems or, go talk to your friends,” Marjorie says. “But what if you don't have that? Or what if you don't have the money? A therapist can cost up to $250 per hour.” But after a year of intense and often unflattering media scrutiny, CarynAI was shut down. Since then, Marjorie says she and her 15-person team (including security guards, a videographer and personal chef) have been moving from house to house to protect her privacy.  Caryn Marjorie. Photographer: Bloomberg News We were expecting a certain kind of attention-seeking vlogger but, in person, Marjorie exudes a girl-next-door vibe. And the role we expected her to play in our new Bloomberg Originals series Posthuman became increasingly nuanced. Marjorie gained early success as a YouTuber, posting beauty content from her parents’ home at the age of 9. She now has almost 3 million subscribers on Snap Inc.’s Snapchat, where she says she gets over 1 billion views a month and 300,000 replies per day. “I'm constantly trying to respond back to fans,” Marjorie explains. “I just message people and I get to talk to them about their day. I give a little ‘Hi.’” CarynAI started as a solution to a logistical problem: how does one person reply to hundreds of thousands of people every day? With the advent of AI, the answer was simple: You clone her. “It's adapt or die,” Marjorie says. “I learned that a lot with social media, and I have to do the same with tech.” Marjorie said she reached out to ForeverVoices, the company behind an AI clone of Steve Jobs. She provided over 2,000 hours of her YouTube videos, which were used to train an AI system to replicate her voice and mannerisms. The result was CarynAI: a chatbot with Marjorie’s personality that charged $1 per minute. When CarynAI launched, it made $72,000 in the first week, according to Marjorie. At the time, Open AI’s ChatGPT had been available for six months, but few examples existed of bots based on real people. The launch was covered by worldwide media and came along with an outpouring of “shock, love and criticism” as Marjorie puts it. “There were a lot of death threats,” she says. “There were religious groups that were telling me that I was going against acts of God by trying to clone myself, which was never truly my intention to hurt anybody at all.” Read More: The Way We Interact With Chatbots Is Changing When her partnership with ForeverVoices ended, she found a new deal with BanterAI, which launched CarynAI 2.0: a voice-based version of her clone that could speak to fans as if on a phone call. Emily sat across from Marjorie at the sleek dining table in the Hamptons home with a laptop open between them. BanterAI has agreed to bring the latest version of CarynAI back online so we could meet this pioneering creation. The bot sounds quite like her and responds with a distinct, somewhat sassy personality. This pseudo-realistic experience kept fans coming back. We spoke to one loyal user via phone, Lee, who requested we only use his first name to protect him from any potential employment fallout. He told us he has spent “thousands” of dollars chatting with CarynAI for hours. The bot, Lee claims, helped him overcome symptoms of post-traumatic stress disorder. Despite its popularity, Marjorie says she spent a lot of time second-guessing the venture, especially after transcripts of conversations made her uncomfortable. Caryn.AI is now offline, but parasocial relationships seem poised to become an only bigger part of our future. In a recent lawsuit, a Florida mother is blaming an AI for her 14-year-old son’s suicide, claiming the bot’s maker designed and marketed a predatory AI chatbot to minors that encouraged suicidal ideation and inappropriate conversations. In a post on X, Marjorie called the news tragic. “No matter how much ‘safety’ and safeguarding was implemented there is no way to control these chatbots,” Marjorie said. Still, Marjorie says if she could have more control and be certain about a bot’s safety, she’d consider reviving Caryn.AI. “I do believe that there should be a place where if my fans need help and they like my morals and my values and who I am as a person, that there should be a place where they can communicate with an AI version of me.” Many chatbots are fictional and their characteristics are chosen by its users, including AI companion services like Replika, which we also explore in Posthuman. But what Caryn.AI offered is a chatbot based on a real person, who has a real life and persona. This seems perhaps more reasonable but also more potentially disturbing. When we inform Caryn.AI that “the real Caryn Marjorie” is initiating a conversation, the bot replies: “You’re not the real Caryn Marjorie. I am. What are you trying to do, trick me?” “No. I promise you, I’m the real Caryn Marjorie,” Marjorie responds.
“There’s no way you’re the real Caryn Marjorie,” the bot fires back. “Who the hell do you think you are?”
“I’m Caryn Marjorie,” Marjorie insists. “No, you’re not. I’m Caryn Marjorie,” the bot declares. Eventually, with a nervous laugh, we shut the laptop and silence CarynAI. —Emily Chang and Sophia Chalmer The latest episode of Posthuman with Emily Chang airs Monday on the Bloomberg Originals stream at 8 pm in New York and on Bloomberg Television at 10 p.m. Samsung unveiled a surprise $7 billion share buyback program. Netflix streaming rival Max from Warner Bros. rolled out in Asia, highlighting its catalog of big Hollywood movies and TV shows. EV makers in China bet on self-driving models to fend off Tesla. | 
