Three governments agree on something the AI industry doesn’t want to hear
The Attachment Economy
SubscribeSign in
Three governments agree on something the AI industry doesn’t want to hear<br>The governments of China, California and New York agree on three harmful aspects of companion AI chatbots.
Mike Elgan<br>Jul 15, 2026
Share
If you believe the press coming out of China, thousands or millions of young people there are suffering from the loss of their AI companions and have spent the week “saying goodbye” to them.<br>(AI companion chatbots and features are designed to simulate social, emotional, and personal relationships with personalized, empathetic, and continuous conversation.)<br>Today is the day when China’s new laws took effect. The laws force free, user-built companions that exist inside general-purpose apps as a feature to shut down.<br>ByteDance’s Doubao chatbot, which has over 300 million monthly users, killed its custom-persona feature today. Alibaba’s Qwen and Tencent’s Yuanbao pulled their human-like agents within the past two months.<br>China still allows companion AI chatbots, as long as they exist within an app dedicated to that function.<br>The logic is that free, user-built companions inside mass-market general-purpose apps are impossible to moderate at scale. They expose everyone, including minors, to the temptation to create chatbots with personalities, which could drift into an addictive, harmful fake “relationship.” A dedicated standalone companion app is a bounded, opt-in environment where age-gating, usage caps, and anti-addiction safeguards can be built as core features.<br>For anyone concerned about the Attachment Economy, which is an extractive business model that uses AI to lure people into forming emotional attachments to chatbots, AI tools and robots, this may sound like a good thing that we should copy.<br>In fact, the US beat China with comparable restrictions — at least two states did. Sort of.<br>Rather than a total ban on AI companions, California (starting January 1, 2026) requires companion apps to maintain protocols preventing suicide and self-harm content and to refer users to crisis services for all users, and to block sexually explicit content specifically for minors.<br>New York’s law, which went into effect November 5, requires companion-AI operators to detect and address expressions of suicidal ideation or self-harm for all users, not just minors.<br>Both New York and California have enacted laws that specifically require companion AI chatbots to disclose to users that they are interacting with a machine and not a human.<br>While the California and New York laws are based mainly on suicide prevention in particular, and exacerbating mental health issues in general, the Chinese laws that went into effect today are motivated by preventing addiction among minors and adults.<br>Chinese regulators call AI companion chatbots “human-like interaction services.” China lumps human-like AI companion bots in with gaming, social networking and other “addictive” digital services, which the Chinese Communist Party deems socially corrosive.<br>The Chinese government is also concerned about declining birthrates. China’s AI Safety Governance Framework 2.0, released in September 2025, explicitly flags “addiction and dependence on anthropomorphic interaction AI products” warns that such AI affects “childbirth, and education—thus challenging the traditional social order.”<br>Another telling detail: China’s law explicitly encourages companion AI for childcare support. That China’s policy bans it for relationships deemed alternatives to human relationships that could lead to childbirth, it encourages it to ease childcare.<br>The law also encourages humanlike relationship AI for elderly care. Declining birthrates have left China with an aging population and not enough caregivers.<br>In other words, the US state restrictions are mainly about protecting individuals, whereas the Chinese restrictions are mostly about protecting the state. The US laws respect the rights of adults to choose. The Chinese laws overrule adult choice.<br>What these laws have in common
Lawmakers on both sides of the Pacific have confronted truths that the AI companion industry and their customers/fans would rather they didn’t confront. These truths are:<br>Some people come to believe that AI chatbots are thinking, feeling “people.” All three laws force AI companion chatbots to disclose to users that the chatbot “personality” is a machine. Mere math. Not a person. Lawmakers added this requirement because it’s clear that part of the problem is user delusion.
Companion AI chatbots can make mental health worse. All three laws specifically call out suicide as a risk. That means people with existing mental health issues can be encouraged to take their own lives by chatbots, who are designed to be sycophantic and agreeable.
Companion AI chatbots can be bad for children. Two of the three regimes (California’s and China’s) impose special provisions to protect minors,...