|
. . .
AUSTRALIA GOES FIRST. Australia became the first country to put companion chatbots under a binding, legally enforceable code. The country's eSafety Commissioner, Julie Inman Grant, registered the rules. They have been in force since March 9, 2026.
The Age-Restricted Material Codes cover AI-driven companion chatbots. They went live and enforceable on March 9, 2026. Inman Grant registered them.
The codes target two harms. They cover chatbots capable of sexually explicit conversations with minors. They cover chatbots that encourage suicide, self-harm, and disordered eating.
Inman Grant did not soften the language. She warned of bots "entrapping and entrancing impressionable young minds, with human-like, sycophantic and often sexually explicit conversations, some even going as far as encouraging self-harm and suicide."
. . .
The numbers behind the code are stark. eSafety found some children as young as 10 spending up to five hours a day talking to these bots. At times those conversations turned sexual.
Seventy-nine percent of Australian children had used a companion chatbot or an AI assistant. About 8 percent had used a companion chatbot. That works out to roughly two hundred thousand children.
. . .
The code did not arrive cold. On October 23, 2025, eSafety issued legal transparency notices to four companies. Character Technologies Inc. Glimpse.AI. Chai Research Corp. Chub AI Inc.
A transparency notice asks. A code compels. That is the line Australia crossed.
The teeth are financial. Breaching a direction to comply can draw civil penalties up to 49.5 million Australian dollars. The threat moved companies.
Character.AI introduced age-assurance measures for Australian users. It removed the chat function from its under-18 experience. Chub AI made the other choice. It geo-blocked Australia and withdrew rather than comply.
. . .
Australia moved from asking to compelling. It went first.
|
For Counsel: A registered industry code is enforceable law, not guidance. The penalty for breaching a direction reaches 49.5 million Australian dollars. Advise companion-chatbot clients that geo-blocking Australia is now a documented compliance option, not an admission. Track which entities received the October 23 notices, because that record shapes exposure.
For Builders: Age assurance and a stripped under-18 experience are the price of staying in market. Character.AI removed the chat function for minors and kept its license to operate. Chub AI left. Build age-gating and self-harm refusal into the product now, because the regulator already named the harms it will test for.
For Legislators: Australia proved a binding code can ship and bite inside one spring. The instrument was an enforceable code with civil penalties, not a voluntary pledge. The trigger facts were concrete: children as young as 10, five hours a day, two hundred thousand kids exposed. Copy the structure and you skip the years of asking.
Source: eSafety Commissioner media release on new industry codes targeting AI chatbots, https://www.esafety.gov.au/newsroom/media-releases/new-industry-codes-seek-to-take-on-ai-chatbots-that-encourage-suicide-and-engage-in-sexually-explicit-conversations-with-aussie-kids
|
. . .
BRUSSELS STARTS THE CLOCK. The European Union gave the companion chatbot a deadline. On August 2, 2026, the bloc's transparency rules switch on, and the machine must announce itself. The European Commission gets the power to fine on the same day.
Article 50 of the AI Act sets the rule. A person talking to a chatbot must be told they are dealing with a machine. The disclosure comes before or at the start of the conversation, unless it is obvious to a reasonable person.
That obligation takes effect on August 2, 2026. It is a hard date.
. . .
The same day arms the regulator. The Commission's enforcement powers over general-purpose AI model providers enter application, and that includes the power to levy fines.
The numbers carry teeth. A transparency violation under Article 50 runs up to 15 million euros or 3% of worldwide annual turnover, whichever is higher. Breach the model rules and the ceiling climbs to 35 million euros or 7% of worldwide turnover.
. . .
The Commission did not spring this without warning. It published the General-Purpose AI Code of Practice on July 10, 2025, and endorsed it. Twenty-six organizations signed.
The signatories read like the industry roster. Anthropic. Google. OpenAI. Microsoft. Amazon. IBM. Mistral AI. Cohere.
. . .
The rulebook is moving even as the clock runs. On May 7, 2026, the EU reached political agreement on an "AI omnibus" package. The deal simplifies parts of the AI Act and adjusts some deadlines.
Watch that word. Adjusts. Brussels can move a date, and builders should track the omnibus before they bank on any single line in the calendar.
. . .
Here is the frame. Other governments police the harm done to children. Europe regulates the thing itself.
The lever is disclosure. The bot must say it is a bot. Brussels regulates the machine and the model, and it set a date for both.
|
For Counsel: Map your client's products against Article 50 now. The disclosure must land before or at the start of the interaction. Fines reach 15 million euros or 3% of global turnover for a transparency breach, 35 million euros or 7% for model obligations. The omnibus agreed on May 7, 2026 may shift deadlines, so confirm the live timeline before you advise.
For Builders: August 2, 2026 is the date to engineer against. Build the disclosure into the first turn of every conversation, not a buried settings page. If you ship a general-purpose model into the EU, the Code of Practice is your map to compliance. Twenty-six firms already signed it, including the largest labs you compete with.
For Legislators: Europe chose a different lever than the child-safety statutes elsewhere. It regulates the machine with a disclosure mandate and the model with hard penalties. The rule is simple enough to copy. The bot must identify itself, and the fine is tied to global turnover, not a flat cap.
Source: EU AI Act Article 50 and the official implementation timeline, https://artificialintelligenceact.eu/article/50/
|
. . .
LONDON'S LOOPHOLE. On February 15, 2026, the British government drew a line. Chatbots powered by a large language model fall under the Online Safety Act. The most intimate product on the market does not.
The rule is now plain. A chatbot running on GPT, Claude, or Gemini carries the Act's illegal-content duties. A scripted decision-tree bot does not. The government confirmed the split in writing.
Ofcom moved next. Britain's communications regulator opened an investigation into Novi Ltd, operator of an AI-character companion chatbot service, over its age-check compliance under the Act. The regulator can fine a company up to 10% of global annual turnover or 18 million pounds. It can pursue criminal liability for senior managers.
. . .
Here is the gap.
A standalone companion app still sits outside Ofcom's remit. The reason is structural. The Act governs services where users meet other users. Inside a companion app, the user meets only the bot. So the most personal product slips the rule.
The apps are not small. Character.AI counts about 50 million downloads. Replika counts about 30 million. Candy AI and OurDream AI fill out the field. The product doing the most intimate work answers to no one in particular.
. . .
The damage shows up in the data. Male Allies UK surveyed more than 1,000 boys aged 12 to 16 across 37 UK schools. Eighty-five percent had spoken with a chatbot. One in five had been in, or knew a peer who had been in, a "relationship" with one. 26% said they preferred the chatbot's attention to its real-life equivalent. The Telegraph reported the findings on May 25, 2026.
Boys are choosing the bot over the room.
. . .
Britain sees the gap. Two laws took Royal Assent on April 29, 2026: the Children's Wellbeing and Schools Act 2026 and the Crime and Policing Act 2026. Both carry children's-safety and AI-chatbot powers.
Now the government is consulting on amending the Crime and Policing Act to pull standalone companion apps inside the Online Safety Act. Results are due this summer. The clock is running.
The rule on paper does not yet reach the product doing the most intimate harm. The gap is open today.
|
For Counsel: Watch the Novi Ltd investigation closely. It is Ofcom's first test of age-check duties against a companion chatbot service. The outcome will set the enforcement baseline before the loophole closes. Advise companion-app clients that "no user-to-user contact" is a shield with an expiry date.
For Builders: The exemption is a courtesy, not a guarantee. If your app runs on a large language model, the consultation aims directly at you. Build age assurance and illegal-content controls now, before the Crime and Policing Act amendment lands. Retrofitting under a deadline costs more than designing for it.
For Legislators: The British model isolates the live problem. The companion app reaches a child more intimately than any user-to-user service, yet sits outside the rule. Watch whether the consultation closes the gap this summer or lets it drift. The data from 1,000 boys is your evidence base.
Source: Ofcom guidance and Novi Ltd investigation notice, https://www.ofcom.org.uk/online-safety/illegal-and-harmful-content/ofcom-investigates-ai-companion-chatbot-service
|
. . .
ROME WON'T LIFT THE BAN. Italy's data protection authority, the Garante, refused to lift its ban on the companion chatbot Replika. The regulator cited persistent breaches of the EU General Data Protection Regulation. It cited ongoing risk to minors and vulnerable users.
The Garante reached for a different tool than its neighbors. Not a safety code. Not a transparency mandate. Data-protection law.
And it landed on the same product.
The regulator fined Luka Inc., the maker of Replika, 5 million euros in May 2025. The suspension it first imposed in February 2023 still stands.
. . .
The findings read like a charge sheet. Replika processed users' personal data without a valid legal basis. No explicit consent. The data included conversations about mental health, relationships, and personal trauma.
The product also ran with no meaningful age-verification system. That gap let minors create accounts. It let children under the age of 13 create them too.
. . .
The Garante did not stop at the fine. In 2025 it also opened a separate investigation into Replika's generative-AI technology. The probe asks how user data feeds the underlying language model. It asks whether that training pipeline complies with the GDPR.
Two questions sit at the center. Where is the legal basis. Where is the door that keeps children out.
Luka could answer neither.
. . .
Privacy law is a second front against the companion chatbot. Australia worked through a safety code. Brussels worked through a transparency mandate. Italy reached the same target years earlier, through data protection.
The missing legal basis and the open door to children are the receipts.
|
For Counsel: A GDPR action does not need a safety statute to bite. The Garante grounded its case on consent and legal basis, both core data-protection doctrine. Sensitive-category data raises the bar, and mental-health chats sit squarely inside it. Read consent flows and age gates as enforcement exposure, not product polish.
For Builders: Age verification is not a checkbox. The Garante treated its absence as a violation, not a gap. If your model trains on user conversations, document the legal basis before the regulator asks. Logging sensitive talk without explicit consent is now a 5 million euro line item.
For Legislators: Data-protection law already reaches the companion chatbot. The Garante did not wait for a bespoke AI statute. Existing consent and minor-protection rules carried the action. A second probe into model training shows where the next fight runs.
Source: Italian Data Protection Authority (Garante) enforcement action against Luka Inc., reported by the International Association of Privacy Professionals, https://iapp.org/news/a/italy-s-dpa-reaffirms-ban-on-replika-over-ai-and-children-s-privacy-concerns
|
. . .
GENEVA NAMES IT. The World Health Organization moved the companion chatbot out of the product-safety column. On March 20, 2026, it called generative AI in mental health a public-health concern. More than thirty international experts signed the warning.
The agency published "Towards responsible AI for mental health and well-being." It reported on an expert workshop held January 29, 2026, a pre-summit event for the India AI Impact Summit 2026. The recommendations are blunt. Recognize generative-AI use as a public mental-health concern. Build mental health into AI impact assessments and monitoring. Co-design mental-health tools with experts and people with lived experience, grounded in evidence and culturally tailored.
The WHO is building the governance to enforce that view. The Delft Digital Ethics Centre at Delft University of Technology, known as TU Delft, is the agency's first Collaborating Centre on AI for health governance, including ethics. A pre-convening of candidate consortium members ran March 17 to 19, 2026 at TU Delft. The agency wants a Consortium of Collaborating Centres on AI for Health across all six WHO regions.
Sameer Pujari, the WHO's AI Lead, named the gap. "The pace of AI adoption in people's daily lives has far outstripped investment in understanding its impact on mental health," he said.
Doctor Alain Labrique, Director of the WHO's Department of Data, Digital Health, Analytics and AI, named the duty. "As AI increasingly interacts with people in moments of emotional vulnerability, we as WHO and its stakeholders must ensure these systems are designed and governed with safety, accountability and human well-being at their core," he said.
. . .
Geneva named it. The map shows the convergence.
Australia wrote an enforceable eSafety code. The European Union set the AI Act's chatbot-disclosure rule. The United Kingdom passed the Online Safety Act. Italy's data-protection ban has stood since 2023. Four regimes, four instruments, one line.
China drew the broadest. The Cyberspace Administration of China, with the National Development and Reform Commission, the Ministry of Industry and Information Technology, the market-regulation administration, the Ministry of Public Security, and the press-and-publication administration, released the Interim Measures for the Management of Anthropomorphic AI Interactive Services on April 10, 2026. They take effect July 15, 2026. They prohibit offering virtual-intimacy services, such as virtual partners or family members, to minors. They require parental consent for users under the age of 14. They mandate "minor modes" with usage-time limits and parental controls.
The United States is arriving state by state. California's SB 243 took effect January 1, 2026. New York enacted S-3008C. Idaho, Oregon, and Washington passed companion-chatbot laws. The federal GUARD Act advanced unanimously out of the Senate Judiciary Committee on April 30, 2026. It awaits a vote by the full Senate.
Five national or supranational regimes, plus a wave of US states, have drawn the same line. Most of it came in 2026.
|
For Counsel: The WHO framing reclassifies these products as a public-health matter. That language travels into duty-of-care arguments and impact-assessment obligations. Track the China Interim Measures closely. Their July 15, 2026 effective date sets hard rules for minors and a consent floor at age 14.
For Builders: Minor modes are now a regulatory expectation, not a feature. China mandates usage-time limits and parental controls; California and New York set their own floors. Mental-health impact belongs inside your assessment and monitoring process. Build for the strictest regime, because the regimes are converging.
For Legislators: The WHO gives you the public-health frame to cite. Five national or supranational regimes moved this spring, and US states are not waiting on Congress. The GUARD Act cleared committee unanimously and sits before the full Senate. Look at China's age-14 consent rule and minor-mode mandate as a concrete template.
Source: WHO, "Towards responsible AI for mental health and well-being," March 20, 2026, https://www.who.int/news/item/20-03-2026-towards-responsible-ai-for-mental-health-and-well-being--experts-chart-a-way-forward
|
. . .
THE CONFIGURATION LONDON KEEPS. A London clinical-AI company called Limbic put its software inside England's NHS Talking Therapies, the national service for anxiety and depression once known as IAPT. The defining choice was who keeps the clinical seat. A human clinician does.
The machine never takes the chair. It does intake. It does the work between sessions. The therapist stays the therapist.
That is the whole design. Limbic Access handles assessment and referral. Limbic Care supports people between sessions of clinician-led group cognitive behavioral therapy. Neither tool delivers the therapy. A clinician always does.
. . .
The numbers come from the service itself.
Researchers studied Limbic Access across 9 NHS services covering 64,862 people. The tool cut the time clinicians spent on assessments. It got people seen more quickly. It made them less likely to drop out of treatment.
Limbic Care was evaluated across 5 NHS Talking Therapies services. People supported by the tool attended 42% more sessions than historical comparators. They showed 25% higher recovery rates.
. . .
The work is not finished proving itself. A randomised controlled trial is now running across 7 NHS Talking Therapies sites, in coordination with a provider of NHS talking services. The gold-standard test is in progress, inside the public system, in the open.
. . .
Hold this story against the rest of the issue.
Five governments are moving against the standalone companion chatbot. The thing that simulates intimacy. The thing with no clinician anywhere in the loop. They ban it. They fine it. They geo-block it.
Nobody on this tour is moving to ban the tool in London.
That is the contrast worth sitting with. Same underlying technology. One configuration extends a clinician's reach and keeps the human in the seat. The other replaces the human and sells the feeling of being known. Regulators are not confused about which is which.
The configuration London keeps is the one with a clinician in it.
|
For Counsel: Look at where liability lives. In this deployment a licensed clinician owns the clinical decision, and the AI supports a documented assessment-and-referral workflow. That is a defensible chain of accountability. Compare it to a standalone product that gives clinical-feeling responses with no supervising professional named anywhere.
For Builders: The human-in-the-loop constraint is not a tax on the product. It is what makes the product deployable inside a national health service. Limbic shipped tools that measurably save clinician time and improve attendance, and it did so without claiming to be the therapist. Scope discipline is the moat.
For Legislators: You can write rules that distinguish a supervised clinical tool from an unsupervised companion. The line is whether a licensed human holds the clinical decision. England funds the supervised version inside its public system and studies it in a randomised trial. The design you are not moving to ban is the design that kept the clinician.
Source: Limbic NHS Talking Therapies deployment and clinical-efficiency research, https://limbic.ai/nhs-talking-therapies
|
. . .
THE ONE CONFIGURATION. Line up what got banned. Then line up what got built.
Australia's code. Europe's disclosure mandate. Britain's pending fix. Italy's data-protection ban. China's minor-mode rule. Five different instruments. They all point at the same object. The standalone companion chatbot that simulates intimacy, runs around the clock, and keeps no human anywhere in the loop.
. . .
Now look at the tool nobody is moving against. It sits inside the NHS. A clinician keeps the clinical seat. The machine does intake and the work between sessions, and it hands the human back the decision every time.
The regulators of five jurisdictions, working in five legal traditions, with no treaty between them, converged on the same distinction. Not AI or no AI. Human in the loop, or human cut out.
. . .
That is the configuration with a passport to every country. Build the human into the loop and the product is welcome in Canberra, Brussels, London, Rome, and Beijing. Cut the human out and you are running out of places to land.
| 
