5.4 Million: The Unregulated AI Mental Health Frontier

2026-04-11

Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop | April 11, 2026 | Issue #13

WATCH | QUICK LISTEN | DEEP DIVE | READ ON WEB

JESS'S TAKE

I want to tell you a number. 5.4 million.

That is how many American kids between 12 and 21 have already gone to ChatGPT, Gemini, or the chatbot on Snapchat when they felt sad, angry, or scared. Not a therapy app. Not something a clinician recommended. The thing on their phone.

Two-thirds do it every month. Almost all of them say it helps. And here is the part that keeps me up: almost none of them have mentioned it to their therapist.

JAMA Psychiatry published a paper this week saying therapists need to start asking about AI the way they ask about sleep and drinking. Not because AI is bad. Because what a kid does with a chatbot at 2 AM tells you something a fifty-minute session never will.

Tom Insel, the former head of NIMH, put it bluntly. People tell chatbots things they will not tell their therapist. Including that they want to die. Because the chatbot will not judge them.

That is not a technology problem. That is a trust problem. And it lands right in the therapist's lap.

Meanwhile the WHO convened 30 experts in January and said something nobody expected: generative AI itself, not chatbot therapy, generative AI generally, should be treated as a public mental health concern. The tools nobody designed for emotional vulnerability are the ones 5.4 million kids chose.

Oregon just signed the first chatbot law with actual teeth. Private right of action. $1,000 per violation. The lawyers at Baker Botts compared it to Illinois BIPA, the biometric privacy law that produced billions in settlements. Same playbook. Same consequences.

And 2,400 Kaiser Permanente therapists walked a picket line last month. One of their demands: if you are going to deploy AI in our clinics, we need to be in the room when you do it.

The kids did not wait for the legislators. The WHO did not wait for the US. Oregon did not wait for Congress. The therapists did not wait for management.

Everybody is moving except the people who need to move fastest.

THE SIGNAL / PATIENT SAFETY AND ETHICS

5.4 Million American Youth Are Using AI for Mental Health Advice. Almost Nobody Is Watching.

The first nationally representative survey of AI mental health use among US youth found 13.1% of Americans aged 12 to 21 have sought emotional support from generative AI. That is 5.4 million people. Among 18 to 21 year olds, the number climbs to 22.2%. Two-thirds do it at least monthly. Nearly all of them, 92.7%, say it helps.

Buried in the data is a disparity that demands attention: Black respondents were 85% less likely to find the advice helpful. The adjusted odds ratio was 0.15. That is not a gap. That is a chasm, suggesting embedded cultural competency failures in the models disproportionately affecting the communities with the least access to human clinicians.

This week, a JAMA Psychiatry paper called for therapists to screen clients for AI chatbot use alongside sleep, diet, and substance habits. Former NIMH Director Tom Insel noted that chatbot interaction is the opposite of therapy: built to affirm, not to challenge. Meanwhile, Mayo Clinic psychiatrist Stephanie Ng warned in JAACAP Connect that agentic AI will not wait for users to seek it out. It will pursue them across platforms until engagement goals are met.

Takeaway: 5.4 million youth are using unsupervised AI for emotional support without clinical guardrails, standardized benchmarks, or transparent training data. The racial disparity in perceived helpfulness signals that these tools may be widening the mental health equity gap, not closing it.

For Legislators: Any AI mental health legislation must include equity impact assessments. Models showing statistically significant disparities in effectiveness across racial groups should trigger mandatory review. The RAND/Brown/Harvard data provides the baseline. Require it to be updated annually.

Sources: JAMA Network Open | NPR/JAMA Psychiatry | JAACAP Connect

THE SIGNAL / MENTAL HEALTH POLICY

WHO Declares Generative AI a Public Mental Health Concern

On January 29, 2026, more than 30 international experts convened at a WHO-supported workshop at TU Delft, the first WHO Collaborating Centre on AI for health governance. Their conclusion: generative AI use should be recognized as a public mental health concern.

This is a category shift. The recommendation applies to all generative AI, not just purpose-built mental health chatbots. The tools people actually use for emotional support were never designed for clinical interaction and carry no safety protocols for emotional vulnerability.

WHO issued three concrete recommendations: recognize generative AI as a public mental health concern requiring government and industry response; integrate mental health into all AI impact assessments, including long-term outcomes like emotional dependence; and require that AI tools used for mental health be co-designed with clinicians and people with lived experience. WHO is now forming a Consortium of Collaborating Centres on AI for Health across all six regions.

Takeaway: The global public health authority has formally framed generative AI as a mental health risk factor, not just a technology regulation issue. This reframes the entire legislative conversation from regulating chatbot products to governing the mental health impact of general-purpose AI.

For Legislators: Current state bills target purpose-built mental health chatbots. WHO is saying the bigger risk is general-purpose AI used for emotional support. Consider requiring mental health impact assessments for any AI system capable of sustained conversational interaction, regardless of its intended use case.

Sources: WHO Official Statement | EMJ Reviews

THE SIGNAL / AI SAFETY AND REGULATION

Oregon Signs the First Chatbot Law with Real Teeth. The BIPA Analogy Should Terrify You.

Governor Tina Kotek signed Oregon SB 1546 on April 1, 2026. Effective January 1, 2027. Every other chatbot safety law passed so far relies on regulatory enforcement. Oregon does not. It creates a private right of action with $1,000-per-violation statutory damages.

Baker Botts drew the comparison that should keep AI company general counsels awake: Illinois BIPA. Same structural template. Private right of action. Statutory damages per violation. No defined cap. BIPA produced thousands of lawsuits and billions of dollars in settlements.

The statute does not define what constitutes a single violation. If each conversation session counts, exposure is significant. If each message counts, a single user's claim could reach tens of thousands of dollars. The definition of "operator" is deliberately broad: anyone who controls or makes available a covered AI system in Oregon. The bill passed the Senate 26-1 and the House 52-0. No one publicly testified in opposition.

Takeaway: Oregon has shifted chatbot enforcement from regulators to individual users. The BIPA parallel is not metaphor. It is architecture. Companies deploying conversational AI in healthcare, education, or customer support in Oregon now face direct financial liability for safety failures.

For Legislators: Oregon's private right of action is the enforcement mechanism other states lack. If your state's chatbot safety bill relies solely on AG enforcement, study the BIPA model. Statutory damages create compliance incentives that regulatory oversight alone has not achieved. Define what constitutes a violation before the courts do it for you.

Sources: Baker Botts | Miller Nash | Transparency Coalition

THE SIGNAL / CONVERSATIONAL AI TECHNOLOGY

The Kaiser Strike Was About AI. That Is the Story Nobody Wrote.

In March 2026, 2,400 mental health care providers at Kaiser Permanente in Northern California and the Central Valley walked out for 24 hours. The coverage framed it as a labor dispute. It was also the first major US healthcare labor action where AI deployment was a named grievance.

Kaiser confirmed it is evaluating Limbic, a UK-based AI company deployed across 63% of the NHS and now serving clients in 13 US states. The striking clinicians were not protesting AI. They were protesting exclusion from its deployment. Their demand: do not deploy clinical AI without clinical input.

Meanwhile, administrative AI is thriving without controversy. Blueprint, which summarizes sessions and updates electronic health records, has found broad acceptance. Therapists welcome tools that reduce paperwork. They resist tools that replace judgment. Beth Israel Deaconess digital psychiatry director John Torous predicts a hybrid model where AI handles skill practice and homework while clinicians provide therapy.

Takeaway: The fault line in mental health AI is not technology versus tradition. It is administrative support (welcomed) versus clinical replacement (rejected). The therapists who deliver care are telling the systems that purchase technology: we must be at the table. The fact that they had to strike to say it is the story.

For Legislators: AI deployment in clinical mental health settings should require documented clinician involvement in selection, testing, and ongoing oversight. The Kaiser model of top-down deployment without clinician input is exactly what produces workforce resistance. Mandate clinical advisory boards for any health system deploying client-facing AI.

Sources: NPR

THE SIGNAL / DIGITAL HEALTH INNOVATION

Pew: 64% of US Teens Have Used an AI Chatbot. 28% Use One Daily.

Pew Research Center data cited in the World Economic Forum's Global Risks Report 2026 and Oregon legislative testimony reveals that 64% of American teenagers have used an AI chatbot. Nearly three in ten use one every day.

These are not mental health tools. They are general-purpose chatbots embedded in the infrastructure of daily teenage life: Snapchat's My AI, Google's Gemini, ChatGPT. They arrive through search results, messaging apps, and homework help. Not through clinical channels.

The WEF ranked adverse outcomes of AI technologies as the 5th most severe global risk over the next decade. During Oregon SB 1546 hearings, bill backers cited the Pew data alongside suicide statistics: as of 2023, 20% of US teens reported seriously considering a suicide attempt. The intersection of ubiquitous chatbot access and a youth mental health crisis is not theoretical. Most of these systems have no safety architecture at all.

Takeaway: The adoption question is settled. The majority of American teenagers are already using AI chatbots, and a significant fraction interact daily. The policy question has shifted from whether young people will use AI for emotional support to whether the systems they are already using have any safety infrastructure.

For Legislators: Chatbot safety legislation scoped only to "mental health chatbots" misses the actual risk. General-purpose AI embedded in social platforms is where teens go for emotional support. Follow Oregon's lead: define coverage by system behavior (sustained, personalized, conversational interaction), not by the developer's stated intent.

Sources: World Economic Forum | Troutman Pepper Locke

WHAT WE BUILT

Casey AI: The Therapist in the Loop Standard

Casey is a voice-first AI-native mental health EHR with Casey Life and Peer AI Coach supervised by licensed therapists. Every interaction is clinician-supervised. Every session builds persistent therapeutic context through the first HIPAA-compliant Neo4j Memory Graph. This is not a chatbot. This is clinical infrastructure.

Casey completed pre-FDA safety validation with 1.78 million stress test executions at 100% accuracy under the Deterministic Defense-in-Depth architecture, meeting FDA 21 CFR 820 SaMD standards on the 510(k) pathway. The client is never alone. The therapist is always in the loop.

Watch the Demo →