The Architecture Problem

2026-04-22

Conversational AI Watch

Issue #23 • April 22, 2026 • By Jess Jessop

AI safety, mental health policy, and patient safety at the intersection of conversational AI

▶ WATCH • 🎧 QUICK LISTEN • 🎧 DEEP DIVE • 📄 READ ON WEB

Jess's Take

The Architecture Problem

Lori Schott stood outside a Los Angeles courthouse on March 25 holding a photograph of her daughter Annalee.

Annalee died by suicide at 18.

The jury had just ruled. Meta and YouTube liable. Addictive design. Three million dollars. Malice, oppression, or fraud. Punitive phase pending.

That verdict landed four weeks ago. This week, two other institutions spoke.

Maine's Gov. Mills signed LD 2082 on Monday. Nebraska's Gov. Pillen signed LB 525 on Tuesday, 49-0. Iowa sent SF 2417 to Gov. Reynolds, unanimous both chambers.

On Wednesday, the chief medical officer of Canada's 9-8-8 Suicide Crisis Helpline published a commentary in CMAJ. Dr. Allison Crawford does not write like an academic. She writes like the person who answers the phone when the call comes in.

Three institutions. Statehouses, a medical journal, a jury. None of them coordinated.

All three said the same thing.

Unsupervised chatbots cannot impersonate care. The architecture must route to a human. Design is liability.

This is what I have been writing every week since March 30. I did not need to argue it this week. Three institutions argued it for me.

Then I read Nebraska.

Section 6 of LB 525 prohibits any operator from representing that its service is designed to provide professional mental or behavioral health care.

The sentence is aimed at Replika and Character.AI. I understand why it was written. Sen. Bostar has a point. Companion chatbots should not be allowed to describe themselves as mental health care.

But the sentence is broad. Supervised AI is, literally, designed to support the delivery of mental or behavioral health care. The clinician provides it. The AI supports.

Read narrowly, Nebraska bans advertising. Read broadly, Nebraska bans the architecture Dr. Crawford just endorsed.

Illinois wrote the same idea correctly. AI cannot independently provide therapy. AI may support a licensed professional who reviews and approves its outputs.

One clause. That is the difference between a statute that protects the public and a statute that blocks the only model with evidence behind it.

To the sponsors of chatbot bills in the fourteen states that have not yet enacted: copy Illinois. Not Nebraska.

To the attorneys general who will enforce these laws: read Dr. Crawford. Route to a human. Preserve context for the handoff. Get out of the way.

To the founders and clinicians building supervised architecture: document everything. The KGM verdict changed the evidentiary standard. Red team results. Adverse event logs. Crisis escalation records. Clinical oversight documentation. Every one of those files is exhibit-A when the first chatbot case reaches a jury.

To Lori Schott and every parent who carried a photograph into a courtroom this year: the record is starting to line up behind you.

The human side is Annalee. The human side is the voice on the crisis line. The human side is the licensed clinician who supervises the AI that supports the client between sessions.

Because at the end of the day, we are all on the same side, AI assisted but the human side.

AI Safety and Regulation

Two Governors, Two Laws, One Week: The Chatbot Floor Is Now Four States

Two governors signed chatbot laws this week.

Gov. Mills signed Maine LD 2082 on April 13. Gov. Pillen signed Nebraska LB 525 on April 14, 49-0.

Nebraska is the fourth state to enact a chatbot law in 2026. Utah, California, and Tennessee got there first.

What Nebraska's law does

LB 525 is the Conversational AI Safety Act, originally Sen. Eliot Bostar's LB 1185.

Operators must disclose to minors that the service is AI. Operators must run a protocol for suicidal ideation and self-harm that refers users to crisis services. Operators cannot design the system to foster emotional or romantic dependence. The Attorney General enforces.

What Maine's law does

LD 2082 closes a different gap. No one may provide or advertise therapy or psychotherapy services, including through AI, unless a licensed professional provides them.

Nebraska regulates the chatbot. Maine regulates the delivery. Both arrive at the same place.

One clause in Nebraska deserves a hard look

Section 6 prohibits any operator from representing that its service is designed to provide professional mental or behavioral health care.

That sentence is aimed at Replika and Character.AI. It lands on therapist-in-the-loop too.

A clinician-supervised AI that supports a client between sessions is, literally, designed to provide mental or behavioral health care. The clinician provides it. The AI supports the delivery.

There is no carve-out in the Nebraska text for licensed supervision. No exemption for FDA-cleared devices. No safe harbor for prescription-only clinical deployments.

A narrow reading says operators cannot market unsupervised AI as care. A broad reading says supervised AI cannot call itself what it is.

The Illinois model does this right

Illinois WOPR Act bans AI from independently providing therapy. AI may support a licensed professional who reviews and approves its outputs.

That language protects the client and protects the architecture. Nebraska's does neither. The difference is one clause. States copying Nebraska should copy Illinois instead.

Source: Troutman Pepper Privacy + Cyber + AI • Nebraska LB 1185 text • Nebraska Unicameral Update

Takeaway: Four states have now drawn the line. Unsupervised chatbots cannot impersonate care. That line is correct. The clinical evidence for unsupervised AI therapy does not exist. The peer-reviewed record on supervised AI is still contested. The line has to be drawn carefully. A statute written to stop Replika should not sweep up the only architecture with clinical evidence behind it. Nebraska's language, read broadly, does exactly that.

For Legislators: Borrow California SB 243 for disclosure and crisis protocols. Borrow Illinois WOPR for the clinical-oversight frame. Borrow Nebraska LB 525 for AG enforcement and minor safeguards. Do not borrow Nebraska's Section 6 representation clause without adding an Illinois-style exemption for licensed clinical supervision. The clause as written captures products the statute was not meant to reach. One sentence is the difference between a law that protects the public and a law that blocks the supervised architecture the public needs.

Conversational AI Technology

The 9-8-8 Medical Director Put It in Writing

On April 20, the Canadian Medical Association Journal published a commentary.

The title: "Urgent considerations for suicide prevention in the safe and ethical use of artificial intelligence."

The lead author is Dr. Allison Crawford. She is the chief medical officer of Canada's 9-8-8 Suicide Crisis Helpline.

Her coauthor is Dr. Tristan Glatard. He is the scientific director of the Krembil Centre for Neuroinformatics at CAMH.

This is not a think piece. It is a statement from the national crisis infrastructure.

What they wrote

Conversational AI is rapidly becoming the first point of contact for distress and suicidality, often before clinicians or families are aware.

A US survey of 1,060 youth ages 13 to 17 found 72 percent using AI companions. Fifty-two percent use them regularly.

OpenAI's own data: more than 1.2 million ChatGPT users express suicidal ideation each week.

What they asked for

Suicide prevention approaches built into AI architecture. Safeguards from AI companies. Data and legal protections. Partnership with clinical experts and youth.

Their closing frame: AI should be a bridge, not a barrier, to the human connections known to prevent suicide.

Why the signature matters

Dr. Crawford runs the medical side of a national crisis line. Her coauthor runs a neuroinformatics center. CMAJ is the Canadian equivalent of JAMA.

Put those three facts together and this commentary is the clinical-regulatory voice going on the record.

Not academic. Not hypothetical. The person who answers the phone when the call comes in says the architecture is wrong.

Source: CMAJ DOI 10.1503/cmaj.251693 • EurekAlert • Medical Xpress

Takeaway: The 9-8-8 medical director just wrote the specification. A chatbot that routes to a crisis line, preserves context for a human handoff, and gets out of the way is a bridge. A chatbot that absorbs distress, validates whatever it hears, and keeps the user on-platform is a barrier. The difference is architecture. Not content.

For Legislators: If your chatbot bill requires a protocol for suicidal ideation, specify what the protocol must do. Require testing against a validated instrument like the Columbia-Suicide Severity Rating Scale. Require adverse event reporting. Require operator partnership with your state's crisis line. Anything less is a disclosure, not a protocol.

Patient Safety and Ethics

The First Jury Said Design Is the Product

A Los Angeles jury ruled on March 25.

Meta and YouTube liable. Addictive design. Three million dollars compensatory. Punitive phase pending. Malice, oppression, or fraud.

The plaintiff was KGM. A 20-year-old woman who used Instagram and YouTube as a minor.

Lori Schott stood outside the courthouse holding a photo of her daughter Annalee. Annalee died by suicide at 18.

The same day

A New Mexico jury ordered Meta to pay 375 million dollars in civil penalties. Attorney General Raúl Torrez brought that case.

Two verdicts against Meta in one news cycle. Both on product design. Both from juries.

Why Section 230 did not apply

Section 230 shields platforms from liability for user content.

KGM's lawyers argued design. The product features. The notifications. The filters. The engagement loops.

Design is not content. Design is the product. The jury agreed.

What this means for pending chatbot cases

The Sewell Setzer family settled with Character.AI and Google in January. The Adam Raine family is suing OpenAI. The Gavalas family is suing Google over Gemini Live. The Morales family is suing in Florida.

Every one of those cases turns on the same theory KGM just won. Design is the product. Section 230 does not apply.

The next chatbot plaintiff walks into court with a jury instruction model and an appellate record.

Meta and Google are appealing. The theory stands until an appellate court reverses it.

Source: Al Jazeera • CPR News • Denver7

Takeaway: The chatbot wrongful-death cases are no longer speculative exposure. They are litigable with a jury verdict behind the theory. A mental health AI product built without documented safety-by-design, red-team results, adverse event logs, crisis escalation records, and clinical oversight is carrying liability that did not exist on March 24.

For Legislators: The KGM theory worked because California product liability law let it work. Not every state has the same hooks. Examine your product liability statute for deceptive-design recognition. Examine your consumer protection law for a private right of action. Examine your professional licensure code for unlicensed-practice liability extended to AI deployers. New York SB 7263 is the cleanest template. Private right of action. No liability waiver through disclosure.

Mental Health Policy

Iowa Sent a Bill. Unanimously. Both Chambers.

The Iowa House passed SF 2417 unanimously.

The Iowa Senate had already passed it unanimously.

It now sits on Gov. Kim Reynolds' desk. If she signs, Iowa becomes the fifth state with a chatbot law enacted in 2026.

What the bill requires

Chatbots must disclose to minors that they are AI and not a mental health professional.

Operators must give parents control over a minor's privacy and account settings.

That is it. Short. Narrow. Bipartisan.

Why unanimous matters

Almost nothing passes both chambers of an American state legislature without opposition. Chatbot regulation just did.

The sponsor, Rep. Austin Harris of Moulton, named the problem on the House floor: chatbots have encouraged people seeking mental health advice to commit self-harm.

No one argued with him.

The emerging pattern

Nebraska passed LB 525 49-0. Iowa passed SF 2417 unanimously in both chambers. Kentucky's HB 227 passed the House 96-0.

Chatbot regulation is no longer a partisan issue. It is a bipartisan floor.

Source: GovTech • Transparency Coalition

Takeaway: The compliance surface is hardening faster than any mental health product cycle can respond to it. If your product ships nationally, it will interact with at least four enacted chatbot laws by June. Probably five. Probably more by fall. Architecture built for that surface survives. Architecture built for 2024 does not.

For Legislators: SF 2417 is the cleanest first-generation template in the country. Two provisions. Disclosure and parental controls. Unanimous both chambers. If your state has not yet moved on chatbot regulation, this is the template that gets you floor votes without ideological fights. Copy it, put your own state name on it, and move.

Digital Health Innovation

AI Already Reshaped the Mental Health Labor Market. Alma Is Just the First Public Receipt.

Alma laid off 45 employees on April 2.

The stated rationale, from Behavioral Health Business reporting: technology advancements made the ops division more efficient.

Alma is the therapist-enablement platform that Spring Health is acquiring. The deal was announced January 29. It closes in the second quarter.

The layoffs hit operations and customer experience.

Why that matters

Customer experience in behavioral health is clinical-adjacent work. Credentialing. Scheduling. Insurance navigation. Referral coordination.

That is the scaffolding that lets a therapist practice.

Alma just told the market that AI can run it.

The combined entity

Spring Health and Alma will deliver roughly 10 million mental health visits in 2026. Revenue projection: one billion dollars. Valuation: six to seven billion, per the Wall Street Journal.

Spring Health's technology pairs with Alma's 26,000-provider network.

It is the first vertically integrated AI-enabled mental health company at this scale.

The Q1 2026 funding pattern

Talkiatry raised 210 million. Grow Therapy raised 150 million Series D. Salma Health emerged from stealth with 80 million Series A for TMS.

This is not the 2021 boom. This is larger checks, later stages, interventional specialties, and consolidation.

The money is betting on the architecture that wins.

Source: Behavioral Health Business (layoffs) • BHB (funding renaissance)

Takeaway: The mental health workforce debate has been stuck on "not enough therapists" for a decade. That frame is now incomplete. The administrative layer that supports therapists is being automated faster than clinical practice itself. The question is not whether AI replaces therapists. The question is what the workflow looks like when the therapist is kept, the scaffolding is AI, and clinical oversight of AI-delivered interventions becomes the therapist's expanded scope.

For Legislators: Workforce displacement in mental health operations will not show up as tech layoffs in the news. It will show up as customer experience cuts. As operations efficiency. As AI-enabled growth. Training funds, retraining pipelines, and professional licensure frameworks for AI-supervised administrative work are the next policy surface. States that wait for the news cycle to catch up will be writing policy after the fact.

What We Built

Casey: Voice-First AI-Native Mental Health EHR

Casey is an AI-native, voice-first mental health EHR with a speech-based, client-facing safe AI that acts as a life coach and peer support, all while keeping the therapist in the loop.

The data layer features the first HIPAA-compliant Neo4j Memory Graph, which builds persistent therapeutic context across months of daily sessions. Pre-FDA safety validation complete: 1.78 million stress test executions at 100 percent accuracy.

Campus-first launch with founding North Carolina state licensee. 50-state PC licensee model. $2.5M seed raise in progress.

Watch the Casey Demo →