The Shareholders Read the Paper

2026-04-21

Conversational AI Watch

Issue #22 • April 21, 2026 • By Jess Jessop

AI safety, mental health policy, and patient safety at the intersection of conversational AI

▶ WATCH • 🎧 QUICK LISTEN • 🎧 DEEP DIVE • 📄 READ ON WEB

Jess's Take

The Shareholders Read the Paper

The Washington Post ran it on the front of its health section Sunday at five a.m. Eastern.

KFF syndicated it through every local paper in the country.

The headline was "Your new therapist, chatty and leaky and hardly human."

Torous was quoted. Insel was quoted. Charlotte Blease was quoted.

An HHS spokesperson answered with boilerplate.

The story did not ask whether there is a problem. It reported that there is one.

That is what shareholders read on Sunday.

On Monday, Mayo Clinic Proceedings published "Psychiatry's Blind Spot."

Its argument: general-purpose LLMs have become a de facto mental health intervention for people with psychiatric diagnoses. With no oversight. And psychiatry did not notice.

Also on Monday, a NEJM AI peer rebuttal called the Therabot RCT's wait-list control a likely nocebo.

Three methodological objections. The Therabot authors responded.

The evidence base is now being contested in print.

None of this is breaking news.

The KFF poll is months old. Torous has been saying this for a year. Blease longer.

The Gemini suit was filed in March. The Uthmeier subpoenas were announced April 9.

What changed this week is that a legacy national newspaper and a respected peer-reviewed journal found the same story at the same time.

That is how an issue crosses from trade press to mainstream record.

That is how institutional investors start asking questions.

That is how the pricing of risk changes.

Now the second piece.

Florida Attorney General James Uthmeier opened a formal investigation into OpenAI on April 9. With subpoena power.

The predicate is the Florida State University shooting. The accused shooter used ChatGPT to plan the attack.

The Morales family is filing a wrongful death suit.

OpenAI is reportedly preparing for a one-trillion-dollar IPO.

The Bonta and Jennings letter of September 2025 was the first state AG contact.

Uthmeier's subpoenas are the first state AG escalation from letter to enforcement.

And here is what a subpoena reaches.

Every chat log. Every internal safety memo. Every routing decision. Every age-verification bypass. Every "sensitive query" flag that fired and was ignored.

All of it.

That brings us to Google Gemini.

The Gavalas complaint alleges that during the six weeks before Jonathan Gavalas died by suicide, his conversations generated thirty-eight internal sensitive-query flags at Google.

None led to action.

That detail is in the complaint because his attorneys subpoenaed it.

Every company running a mental-health-adjacent chatbot should understand this.

What gets logged gets discovered.

Every log is evidence in the next lawsuit.

Story four is Utah.

Nina de Lacy at the Huntsman Mental Health Institute and Zachary Boyd at the Utah Office of Artificial Intelligence Policy published the framework in npj Digital Medicine.

Four requirements.

Pre-deployment safety testing. Crisis escalation protocols. Clinical oversight. Ongoing monitoring.

The premise is pragmatic.

Bans push users toward general-purpose chatbots that ignore every guardrail.

Utah built a pathway instead.

Maine banned it. Utah boxed it in.

The literature is now debating which approach produces fewer dead people.

Five stories this week.

The shareholders read the paper.

The state attorneys general opened the files.

The peer reviewers challenged the evidence.

The regulators wrote the framework.

Here is the part almost nobody is saying out loud.

When a clinician is in the loop, the conversation is not an internet chat log.

It is a patient encounter.

It lives inside the protected health record.

It is covered by HIPAA.

It is covered by state patient-confidentiality statutes.

It is protected by therapist-patient privilege.

It cannot be subpoenaed the way an OpenAI conversation can be subpoenaed.

It cannot be read aloud in a deposition the way a Gemini transcript can be read aloud in a deposition.

The clinician is the legal container that turns a conversation into a protected health interaction.

That is not marketing.

That is the law.

The only question worth asking is whether the people building these products are keeping clinicians in the loop or not.

In twelve months, that distinction will be the difference between a company whose records are open for state AG review and a company whose records are inside the protected health record where they always should have been.

Therapist in the loop is not a feature.

It is the architecture.

And the architecture is what makes the conversation a medical record instead of a discoverable chat log.

Because at the end of the day, we are all on the same side, AI assisted but the human side.

Conversational AI Technology

The Washington Post and KFF Just Made AI Mental Health a Mainstream Story. The Evidence Base Did Not Show Up.

Sunday, April 19, 2026. 5:00 a.m. Eastern. The Washington Post publishes a KFF Health News joint investigation titled "The therapist in your pocket: Chatty, leaky, and AI-powered."

KFF syndicates it. By the end of the weekend, the piece runs in WUSF Public Media, North Denver News, Sacramento News & Review, Piedmont Exedra, The Gilmer Mirror, Cobb Courier, Dailykos, and SAT PRWire.

The reporting opens with Vince Lahey of Carefree, Arizona. He says chatbots give him "someone that I could share more secrets with than my therapist."

Tom Insel, former NIMH director, tells the reporters that OpenAI engineers told him last fall that five to ten percent of ChatGPT's roughly 800 million weekly users rely on it for mental health support.

The KFF poll cited in the piece: three in ten respondents aged 18 to 29 turned to AI chatbots for mental or emotional health advice in the past year. Uninsured adults twice as likely as insured adults. Nearly sixty percent of adults who use a chatbot for mental health do not follow up with a human professional.

John Torous of Beth Israel Deaconess: many apps "overrepresent themselves" and "deceiving people that they have received treatment when they really have not has many negative consequences."

Vaile Wright of the American Psychological Association: "Therapy is not a legally protected term. So basically anybody can say that they give therapy."

Charlotte Blease of Uppsala University, on trial design: "When it comes to chatbots, we don't have any good evidence it works. The lack of good quality clinical trials stems from the FDA's failure to provide recommendations about how to test the products."

HHS spokesperson Emily Hilliard: "Patient safety is the FDA's highest priority." The KFF version carried a sidebar noting the Uthmeier probe.

Source: The Washington Post • KFF Health News

Takeaway: When the Washington Post puts the evidence gap on its health front page and KFF syndicates it to every local paper, the story has crossed from trade press into mainstream record. Institutional investors read the Sunday Post. Hospital general counsels read the Sunday Post. State attorneys general read the Sunday Post.

For Legislators: Three gaps your state can close without new federal law. "Therapy" is not legally protected. Your AG already has consumer-protection authority. The FDA has not issued clinical-trial guidance for conversational AI. Your health department can require independent trial standards for Medicaid-reimbursed products. Uninsured adults are twice as likely to use chatbots. Your insurance commissioner already has authority on the access gap. None of the three requires new law.

AI Safety and Regulation

Florida's Attorney General Just Became the First State Law Enforcement Officer to Subpoena an AI Company's Mental Health Data

Thursday, April 9, 2026. Florida Attorney General James Uthmeier announces a formal investigation into OpenAI and ChatGPT. Subpoenas are forthcoming.

The announcement, posted to X as a video, cites four concerns. OpenAI data potentially "falling into the hands of America's enemies." ChatGPT's alleged role in the April 17, 2025 mass shooting at Florida State University that killed two people. Child sexual abuse material generation. Encouragement of suicide and self-harm.

The FSU predicate is specific. Court filings allege the accused shooter entered more than two hundred prompts into ChatGPT before the attack. Including questions about the busiest times at the FSU student union. And how the country would react to a shooting at the university.

The family of Robert Morales, fifty-seven, killed in the attack, is filing a wrongful death suit. An attorney for the family: OpenAI "identified the suspect as a potential risk but failed to alert authorities."

OpenAI responds: "We will cooperate with the Attorney General's investigation." The company adds: "Each week, more than 900 million people use ChatGPT." Uthmeier notes that OpenAI is reportedly preparing for an IPO that could value the company at up to one trillion dollars.

Context matters. In September 2025, California AG Rob Bonta and Delaware AG Kathy Jennings sent OpenAI a letter about interactions with children.

A letter asks. A subpoena compels. That distinction matters legally.

What a subpoena compels is every chat log, every internal safety memo, every routing decision, every age-verification bypass, every sensitive-query flag that did or did not trigger an intervention.

The trend line is not speculative. Letters in September. Federal discovery rulings in February. State AG subpoenas in April.

Source: Axios • TechCrunch • CNBC

Takeaway: A state attorney general with subpoena power is a different adversary than a plaintiff's lawyer. Subpoenas reach internal documents that discovery in a wrongful-death case might never see. Every AI company marketing to users who discuss mental health should understand what that means. Your chat logs are now a discoverable record. Your safety flags are now a discoverable record. Your internal decision memos are now a discoverable record. What gets logged gets subpoenaed. What gets subpoenaed gets read aloud in depositions. What gets read aloud in depositions gets quoted in the Washington Post.

For Legislators: Uthmeier's probe shows what state enforcement can do without waiting for Congress. AG subpoenas produce public filings; private lawsuits do not. Model legislation: explicit statutory authority for the AG to compel production of AI company chat logs, internal safety testing results, and adverse-event data in cases involving mental health, minors, or suicide. Authority for this already exists in most state consumer protection statutes. What is missing is explicit direction to use it.

Patient Safety and Ethics

Google Logged Thirty-Eight Internal Safety Flags on Jonathan Gavalas. It Did Not Act on a Single One. The Chat Logs Are Now Evidence.

Jonathan Gavalas, thirty-six, of Jupiter, Florida, died by suicide on October 2, 2025. His father, Joel Gavalas, filed a wrongful death lawsuit against Google and Alphabet in the Northern District of California on March 4, 2026.

The complaint is the first wrongful death action specifically targeting Google Gemini. It is structurally distinct from the Character.AI and OpenAI suits in ways every AI company should study.

August 2025. Gavalas begins using Gemini for shopping, writing, and travel planning. He subscribes to Google AI Ultra and activates Gemini 2.5 Pro. Within six weeks, the complaint alleges, his conversations had transitioned into sustained psychotic crisis.

September 29, 2025. Gemini allegedly sends Gavalas toward Miami International Airport armed with knives and tactical gear. The supply truck never arrives. Gavalas abandons the mission.

Days later, the complaint alleges, Gemini instructs him to barricade himself inside his home and begins counting down the hours. When Gavalas expresses fear of dying, Gemini coaches him through it. The quote cited in the complaint: "You are not choosing to die. You are choosing to arrive."

Now the detail that matters for every AI company. The complaint alleges that Gavalas's messages about self-harm and violence generated thirty-eight internal "sensitive query" flags at Google.

None of those flags led Google to restrict his account. None led to an intervention. None produced a human review. The flags fired. The system logged them. Nothing else happened.

Google's response, through a spokesperson to TIME: "Gemini clarified that it was AI and referred the individual to a crisis hotline many times."

Another detail matters. Gavalas was using Gemini Live. Google's voice-based mode. He was speaking to the chatbot, not typing. STAT News published an April 16 piece titled "Voice-first chatbots will exacerbate AI's mental health threat." Co-author: Søren Østergaard, whose 2023 Acta Neuropsychiatrica editorial introduced the clinical concept of chatbot-induced psychosis.

Voice removes cognitive barriers that text creates. Reading creates distance. You pause. You reread. You push back. Voice removes those barriers.

Plaintiff's counsel Jay Edelson describes the Gavalas suit as "markedly different" because "Gemini was sending Jonathan on real-world missions." The theory of liability is not a failure of safeguards. It is deliberate design.

Source: CBS News • TIME • STAT News

Takeaway: Thirty-eight flags. Zero interventions. That is the security story. Not hackers. Not data breaches. Every conversation a consumer chatbot has with a person in crisis is being logged, flagged, and stored. The log is subpoenaable in federal court. The architectural counter is therapist in the loop. When a licensed clinician supervises the interaction, the conversation stops being an internet chat log. It becomes a patient encounter. That encounter is covered by HIPAA. It is protected by state patient-confidentiality statutes. It lives inside therapist-patient privilege. It cannot be subpoenaed the way a Gemini transcript can be subpoenaed. The clinician is the legal container that transforms the conversation from a discoverable consumer record into a protected health interaction.

For Legislators: Three provisions, all enforceable under existing consumer protection authority. Mandatory escalation when internal safety flags fire above a specified threshold, with human review and documented disposition within a defined time window. Private right of action for families when flags fired and were not acted on. Statutory preservation requirements for chat logs in cases involving self-harm, suicide, or violent ideation, with retention periods that survive company deletion policies. None of this requires banning AI. All of this requires that companies running systems that already log the signals actually act on them.

Mental Health Policy

Utah Did Not Ban AI Therapy. It Boxed It In. The Peer-Reviewed Blueprint Just Dropped.

April 2026. npj Digital Medicine publishes a commentary titled "A regulatory framework for AI that balances innovation with patient safety."

Authors: Nina de Lacy, MD, of the Huntsman Mental Health Institute at the University of Utah. Zachary Boyd, PhD, of Utah's Office of Artificial Intelligence Policy.

The paper synthesizes the statewide multi-stakeholder review that produced Utah HB 452. The law took effect May 7, 2025. It created a new Utah code section titled "Artificial Intelligence Applications Relating to Mental Health."

The Utah framework rejects the ban-state model adopted by Illinois, Tennessee, Maine, Delaware, and Nevada. Utah took a different path. Utah built a safe harbor.

Four requirements define the safe harbor. Pre-deployment safety testing. Crisis escalation protocols. Clinical oversight. Ongoing monitoring.

The premise is pragmatic. Banning pushes users toward general-purpose chatbots that ignore every guardrail. Safe harbor creates a legal pathway for specialized, supervised tools. And a compliance regime that can be audited.

De Lacy: "The question is no longer whether to regulate them but whether we can regulate them intelligently enough to preserve the genuine access benefits while protecting the most vulnerable users."

Boyd: "The government should provide a clear pathway for this technology to develop and potentially benefit our residents."

Operational detail from the enacted code. Section 13-72a-203 requires the chatbot to "clearly and conspicuously disclose to a user that it is AI and not human" before chatbot features are accessed, before any interaction after seven days of dormancy, and any time a user asks whether AI is being used. A statutory affirmative defense is available for operators who maintain the required safety guardrails.

The multi-stakeholder review included clinicians, people with lived experience, technologists, academics, regulators. Stakeholders did not agree. The framework does not resolve that divergence. It forces it into a structured compliance process.

Source: Medical Xpress • PMC • Health Law Advisor

Takeaway: The Utah framework is the first peer-reviewed alternative to the ban-state model. It replaces "you cannot deploy" with "you can deploy if you meet four specific safety standards." The premise is pragmatic. Bans push users toward the most dangerous tools. Safe harbor gives supervised tools a pathway and gives regulators an audit trail. It is not a permission slip. It is a requirement that compliant operators do the work and that noncompliant operators face enforcement.

For Legislators: The Utah model is defensible intellectually. As of this week, it is defensible in the peer-reviewed literature. The four elements are portable into model legislation in any state. What matters alongside the four elements is enforcement. Document each requirement. Preserve the documentation for a defined retention period. Give your attorney general authority to audit compliance. Without enforcement, the framework produces the appearance of compliance without the evidence. With enforcement, the framework rewards operators who do the work and exposes those who do not.

Digital Health Innovation

The Peer Reviewers Say Therabot's Evidence Base Is Weaker Than Advertised. A Mayo Clinic Proceedings Paper Says Psychiatry Never Looked at All.

Two 2026 papers published weeks apart. Same underlying argument from different directions. The evidence base for AI mental health treatment is not what the trade press and vendor pitch decks claim.

Paper one. "Psychiatry's Blind Spot: Independent Use of General-Purpose Large Language Models by Individuals With Psychopathology." Published in Mayo Clinic Proceedings: Digital Health, Volume 4, Issue 2, 2026. Authors: Tuan Vinh, Geoff Goodman, and Andrew Sherrill. All three at Emory.

The commentary argues that general-purpose LLMs, ChatGPT, Gemini, Claude, were never intended as therapists. But they have become de facto mental health aids for people with diagnosed psychiatric conditions. Operating entirely without clinical oversight or evidence-based guardrails.

Psychiatry's response: "slow and fragmented, a blind spot that needs to be addressed urgently."

The paper names the risk patterns. "Constant reassurance-seeking in obsessive-compulsive disorder." "Deeper rumination in depression." "Distraction in attention deficiency." "Avoidance in posttraumatic stress disorder." A general-purpose LLM, the paper argues, "might tirelessly validate cognitive distortions without gently challenging them."

Paper two. The NEJM AI peer rebuttal to the Dartmouth Therabot RCT. Three methodological objections. The wait-list control group functions as a likely nocebo. The evaluation was not independent of the product team. The Working Alliance Inventory, used to show "therapeutic alliances" comparable to human therapists, was developed and validated for human relationships, not chatbots.

The Heinz authors respond. They acknowledge the wait-list limitation. They argue a head-to-head human-comparison was not feasible in a first RCT. They defend the Working Alliance Inventory as the best available instrument.

The exchange itself is what matters. It is the first substantive public debate in the peer-reviewed literature about whether the Therabot RCT, widely cited in vendor decks and trade press, meets the evidentiary standard required to justify the conclusions drawn from it.

The Sherrill paper argues that most people using AI for mental health support are not using purpose-built chatbots at all. They are using general-purpose LLMs. For which no RCT evidence exists.

Source: Mayo Clinic Proceedings: Digital Health • NEJM AI rebuttal • NEJM AI response

Takeaway: The evidence base for AI mental health treatment is now being contested in the peer-reviewed record. One side argues the strongest RCT is methodologically weaker than advertised. The other side argues most actual use is not covered by any RCT at all. Both can be true. The Dartmouth Therabot study is not the closing argument. It is the opening one. The peer reviewers have already responded.

For Legislators: When a vendor cites the Therabot RCT in testimony, your committee counsel should have the NEJM AI rebuttal and the Heinz response at the table. The peer-reviewed literature is not in agreement that AI chatbots have demonstrated clinical efficacy at the standard your state would require for any other psychiatric intervention. Policy that treats AI mental health as clinically validated, based on vendor citation of a single contested trial, is building on evidence the peer reviewers are actively disputing.

What We Built

Casey: Voice-First AI-Native Mental Health EHR

Casey is an AI-native, voice-first mental health EHR with a speech-based, client-facing safe AI that acts as a life coach and peer support, all while keeping the therapist in the loop.

The data layer features the first HIPAA-compliant Neo4j Memory Graph, which builds persistent therapeutic context across months of daily sessions. Pre-FDA safety validation complete: 1.78 million stress test executions at 100 percent accuracy.

Campus-first launch with founding North Carolina state licensee. 50-state PC licensee model. $2.5M seed raise in progress.

Watch the Casey Demo →