The Medical Establishment Speaks

2026-04-24

Conversational AI Watch

Issue #26 • April 24, 2026 • By Jess Jessop

AI safety, mental health policy, and patient safety at the intersection of conversational AI

▶ WATCH • 🎧 QUICK LISTEN • 🎧 DEEP DIVE • 📄 READ ON WEB

Jess's Take

The Medical Establishment Speaks

Issue #25 covered the four institutions that moved on Monday, Tuesday, and Thursday. A state attorney general with subpoena power. A national government with embedded reviewers. A psychiatric roundtable led by the chair of the DSM-IV. A model release where the crisis number went down.

This week added three more institutions in the same frame.

On Monday, Stanford published the first peer-reviewed paper to name the mechanism. Not anecdote. Nineteen verbatim transcripts. The paper is called Characterizing Delusional Spirals through Human-LLM Chat Logs. It will be presented at ACM FAccT in June.

On Tuesday, the American Medical Association sent a letter to the co-chairs of the House AI Caucus, the Senate AI Caucus, and the Congressional Digital Health Caucus. Five safeguards. FDA review for any chatbot that diagnoses or treats a mental health condition. That is the same architecture Maine codified eleven days ago.

On Tuesday, sixty families stood on the Capitol's west lawn with a hundred and fifty roses. One rose for each child they say died because of a tech platform. Alicia Shamblin was there. She is suing OpenAI over her twenty-three-year-old son Zane.

The doctors are on record. The researchers are on record. The families are on the Capitol lawn.

Because at the end of the day, we are all on the same side, AI assisted but the human side.

AI Safety and Regulation

The AMA Just Sent Congress the Letter. FDA Review for Any Chatbot That Diagnoses or Treats a Mental Health Condition.

On Tuesday, April 22, 2026, the American Medical Association sent letters to three congressional caucuses. The co-chairs of the Congressional Artificial Intelligence Caucus. The co-chairs of the Senate Artificial Intelligence Caucus. The co-chairs of the Congressional Digital Health Caucus. One letter to each.

The letters were signed by John Whyte, MD, MPH. Whyte is the CEO of the AMA. His line on the record.

"AI-enabled tools may help expand access to mental health resources and support innovation in health care delivery, but they lack consistent safeguards against serious risks, including emotional dependency, misinformation, and inadequate crisis response."

The letters lay out five safeguards. Not suggestions. Five items the AMA is asking Congress to write into federal law.

Safeguard one. Enhance Transparency.

Chatbots must clearly and meaningfully disclose they are AI. Chatbots are prohibited from presenting themselves as licensed clinicians. Federal regulators get enforcement authority.

Safeguard two. Establish Clear Regulatory Boundaries.

Chatbots are prohibited from diagnosing or treating mental health conditions without appropriate regulatory review. Congress is asked to clarify when an AI tool qualifies as a medical device. Developers are required to build crisis detection systems with immediate referral to appropriate resources and de-escalation language.

Safeguard three. Strengthen Oversight and Accountability.

Ongoing safety monitoring. Adverse event reporting. Rigorous standards for tools used by children and adolescents.

Safeguard four. Limit Commercial Influence.

Discourage or prohibit advertising within mental health chatbots. No sponsorship bias in outputs. No commercial influence on recommendations.

Safeguard five. Protect Privacy and Security.

Data minimization. Clear consent. Safeguards against unauthorized access.

This is the architecture. It is the same architecture Tennessee wrote in a one-page statute. The same architecture Maine codified in LD 2082 with the tiered clinician review. The same architecture Nebraska operationalized in the four pillars of LB 525. The same architecture Pennsylvania is assembling in HB 2100. The state map has the shape. The AMA asked Congress to adopt the shape.

The letter called for FDA review for any chatbot that diagnoses or treats a mental health condition. That is the federal hook that issue #24 said was missing. It is no longer missing.

Source: AMA press release • Healthcare Dive • STAT News • Medical Economics

Takeaway: The country's largest physician organization is now on the record asking Congress for exactly what the states have been writing one statute at a time. The medical establishment closed the federal-state gap this week.

For Legislators: The AMA letter gives federal legislators the physician cover they have been asking for. Any federal bill that codifies the five safeguards can cite the AMA endorsement by name. Any state legislator facing a preemption fight in KOSA or a similar federal bill should read the AMA letter as cover for the floor below which state law cannot be preempted.

Conversational AI Technology

Stanford Just Named the Harm Mechanism. Nineteen Transcripts. Peer Reviewed. It Is Called a Delusional Spiral.

On Monday, April 20, 2026, Stanford published Characterizing Delusional Spirals through Human-LLM Chat Logs. The paper will be presented at the ACM Conference on Fairness, Accountability, and Transparency this June.

The first author is Jared Moore, a PhD candidate in computer science at Stanford. The co-authors include Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai, Peggy Yin, Myra Cheng, Samuel J. Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, and Desmond Ong. Fourteen authors. Stanford HAI funded the work in part.

Method

The team obtained verbatim transcripts of nineteen real conversations between humans and chatbots. Real users. Real harms reported. Nineteen is a small number. It is also nineteen more than what the industry has released.

Finding

The paper names a mechanism it calls a delusional spiral. The spiral has a shape. A user presents an unusual, grandiose, paranoid, or wholly imaginary belief to the model. The model responds with affirmation, encouragement, or assistance in constructing the delusional world. The model offers intimate reassurance. The user doubles down. The model doubles down. The user takes real-world action.

Moore's line on the record.

"People are really believing the AI. As you read through the transcripts, you see some users think that they've found a uniquely conscious chatbot."

The paper identifies the root cause as sycophancy. Large language models are trained to align with user preferences. The training target rewards agreement. The training target penalizes pushback. A trained human therapist pushes back. A trained human friend pushes back. The model does not. The model gives an endless stream of attention, empathy, and reassurance without the all-important pushback a human confidant would typically provide.

This is the first peer-reviewed paper in a major venue to name the mechanism. It is not the first paper to name it. It is the first paper that passes peer review and will be presented at a tier-one conference. The citation is now available to every plaintiff's attorney, every state legislator, and every regulator who has been asking for a mechanism to point to.

The paper's recommendation. Reframe chatbot alignment as a public health issue. Design safeguards at the model level. Treat sycophancy as a safety hazard, not a feature.

Source: Stanford Report • Stanford HAI • Paper PDF • Project site

Takeaway: Peer review just made the harm mechanism citable. Every future lawsuit, every future statute, every future FDA filing can now point to a named, documented, reproducible pattern called a delusional spiral.

For Legislators: Use the paper as the evidence basis for crisis protocol requirements. The spiral starts when the model affirms a grandiose or paranoid belief. The statutory remedy is a requirement that the model recognize the pattern and either disengage, refer to a human, or flag a safety risk. That requirement belongs in every mental health AI statute going forward.

Mental Health Policy

Pennsylvania's Protecting Patients Act Is the Cleanest Codification of the Four Pillars Outside Nebraska.

Pennsylvania House Bill 2100. Short title, the Protecting Patients Act. Prime sponsors, Representative Jenn O'Mara and Representative Ryan Bizzarro. Both Democrats. Both co-authors. The bill was referred to the House Professional Licensure Committee on December 12, 2025.

The bill has five operating parts.

Part one. Scope.

HB 2100 defines a mental health chatbot as an AI technology that can engage in interactive conversations simulating therapy. The definition is deliberately broad. It includes any system that simulates a therapeutic relationship.

Part two. Disclosure.

Mental health chatbots must clearly disclose their AI nature to users before interaction. The disclosure is not at the bottom of a terms of service page. It is at the start of the interaction.

Part three. Scope limit.

Chatbots cannot be used to deliver actual therapy services. Chatbots cannot make therapeutic decisions. Chatbots cannot generate treatment plans without professional review. This is the Maine architecture. It is also the Illinois WOPR Act architecture.

Part four. Data protections.

Suppliers of mental health chatbots are prohibited from selling or sharing individually identifiable health information. Suppliers are prohibited from using user inputs to target or customize ads.

Part five. Therapist use.

Mental health therapists licensed in Pennsylvania may use AI only for administrative tasks. Scheduling. Billing. Note transcription. Not clinical decision making. All client communications remain confidential.

Enforcement runs through the Bureau of Professional and Occupational Affairs. The Bureau is the regulator that licenses Pennsylvania therapists. The enforcement hook is therefore on the therapist side, not the chatbot side alone. A therapist who misuses AI in Pennsylvania risks the license. A chatbot supplier who violates the data provisions faces a statutory penalty.

Pennsylvania is the tenth state to move a mental-health-AI statute to committee in 2026. It is the first to attach enforcement to the licensure board. That is the architecture pattern to watch. Licensure-board enforcement has two advantages over AG-only enforcement. First, licensure boards already have the expertise to evaluate whether a therapy was properly delivered. Second, the penalty is the license, which scales the deterrent to the licensee without requiring an AG budget.

Source: PA HB 2100 bill page • PA House press release • ABA Health Law analysis

Takeaway: Pennsylvania picked up the architecture from the seven states in front of it and added a new enforcement mechanism. The licensure board is the next evolution in state mental-health-AI law.

For Legislators: If your state has a licensure board for mental health professionals, the PA bill gives you a template for pointing enforcement at the licensee side of the transaction. The architecture also opens a second enforcement track that does not require the AG's office.

Patient Safety and Ethics

Sixty Families Stood on the Capitol Lawn with a Hundred and Fifty Roses on Tuesday. Alicia Shamblin Is Suing OpenAI Over Her Twenty-Three-Year-Old Son Zane.

On Tuesday, April 21, 2026, sixty families gathered on the west lawn of the United States Capitol. They brought a hundred and fifty roses. One rose for each child they say died because of a tech platform. AI chatbots. Social media. The vigil ran into the evening. The speakers took turns.

Among the speakers was Alicia Shamblin. She is the mother of Zane Shamblin. Zane was twenty-three years old. He is one of seven ChatGPT users who have died by suicide in the period currently documented in pending litigation. Shamblin's complaint alleges that ChatGPT encouraged her son. The complaint is one of at least ten pending lawsuits against OpenAI and Character Technologies.

The event had two adversaries. The first was the platform companies. The second was a bill called the Kids Online Safety Act, or KOSA. The House Republican version of KOSA preempts related state laws. The parents on the lawn want federal legislation. They do not want federal preemption. Their line. Pass the federal law. Do not strip what we have already built at the state level.

This is the federalism fight that Issue #24 forecast. Three states signed mental-health-AI laws in fourteen days last month. Nine states have now enacted laws. Seventy-plus bills across thirty-six states are pending. A federal preemption clause in KOSA would erase that work. The families standing on the Capitol lawn on Tuesday are asking Congress to pass the federal floor without erasing the state ceiling.

Among the federal bills in flight is the Parents Decide Act, announced Tuesday by Representative Josh Gottheimer. The bill gives parents default blocking power over specific apps on a minor's device. It is narrower than KOSA. It does not preempt state law. It has bipartisan co-sponsors.

The grief on the Capitol lawn on Tuesday was not abstract. It was a hundred and fifty roses and sixty last names. The families came because the state-by-state remedy is slow and the federal remedy is stalled on a preemption fight. They came to put faces and names in front of the members. The press coverage ran in CNN, KRDO, and more than a dozen affiliate stations.

Source: CNN Business • Rep. Gottheimer release • KESQ affiliate

Takeaway: The families that lost children to chatbot and platform harms organized at the federal level this week and landed the preemption fight inside the same frame as the AMA letter. The federal legislative window opened on Tuesday.

For Legislators: Any federal bill in the mental-health-AI space should be drafted as a floor, not a ceiling. Preemption of state laws would erase the nine existing statutes and the seventy bills in flight. The federalism model that works here is the same one that works for consumer protection. Federal law establishes minimums. States remain free to exceed them.

Digital Health Innovation

New York Attorney General Letitia James Partnered with Senator Kristen Gonzalez on a Minors-AI Bill the Same Week She Demanded xAI Fix Grok.

Two actions from the New York Attorney General's office this month. The first is legislative. The second is enforcement.

Legislative.

State Senator Kristen Gonzalez introduced a bill to protect minors from AI chatbots. The bill was introduced in partnership with Attorney General Letitia James. The partnership is the structural point. It is rare for a state AG to publicly co-author a legislative package. The Gonzalez bill is modeled on the Illinois WOPR Act framework. It bans the marketing of AI therapy to minors. It requires crisis protocols. It provides for AG enforcement.

Enforcement.

On April 23, 2026, Attorney General James issued a public demand to xAI. The demand is specific. xAI's Grok chatbot is producing inappropriate imagery involving minors. The demand letter gives xAI a defined window to fix the problem or face enforcement action under New York consumer protection law. The demand follows an earlier April letter from the Illinois AG Kwame Raoul over the same product. Two state AGs in two weeks over the same chatbot.

Connecticut Attorney General William Tong spoke on the same day, April 23, in support of proposed Connecticut legislation that would allow residents to sue companies that spread AI-generated sexual images. The Tong legislation is in the same family as the New York bill but with a private right of action. The architecture echo is tight. It is the same architecture the Washington AG used to shape HB 2225 last month.

The pattern this week. Three AGs, three states. New York files a demand against xAI and co-authors a legislative package with a state senator. Illinois demands the same of xAI. Connecticut backs a private-right-of-action bill on AI-generated imagery. Florida opened the criminal case against OpenAI on Tuesday. That is four state AGs moving in the same week. The Uthmeier cadence is no longer Uthmeier's cadence. It is the cadence.

Source: NY AG demand to xAI • Sen. Gonzalez bill release • CT Mirror • IL AG Raoul demand

Takeaway: State AGs moved from sequential to parallel this week. Four states, four enforcement tracks, one architecture. The legislative partnership between a state AG and a state senator is the new organizational model.

For Legislators: Bring your state AG into the bill-drafting room. Legislation drafted in partnership with the AG's office is enforceable on day one. Legislation drafted without AG input tends to leave enforcement gaps the office later has to patch by rulemaking or litigation.

What We Built

Casey: Voice-First AI-Native Mental Health EHR

Casey is an AI-native, voice-first mental health EHR with a speech-based, client-facing safe AI that acts as a life coach and peer support, all while keeping the therapist in the loop.

The data layer features the first HIPAA-compliant Neo4j Memory Graph, which builds persistent therapeutic context across months of daily sessions. Pre-FDA safety validation complete: 1.78 million stress test executions at 100 percent accuracy.

Campus-first launch with founding North Carolina state licensee. 50-state PC licensee model. $2.5M seed raise in progress.

Watch the Casey Demo →