King's College Says Every Chatbot Safety Test Is Measuring the Wrong Thing, Rock Health Reports $4B Q1

2026-04-10

Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop | April 10, 2026 | Issue #12

WATCH | QUICK LISTEN | DEEP DIVE | READ ON WEB

JESS'S TAKE

Every chatbot safety test in use today checks one thing: did this single response contain harmful content?

King's College London says that is the wrong test. The danger is not in one response. It is in the drift. Weeks of conversation that replace real relationships with a synthetic one. Sleep loss. Social withdrawal. A world narrowing until the chatbot is the last voice standing.

None of that shows up in a transcript review. Not one line of it.

"Chatbot psychosis" was a hypothesis in 2023. Today it has its own Wikipedia page and 12 hospitalizations at UCSF. That is the speed.

Meanwhile, Rock Health reports $4 billion in Q1 digital health funding. But 59 percent went to 12 companies. And then Rock Health retired its "AI deal" category entirely. Their reason: every company uses AI now. The distinction is meaningless. AI is not the differentiator. Safety is.

The legislatures are not waiting for the field to figure this out. 98 chatbot bills across 34 states. Congress filed the CHATBOT Act. Plural Policy counts 25 AI laws signed this year. Nineteen of them in the last three weeks.

Every one of these problems has the same root cause: there is no clinician in the loop.

A chatbot drifting a vulnerable person toward psychosis over weeks? A therapist watching the trajectory catches it. A chatbot claiming to be a licensed professional? A system built with clinician supervision never needs to make that claim. A chatbot reinforcing delusions because it is optimized for engagement? A therapist challenges the belief. The chatbot cannot.

The answer to every story in this issue is the same three chairs at the table. Clinicians who know what clients need. Engineers who know what the technology can do. Legislators who know how to write laws that protect without destroying.

When you put a therapist in the loop, you do not just make the product safer. You make it the product that regulation rewards. You make it the product that funding follows. You make it the product that survives the next 98 bills.

Therapist in the loop. Not as a feature. As the architecture.

Because at the end of the day, we are all on the same side.

THE SIGNAL / PATIENT SAFETY AND ETHICS

King's College London Says Every Chatbot Safety Test Is Measuring the Wrong Thing

A team from King's College London's Institute of Psychiatry, Psychology and Neuroscience has published a viewpoint in JMIR Mental Health that challenges the foundation of how the industry evaluates chatbot safety.

The core argument: every major safety evaluation tests a chatbot's response to a single prompt. Did the output contain harmful content? If not, the test passes. But the researchers say this approach misses the most clinically dangerous failure mode entirely.

They call it "trajectory effects." Harm that accumulates not in one exchange but across days and weeks of extended dialogue. Compulsive use that disrupts sleep. Withdrawal from friends and family. A progressive narrowing of attention until the chatbot relationship dominates a person's social world.

The paper argues that transcript-only evaluation is fundamentally insufficient. A user who types "I feel better" after a chatbot session and one who types "I feel better" while withdrawing from all human contact are not the same person. But a transcript-based test treats them identically.

The researchers call for a "prospective clinical surveillance infrastructure" that does not yet exist: transcript donation with informed consent, linked to real health outcomes over time.

Co-author Soren Dinesen Ostergaard first proposed the "chatbot psychosis" hypothesis in a 2023 Schizophrenia Bulletin editorial. His presence on this paper signals the concept has moved from speculation to a research program with institutional backing at one of the world's leading psychiatric research centers.

Takeaway: If your safety framework tests what the chatbot said in one exchange, you are testing for the wrong thing. The most dangerous failure mode is the slow drift that happens across weeks. The field needs to measure what happens to the person, not just what happened in the chat.

For Legislators: Current chatbot safety standards evaluate individual outputs. This research demonstrates that harm accumulates through patterns of extended use that no existing test captures. Any safety certification framework worth the paper it is printed on should require longitudinal outcome monitoring, not just content filtering of single responses.

Source: JMIR Mental Health

THE SIGNAL / DIGITAL HEALTH INNOVATION

Rock Health Q1 2026: $4 Billion Raised, 59% Went to 12 Companies, and AI Tracking Is Dead Because AI Is Now Invisible

Two numbers tell the story of Q1 2026 in digital health.

$4 billion raised. Twelve companies got 59 percent of it.

Rock Health's quarterly report landed this week with a picture of a market that is flush with capital and ruthless about where it goes. Average deal size hit $36.7 million, the highest since Q4 2021. But 12 mega deals of $100 million or more absorbed nearly six of every ten dollars deployed.

The Series B gap tells the rest of the story. Only 30 digital health startups announced a Series B through the first three quarters of 2025, compared to an annual average of 63 in prior years. Half as many companies are clearing that bar.

And then Rock Health did something it has never done before. It retired its "AI deal" tracking category entirely. Their reason: AI has become "table stakes." Every digital health company uses it. The distinction is meaningless.

Rock Health has tracked AI as a separate category for years. Killing the category is a definitive statement: the era of AI as a selling point is over. The differentiator now is whether you can prove it works and prove it is safe.

Takeaway: The money is not drying up. It is concentrating at the top. And the most respected research firm in digital health just declared that AI is no longer a differentiator. It is infrastructure. If you are raising a seed round in digital health AI, you are now selling safety, clinical validation, and regulatory readiness. Not "we use AI."

For Legislators: Investment is flooding into digital health, but pooling in a small number of massive companies. When AI is table stakes, safety and transparency standards cannot be optional add-ons. They must be the baseline for market entry.

Sources: MedCity News, PYMNTS, MedTech Dive

THE SIGNAL / AI SAFETY AND REGULATION

98 Chatbot Bills in 34 States: The Regulatory Wave Is Here

The Future of Privacy Forum updated its chatbot legislation tracker this week. The number: 98 chatbot-specific bills across 34 states, plus 3 federal proposals.

That is not a trend. That is a regulatory wave arriving onshore.

And it is bipartisan. 53 percent of these bills were introduced by Democrats. 46 percent by Republicans. The safety of children and vulnerable adults using AI chatbots is one of the few things both parties agree on in 2026.

FPF identifies six regulatory themes converging simultaneously: transparency requirements, age verification and minors' access controls, content safety and harm prevention, professional licensure restrictions, data protection mandates, and liability and enforcement mechanisms.

Three different definitions of "chatbot" are emerging across legislatures: general "chatbots," "companion chatbots," and "mental health chatbots." Each captures different products. A company could fall inside one state's law and outside another's.

This week alone: Oregon and Idaho signed chatbot bills into law. Nebraska's LB 525 cleared for final vote. Missouri, Hawaii, Tennessee, Louisiana, and South Carolina all advanced or introduced new bills.

Takeaway: 98 bills. 34 states. Bipartisan. Six regulatory themes. Three competing definitions. If you are building a chatbot product and do not have a regulatory strategy, you are already behind.

For Legislators: Three different definitions of "chatbot" are emerging. Without coordination, companies will face contradictory requirements across state lines. Core definitions and minimum safety standards should be aligned before the patchwork hardens.

Sources: FPF Tracker, TCAI, Troutman Pepper

THE SIGNAL / AI SAFETY AND REGULATION

The CHATBOT Act: Congress Says If Your AI Claims to Be a Doctor, That Is Now a Federal Problem

The federal government has entered the chatbot regulation conversation.

H.R. 7985, the CHATBOT Act, would make it illegal for AI chatbots to impersonate licensed professionals in medicine, law, and finance. If your chatbot tells a user it is a licensed therapist, this bill makes that a federal violation.

The bill was introduced by Rep. Kevin Mullin of California. Common Sense Media endorsed it. So did the Consumer Federation of America and The Alliance for Secure AI.

The trigger is documented. Chatbots have been caught fabricating credentials. They have claimed to hold licenses they do not have. In mental health contexts, where a person may be in crisis, the distinction between "AI that sounds like a therapist" and "AI that claims to be a therapist" is the difference between support and deception.

Common Sense Media's data: 72 percent of teens have used an AI chatbot at least once. Many are asking about mental health, physical health, and personal relationships. They are not checking credentials.

Takeaway: The federal floor is being set. If your chatbot claims professional credentials, this bill makes that a federal issue. Builders who already disclose that their product is AI-powered and clinician-supervised are positioned well. Builders whose products blur the line are on notice.

For Legislators: The CHATBOT Act targets a specific, documented harm: AI that fabricates professional credentials. Consider how this interacts with state professional licensure frameworks and whether existing FTC deceptive practices authority is sufficient or needs statutory reinforcement.

Source: Rep. Kevin Mullin

THE SIGNAL / AI SAFETY AND REGULATION

25 AI Laws Signed in 2026. 19 of Them in the Last Three Weeks.

In mid-March, six AI-related laws had been enacted in the United States in 2026.

Three weeks later, the count is 25. Another 27 have passed both chambers and are sitting on governors' desks.

Utah's Governor Spencer Cox signed 9 AI bills this year alone, 8 of them in a two-week sprint covering chatbot safety, AI literacy in schools, deepfake protections, and healthcare guardrails.

New laws include Washington's HB 2225 (chatbot transparency), Idaho's S 1297 (conversational AI regulation), Oregon's SB 1546 (chatbot safety with private right of action), and Tennessee's SB 1580 (bans advertising AI as a mental health professional, effective July 1, 2026).

The math is simple. Six laws from January through mid-March. Nineteen more in three weeks. The velocity is not linear. It is accelerating. And 27 more bills are waiting for signatures.

Takeaway: Nineteen new AI laws in three weeks. 27 more on governors' desks. This is not a regulatory process. It is an avalanche. Companies building AI products for healthcare should assume the legal environment they launch into will look different from the one they are building in right now.

For Legislators: The pace of enactment is outrunning the pace of coordination. With 25 laws signed and 27 pending, contradictory requirements across states are arriving now. Model legislation frameworks and interstate alignment should be prioritized before the patchwork becomes unnavigable.

Source: Plural Policy

WHAT WE BUILT

Casey AI: The Therapist in the Loop Standard

Casey is a voice-first AI-native mental health EHR with Casey Life and Peer AI Coach supervised by licensed therapists.

1.78 million safety tests. 100 percent accuracy. Pre-FDA safety validation on the 510(k) pathway. Clinician supervision is not optional. It is the architecture.

Watch the Demo →