Doing It Right: The Psychologists, Regulators, and Clinicians Building AI Mental Health Tools That Actually Follow the Science

2026-04-16

Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop | April 16, 2026 | Issue #18

WATCH | QUICK LISTEN | DEEP DIVE | READ ON WEB

JESS'S TAKE

Five stories crossed my desk this week. Every one of them was someone trying to do it right.

Not "move fast and fix it later" right. The real kind. The kind where someone reads the research, designs the guardrails, and publishes the work.

A Stanford psychologist wrote in Nature Reviews Psychology that clinicians belong in AI development from day one. Not advising from the sideline. In the room when the architecture is drawn. Then she proved it. Built the platform. Ran the trial across five universities. Published the rules.

A Swiss clinical AI company published the procurement checklist state regulators have been trying to write for two years. Not lobbying for favorable treatment. Handing buyers the questions to ask their vendors.

JAMA Psychiatry told therapists to stop debating whether clients should use AI. Start asking how. Not endorsement. Intake.

WHO convened 30 experts across six regions. Not for a conference. For a consortium. That is not a white paper. That is infrastructure.

A Frontiers scoping review mapped the full road. Phased evaluations. Co-design with end users. Escalation pathways for acute distress. The road is not missing. It is published.

The science is written. The frameworks are built. The receipts are in.

The only question left is who follows them.

THE SIGNAL / CONVERSATIONAL AI TECHNOLOGY

A Stanford Psychologist Says Build It Right or Do Not Build It. Then She Built It Right.

Xuan Zhao, research scientist at Stanford and CEO of Flourish Science, published in Nature Reviews Psychology this week. The argument should be obvious but is not: psychologists must be embedded in AI mental health development from day one. Not reviewing after launch. Not advising from the sideline. In the room when the architecture is drawn.

Then she backed it up. Flourish is grounded in positive psychology, CBT, and acceptance and commitment therapy. A multi-institutional RCT across Harvard, Stanford, Chapman, University of Washington, and Foothill College enrolled 486 undergraduates. Six weeks later: improved emotional, social, and general well-being. Published.

The tension she names is the one this industry keeps dodging. AI chatbots could democratize access or amplify psychological risk. The difference is not the technology. It is who designed it.

Takeaway: The gap between AI development and psychological science is a design choice. Zhao did not just write about what should happen. She showed what happens when a psychologist leads the build. Flourish is the proof of concept for a process, not just a product.

For Legislators: Require that any AI tool marketed for mental health support include at least one licensed psychologist on the development team. Not the advisory board. The development team. Disclosure is necessary. Expertise at the design table is better.

Sources: Nature Reviews Psychology | Flourish Science

THE SIGNAL / AI SAFETY AND REGULATION

Kyan Health Just Published the Procurement Checklist Every HR Team Should Be Using. The Questions Reveal the Industry.

Kyan Health, the Zurich-based clinical AI companion startup backed by Swisscom Ventures and recognized with the Swiss Economic Award 2025, published a workplace mental health AI safety framework this month. It is not a white paper. It is a procurement checklist. And the questions reveal the shape of the industry better than any regulator has managed.

The framework draws the line every state is arguing about. A purpose-built clinical AI companion operates within a defined scope set by clinical psychologists. Psychoeducation for mild distress. Mood tracking for early detection. Care navigation. Hand-off to a human clinician when the situation requires it. In a well-designed system, the AI knows precisely when to stop.

KAI, Kyan's AI companion, was designed with clinical psychologists. End-to-end encryption. Never shares individual data with employers. Operates in over 40 languages. The clients include Hitachi Energy, Hilti, STADA, and Deutsche Börse. The framework references New York's November 2025 self-harm detection law and EU AI Act provisions as baseline compliance.

The question Kyan is asking procurement teams to ask vendors is the question state regulators keep trying to answer. What does this AI do? What does it not do? Who decides?

Takeaway: Kyan did what regulators have struggled to do. They wrote the questions. A clinical AI companion has a defined scope, clinical oversight, and a hand-off protocol. Anything else is a general-purpose chatbot in a wellness jacket. The distinction is now a procurement question, a compliance question, and a liability question.

For Legislators: Adopt Kyan's procurement framework as a standard for any employer-deployed or state-funded mental health AI tool. Require vendors to answer on the record: clinical scope, oversight structure, escalation protocol, data privacy boundary. Procurement discipline enforced by the buyer is faster than legislation drafted by committee.

Sources: Kyan Health Safety Framework | Swisscom Ventures

THE SIGNAL / MENTAL HEALTH POLICY

JAMA Psychiatry to Therapists: Your Clients Use AI. Start Asking How.

Shaddy Saba from NYU's Silver School of Social Work and William Weeks from NYU's School of Global Public Health published a commentary in JAMA Psychiatry that reframes the entire clinical conversation. Not "Should clients use AI?" That ship sailed. The question is how.

The authors describe AI as "a well-read friend who is sometimes confidently wrong." They tell therapists to approach with curiosity, not judgment. The real insight: AI conversations can surface psychological experiences clients might never volunteer to their therapist. Saba now directs an AI in Social Work and Mental Health seminar at NYU Silver. NPR ran it April 10 across dozens of stations.

Asking "How are you using AI?" is not endorsement. It is intake. And it may surface things clients would never bring up on their own.

Takeaway: 58% of people who use AI for mental health never see a clinician. They are not going to stop using AI. But they might start seeing a clinician if that clinician meets them where they are. "How are you using AI?" is not a tech question. It is a clinical one.

For Legislators: Require continuing education credits for licensed therapists on AI literacy and clinical integration. JAMA Psychiatry just called this a clinical skill, not an awareness topic. States that update CE requirements first will have the most prepared workforce.

Sources: NYU News | NPR (April 10)

THE SIGNAL / DIGITAL HEALTH INNOVATION

Developers Say the Roadmap Does Not Exist. Frontiers Just Published It.

Abu-Mahfouz, AlFehaid, Burqan, and El Arab published a scoping review of reviews in Frontiers in Psychiatry. They mapped the full landscape of AI in mental health care. The finding: most AI mental health models remain proof-of-concept. Limited external validation. Substantial overfitting risk.

Applications span NLP, machine learning, and deep learning across speech, text, and biometric signals. Adaptive CBT, relapse prediction, treatment recommendation. Users value 24-hour access and non-judgmental support. But the reviewers mapped the protocol: phased evaluations starting with feasibility, then acceptability, then hybrid effectiveness-implementation trials. Co-design with end users at every stage.

One gap appeared across every review: escalation pathways for users in acute distress. The road from proof-of-concept to safe deployment is published. The question is who walks it.

Takeaway: The roadmap is published. Phased evaluations. Co-design with end users. Escalation pathways for acute distress. The developers doing it right are following a protocol the evidence community already laid out. The developers doing it wrong have run out of excuses.

For Legislators: Require that any AI mental health tool deployed in clinical or educational settings complete at least a feasibility and acceptability study with the intended population before market access. Proof-of-concept is not proof-of-safety.

Sources: Frontiers in Psychiatry

THE SIGNAL / PATIENT SAFETY AND ETHICS

WHO Convened 30 Experts Across Six Regions. Not for a White Paper. For a Consortium.

The World Health Organization announced on March 20 that it is building a Consortium of Collaborating Centres on AI for Health spanning all six WHO regions. The pre-convening took place March 17 through 19 at TU Delft in the Netherlands, home to the first WHO Collaborating Centre on AI for health governance.

On January 29, over 30 international experts in AI, mental health, ethics, and public policy gathered for a workshop organized by the Delft Digital Ethics Centre. Dr. Alain Labrique, WHO Director of Data, Digital Health, Analytics, and AI, set the frame: AI that interacts with people in moments of emotional vulnerability must be designed with safety, accountability, and human well-being at its core.

WHO is not asking whether AI belongs in mental health. They are building the governance to make sure it arrives safely. That is not a panel discussion. That is an institutional commitment.

Takeaway: The Consortium spans all six WHO regions. Any developer building for global markets now has a framework to meet or explain why they do not. When domestic legislation references international standards, it signals permanence. These are not temporary guidelines. They are the emerging global floor.

For Legislators: Reference WHO's responsible AI framework in upcoming legislation. International alignment strengthens domestic bills and signals to developers that the direction is set. Build on it. Do not reinvent it.

Sources: WHO News (March 20)

WHAT WE BUILT

Casey AI: The Therapist in the Loop Standard

Casey is a voice-first AI-native mental health EHR with Casey Life and Peer AI Coach supervised by licensed therapists. Casey has completed 1.78 million pre-FDA safety validation tests with 100% accuracy, is built for FDA 510(k), and operates under continuous clinician supervision. Every interaction is reviewed. Every crisis is escalated to a human. The client is never alone.

When other platforms skip clinical testing, Casey runs 1.78 million of them. When other platforms let chatbots operate unsupervised, Casey puts a licensed therapist in the loop. That is not a feature. That is the standard.

Watch the Demo →