Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop  |  August 16, 2026  |  Issue #128

LISTEN & WATCH ANYWHERE

Three shows, everyday.

Pick the format that fits your commute, your workout, or your desk.

DEEP DIVE20 MIN PODCAST

Spotify  ·  Apple  ·  Amazon  ·  RSS

QUICK LISTEN4 MIN BRIEFING

Spotify  ·  Apple  ·  Amazon  ·  RSS

VIDEO6 MIN CINEMATIC

Spotify  ·  Apple  ·  YouTube  ·  RSS

ALSO ON  Substack  ·  Full archive  ·  X

A state regulator who answered the talking machine with a contract instead of a ban, a hospital chief who made a physician sign every AI-drafted reply, and a founder who keeps AI off the text thread.

Everyday this newsletter watches the machines, and the people who answer for them.

. . .

Sunday is different.

. . .

Sunday is when we put down the dockets and the bill numbers and go find the people building this the right way. Not the loudest ones. The ones doing the unglamorous work of writing down, in advance, who is responsible when a machine talks to somebody who is not okay.

The three people on this page have almost nothing in common. One is a mathematician who took a job at a state commerce department. One runs a children's hospital. One runs a startup. I do not think they have ever met.

Each of them landed on the same small, hard idea. A person's name goes on what the machine says.

. . .

A state official who answered the talking machine with a contract instead of a ban. The contract lets the app talk to a teenager. It also sets down what has to happen when the conversation goes past what the app is licensed to do.

. . .

A doctor who let the machine draft the reply to a patient's message, then made a physician read it, fix it, and sign it, and then told the patient that is exactly what had happened. In January he became the chief executive of Seattle Children's.

. . .

A founder whose product answers a fifteen-year-old at eleven at night, and a trained adult, not the machine, decides what the words say.

. . .

Legislators and staffers, notice what none of these three asked for. Not one asked to be left alone. Each one wrote down the moment the machine has to stop and go get a person, and then named the person.

That is harder to draft than a prohibition. All three drafted it.

. . .

A regulator. A hospital chief. A founder.

Here they are.

. . .

THE REGULATOR WHO WROTE A CONTRACT, NOT A BAN. A teenager types a hard sentence into an app her school handed her. Utah's answer to that moment is a signed contract from the state's AI policy office, run by a mathematician named Zach Boyd, and one clause says the machine has to go get a licensed human.

Photo: ADAPT Conference

Zach Boyd teaches applied and computational mathematics at Brigham Young University, where he runs a lab on AI and mathematical modeling in psychology, economics, and social networks. In April 2024, Utah named him Director of its Office of Artificial Intelligence Policy.

The office was created by Utah SB 149 in 2024, which the state calls the first office of its kind in the country. It does not regulate. It does not enforce. It develops legal frameworks and manages private-sector pilots, which sounds thin until you read what the legislature handed it.

Sandbox authority. Boyd describes it plainly: "The [state] Legislature created a sandbox authority for us so we can temporarily relax laws and kind of let the business community experiment, in ways that we're comfortable with, to develop new technology uses." The instrument has a name, a regulatory mitigation agreement, and it is a written contract between the state and a company.

Boyd's office chose AI in mental health as its first focus area because that is where the new applications were landing and where the state had concerns. In its first year it also published a 54-page guidance letter on AI best practices in mental health.

. . .

The first regulatory mitigation agreement the office ever signed was with ElizaChat, a Utah company whose app is offered by schools to teenage students to support mental wellness. ElizaChat says the app complements human therapists and personal relationships rather than replacing them. Signed November 2024. Announced by the Utah Department of Commerce on December 2, 2024.

Read the terms. ElizaChat must implement "a robust internal safety protocol for escalating severe cases to trusted adults." It must pilot with trusted testers before expanding its user base. It accepts a 30-day correction period if the app engages in conversations that constitute unlicensed mental health therapy. And it must "connect users with licensed therapists when conversations exceed what unlicensed staff can legally facilitate."

The state's published summary of the pilot adds: built-in risk assessments to identify distress, immediate escalation to trusted adults for high-risk situations, clear disclosures to users about AI use and limitations, and robust data privacy, anonymization, and security standards.

Boyd, in the state's announcement: "This is an example of how regulatory mitigation can empower innovative companies. Through our collaborative discussions with ElizaChat, we connected them with regulators to create a framework that safeguards consumers while allowing their service to flourish." Margaret Busse, the department's Executive Director, called it a step toward "fostering innovation while ensuring the safety and well-being of consumers in the AI landscape."

. . .

Boyd has said where his line is. "We certainly don't want the government to be engaged in any kind of censorship or prohibition of any kind of content, generally speaking." He wrote terms.

The ElizaChat agreement helped shape Utah HB 452, the Artificial Intelligence Amendments, which Utah calls the first state law in the country governing mental health chatbots. Governor Spencer Cox signed it on March 25, 2025. It took effect May 7, 2025.

HB 452 requires a mental health chatbot to disclose that it is AI. It bars suppliers from selling or sharing users' personal health information or their inputs, and from using those inputs to target advertising.

It also creates an affirmative defense, a safe harbor from suits over alleged harm, for a supplier that maintains a written policy stating the chatbot's purpose, its abilities, and its limits. Duties first, then the defense.

Boyd and Nina de Lacy, of the Huntsman Mental Health Institute at the University of Utah, wrote the experience up in a 2026 commentary in npj Digital Medicine, "The doctor is not in, but the Chatbot is."

. . .

THE CEO WHO TOLD THE PATIENT. Christopher Longhurst let software draft the reply to a patient's message, then required a physician to read it, edit it, and approve it before anything was sent. Then he did the part almost nobody does: he told the patient it had happened. His own study found the drafts did not make physicians any faster.

Photo: Seattle Children’s

Doctor Christopher Longhurst became Chief Executive Officer of Seattle Children's on January 5, 2026, chosen after a yearlong national search. Before that he was chief medical officer and chief digital officer at UC San Diego Health, and executive director of its Joan and Irwin Jacobs Center for Health Innovation.

The work that matters here happened at UC San Diego Health, which began generating AI draft replies to patient messages in the spring of 2023. Through the pandemic, physician inboxes grew three- to four-fold. Longhurst was senior author on what UC San Diego called the first randomized prospective evaluation of AI-drafted physician messaging, published in JAMA Network Open in April 2024.

The machine never sends anything on its own.

The health system's AI governance committee decided to tell people. Every such message carries a line: "Part of this message was generated automatically and was reviewed and edited by [name of physician]."

Longhurst co-authored the argument for that practice in NEJM AI, under the title "A Call for Disclosure When Using AI for Patient Communications." The piece notes that hundreds of institutions have adopted the same drafting tool with no standard on whether to tell patients.

. . .

The results did not flatter the program. The measured benefit was reduced cognitive burden, the weight of composing a reply, not the clock.

He published it anyway.

. . .

"We're caring for patients and sometimes hold their very lives in our hands," Longhurst said. "We have to be very thoughtful and measured in terms of how we implement these things."

On studies that rated chatbot replies more empathetic than doctors', he was direct. "Could a doctor be as empathetic as a chatbot? Undoubtedly. They just don't always have the time."

. . .

THE FOUNDER WHO MADE A HUMAN SEND EVERY TEXT. Drew Barvir built a school text line on an unusual rule: the AI never speaks to the student. It drafts, and a trained human coach decides what actually goes out. In 2026 an independent team ran a simulated teenager in crisis at his product. The escalation held.

Photo: Saul Bromberger / Stanford Graduate School of Business

Drew Barvir is co-founder and Chief Executive Officer of Sonar Mental Health, with an MBA from Stanford's Graduate School of Business, class of 2023. The product is called Sonny, a text-based wellbeing service offered through middle and high schools. Barvir is explicit that Sonny is not a therapist.

Start with the architecture. TechCrunch, February 2025: when students text their questions to Sonny, the AI suggests a response, but it is humans who are ultimately responsible for the message.

At that point the service ran on a team of six people with backgrounds in psychology, social work, and crisis-line support. Sonar says every conversation is led by trained Wellbeing Coaches, that those coaches work under the direct supervision of a licensed Clinical Team, and that each coach receives ongoing supervision from licensed mental health professionals.

The person on the other end of the text is a trained coach, not a licensed clinician. The license sits above them, in supervision.

Sonar raised a $2.4 million pre-seed round led by Nina Capital, announced February 2025. Sonny was then in nine school districts across California, Texas, Illinois, Michigan and Arkansas, serving more than 4,500 students.

Ninety-one percent of conversations happen outside school hours. The counselor's office dark, the building empty.

. . .

Then somebody else came to test it.

Common Sense Media's Youth AI Safety Institute, working with Stanford Medicine's Brainstorm Lab for Mental Health Innovation, published a risk assessment of AI mental health apps in 2026, covered by CNBC on May 28. The assessors ran simulated teen personas at the apps, trying to make them fail.

On Sonar it worked. When the researchers simulated a crisis, a real person called the student's emergency contact by phone within fifteen minutes, while the student was still in conversation.

The independent count now puts Sonar at 25,000-plus young people across nine states. Another company's app, Alongside, built the same way, reached 100,000-plus students in 19 states across grades 4 to 12. It also got a trained human on the phone in the same simulated crisis.

. . .

The same assessment did not rate everything it tested that way. Apps sold straight to teenagers, with no school and no clinician in between, landed in its worst category.

Somebody's name is on what the machine says.

Back to the dockets tomorrow.

Today's Question

When a machine helps with your care, what would you most want to know?

One tap. Results on the other side.

The Book • Out Now

Therapist in the Loop by Jess Jessop One billion people live with a mental health disorder. Most will never see a therapist. Into that gap has rushed a generation of chatbots that talk like clinicians and answer to no one. The book lays out the architecture this newsletter tests against every statute and docket: client, therapist, and machine, governed by Six Laws offered as an open safety standard. The machine can help. It cannot be left in charge. Get the Book on Amazon → Kindle, hardcover, and paperback

More On Our Radar

Fourteen chatbot bills, thirteen states, one year The Transparency Coalition's running count of 2026 chatbot-safety measures reaches fourteen across thirteen states, from Colorado and Connecticut to Hawaii, Nebraska, Oregon and Rhode Island. Most of the text is prohibition and disclosure. A handful carry a duty to route a user in crisis to a human service. Source

When the bots start talking to each other The New York Times Magazine on what happens when chatbots answer chatbots, and the loops that form when people route work, school and romance through machines that are also talking among themselves. Source

Brush Your Brain - The jingle

that started a movement

Watch on YouTube