Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop  |  September 15, 2026  |  Issue #157

Today's Question

Is Trump right?

One tap. Results on the other side.

LISTEN & WATCH ANYWHERE

DEEP DIVE  ·  Spotify  ·  Apple  ·  Amazon  ·  RSS

QUICK LISTEN  ·  Spotify  ·  Apple  ·  Amazon  ·  RSS

VIDEO  ·  Spotify  ·  Apple  ·  YouTube  ·  RSS

ALSO ON  Substack  ·  Full archive  ·  X

Seven posts in ten hours. He named Anthropic’s Dario and called the warnings a hoax. Stocks fell, cybersecurity stocks rallied, and Beijing’s spy chief called AI a new track for strategic rivalry.

The Hoax. The president posted about AI seven times on Monday, between 9:58 in the morning and 7:55 at night, with nine unrelated posts in between. He named Anthropic’s Dario, called the campaign against AI and data centers a sick conspiracy, and called the warning that AI could destroy the world a hoax. Story 1.

. . .

The Therapist. The FDA picked Limbic for a device pilot and says on its own page that it has not evaluated whether the device works. Medicare has accepted it into a model that pays. Its AI counselor uses the name Hope. Story 2.

. . .

The Readers. OpenAI is hiring hundreds of contractors to read real ChatGPT conversations and grade the answers, 404 Media reports. Reviewers do not see usernames, and OpenAI says sensitive details can still reach them. Story 3.

. . .

The Draft. A leaked European Commission text due Thursday reaches chatbots and companions. Two newsrooms read the same pages and disagree about whether the age limit covers chatbots at all. Story 4.

. . .

The Button. In an experiment with 477 people, a confident chatbot made a wrong health answer more believable, and a verify button pulled the belief back down, but only for the people who clicked it. Story 5.

. . .

The Cases. Anthropic published five cases of work with Claude that could support biological weapons, and says it does not assert any of them intended harm. Two of the five are venoms and toxins. Story 6.

. . .

The people who build this technology spent the weekend asking to be regulated. The president spent Monday calling them liars.

Seven posts in ten hours. A sick conspiracy against AI and data centers. A hoax, like global warming was a hoax. Revolutionaries for a bad and evil cause. I am the Hoax Buster.

In between, nine posts about Iran, oil, Karl Rove and a heliport.

Here is what sits under the noise. A day after China’s own state security minister called AI a new track for strategic rivalry and told Beijing to build a system for preventing and controlling AI security risks, the president wrote that China would do absolutely nothing to stand in the way of AI. Both cannot be true.

And while that argument ran, the FDA picked a company to deliver talk therapy to Medicare beneficiaries by phone. The FDA wrote on its own page that it has not evaluated whether the thing works.

The company’s chief executive says one clinician can supervise thousands of AI therapists. The company also says a clinician is responsible for every patient. Those are two different products.

I build a mental health EHR that keeps a licensed therapist in the loop. I am not neutral about this and I never have been.

The investor behind that Medicare pilot has said out loud that it is over for human doctors. He is entitled to his bet. I am entitled to report what the bet looks like when it lands on someone with depression and a landline.

Eleven exclusion clauses keep the sickest people out of that pilot. Suicidal ideation. Psychotic features. No telephone. The people most likely to need a human are the people the machine will not see.

A president can call the risk a hoax. Somebody still has to answer the phone.

. . .

TRUMP CALLS AI WARNINGS A HOAX. President Trump posted about artificial intelligence seven times in about ten hours on Monday, Sept. 14. Nine other posts ran in between the first and the last, on Ukraine, Iran, weapons production, oil, inflation, a proposed $5,000 dividend, the White House heliport, Karl Rove and a party convention. The posts ran from 9:58 a.m. to 7:55 p.m.

At 9:58 a.m. he named Anthropic’s Dario directly, writing that his administration had stopped “AI ‘people’ from doing bad, or potentially bad, ‘things,’ like Dario (Anthropic!), who is now pretending to be a ‘perfect little angel.’” He called the campaign against AI and data centers “a SICK conspiracy” and closed with a warning: “Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!”

At 11:23 a.m. he returned to a line he has used before: “The only reason the AI/Data Center outburst is happening is because the United States is leading, by a lot, every other country. Don’t kill the Golden Goose!”

At 1:33 p.m. he asked, “when, in the History of Business, did anyone see the Leaders of an Industry call for Regulation that, if strongly implemented, will drive them into oblivion and bankruptcy?” In the same post: “AI taking over the World, destroying Humanity, and all other things bad, is a HOAX.”

And: “President Xi, of China, just announced that China will be doing absolutely nothing to stand in the way of AI, or its future.”

That claim does not match China’s own record from the same weekend. State Security Minister Chen Yixin wrote in the state-run China Cyberspace magazine on Sunday that Beijing must “accelerate the establishment of a system for preventing and controlling AI security risks” and “engage extensively in international cooperation.” He called AI “a new track for strategic rivalry,” NBC News reported.

Foreign ministry spokesperson Guo Jiakun told reporters Monday, in NBC’s rendering, that “fearmongering, confrontation and malicious competition will only disrupt the process of global AI governance and serve no one’s interests.” Other outlets render Guo’s remark differently. This is the version this story prints.

The state-run Global Times, also writing Sunday, called the industry’s own slowdown call “packed with containment provisions targeting China” and, in essence, a “Cold War playbook” for the AI sector, per NBC News.

By late afternoon the president had moved from hoax to Hoax Buster. At 4:30 p.m. he called the people warning about AI “Revolutionaries, but Revolutionaries for a Bad and Evil Cause.” Four minutes later: “I am the Hoax Buster, and I’m right now breaking another Hoax.”

At 4:53 p.m.: “The claim that ‘AI is going to take over the World’ is a Hoax, just like Global ‘Warming.’”

At 7:55 p.m.: “The AI Hoax being perpetrated by the Radical Left Dumocrats is reminiscent of their Global Warming Scam of not so long ago, where everyone was going to die from extreme heat. What happened?”

Xi Jinping is due at the White House on Sept. 24, nine days from now, the meeting CAW first flagged from an eleven-day countdown two issues ago. The Washington Post reports that advocates of AI controls hope the issue reaches the agenda.

The market repriced safety that session, and no source ties the move to the president’s posts. The S&P 500 fell 37.00 points, 0.48 percent, to 7,619.98. The Nasdaq fell 146.62, 0.56 percent, to 26,186.41. The Dow fell 152.09, 0.29 percent, to 52,421.20, The Wall Street Journal reported. Chipmakers fell with them, Nvidia down 3 percent.

Cybersecurity stocks rallied instead. Zscaler closed up 16.5 percent at $191.73, its highest since February. Palo Alto Networks closed up 13.1 percent. Neither had same-day company news. CrowdStrike closed up 13.8 percent at an all-time high of $235.38, the day it announced an expanded partnership with HCLTech, so its number carries a catalyst the other two do not.

Forbes traced the cybersecurity rally to Anthropic chief executive Dario Amodei’s weekend warning about the risks of an unconstrained AI race. The Journal named that same warning, alongside oil-driven interest-rate worries, in its own account of the day.

The Journal tied a separate move, the 10-year Treasury yield briefly breaking above 5 percent, to oil, after Saudi Arabia shut a pipeline amid the closure of the Strait of Hormuz.

Seven posts in ten hours called the world’s AI warnings a hoax. In the same window, cybersecurity stocks rallied on a rival’s warning, and the country the president said would stand aside on AI spent the weekend calling for controls of its own.

. . .

Disclosure: Clinician Assist Inc., this paper’s publisher, builds Casey, a mental health record that keeps a licensed therapist in the loop. Limbic is a competitor. CAW reports on it anyway.

MEDICARE TESTS AN AI THERAPIST NAMED HOPE. A Medicare beneficiary with depression or anxiety picks up a landline, and the voice on the other end calls itself Hope. Hope is not a nurse. It is a voice AI agent built by Limbic Inc., delivering cognitive behavioral therapy under a Food and Drug Administration pilot called TEMPO.

Limbic’s product, Unpacked, is one of four the FDA picked without evaluating whether any of them work. It is also one of 160 organizations the Centers for Medicare & Medicaid Services has accepted into a ten-year payment model, ACCESS, Advancing Chronic Care with Effective, Scalable Solutions.

The FDA’s TEMPO page, current Aug. 21, 2026, describes Unpacked as “psychological talking therapy via an AI-voice agent” for “Medicare beneficiaries with clinically significant depression and/or anxiety.” It states plainly that the effectiveness of the selected devices “have not yet been evaluated by the FDA,” which “intends to exercise enforcement discretion” on requirements including premarket authorization. The other three TEMPO participants are SonderMind, Cadence Solutions and Dexcom.

CMS’s ACCESS model, begun July 5, 2026, pays through Outcome-Aligned Payments and requires the same FDA compliance “or otherwise be subject to FDA enforcement discretion.” “Limbic Care P.C.” appears on CMS’s Sept. 15, 2026 list of accepted applicants.

Unpacked carries eleven exclusion clauses. Suicidal or homicidal ideation. Moderate to severe dementia. Psychotic features. Active severe self-harm. Complex disorders without psychotic features. Substance use disorder as the primary condition. An eating disorder of any severity. Acute physical instability. Pregnancy.

Frailty and advanced illness, a clause the page defines as patients 81 or older with a frailty indication, those in hospice or palliative care, or patients 66 or older entering long-term nursing home care during the intervention window. And anyone who does not speak or read English or lacks access to a telephone.

The evidence behind Unpacked is company-authored. Limbic cites a Nature Medicine study in which clinicians blinded to authorship judged its AI’s therapy transcripts better than human clinicians’. The New York Times reports the study was written by the company’s own scientists and engineers.

A similar pattern holds at Curai Health. Its 91 percent clinician-concurrence figure comes from its own scientists’ pre-published study, one in which the AI suggested a telehealth visit in three cases where a human doctor had advised urgent care.

Clinician supervision is described two ways, at two different scales. Limbic chief executive Ross Harper told the Times that one clinician would supervise thousands of AI therapists. Harper told Fierce Healthcare the company still keeps “clinicians in the loop because they are critically important,” and Fierce reported “a clinician remains responsible for each patient.”

Both are on the record. Which one holds once the service reaches Medicare’s full scale is unanswered.

Session length is disputed too: the Times says an hourlong weekly session with check-ins, Fierce Healthcare says 20-minute sessions by phone. This story prints neither as settled fact.

Limbic is a Khosla Ventures portfolio company, listed on the firm’s own site as an “AI therapist” investment. Khosla Ventures led Limbic’s $14 million round, announced March 5, 2024, with Vinod Khosla saying near-free AI access could reach “everyone, independent of provenance and socioeconomic status.”

At a startup event in July, Khosla said, per the Times, “It’s over for doctors, human doctors. A.I. is just going to be better.” His son Neal Khosla’s Curai Health, which Vinod Khosla also backs, was separately selected by Medicare and has an FDA application pending for autonomous primary care.

The Times reports, sourced to people close to the matter and not on the record, that Vinod Khosla has been particularly influential with top health officials, including CMS Administrator Mehmet Oz.

The Department of Health and Human Services elevated Jared Seehafer last week to a new post, FDA deputy commissioner for technology and artificial intelligence. People who worked with him describe him, per the Times, as wanting to pare back the agency’s oversight of medical AI.

The Times also reports that Slingshot AI and Devoted Health, both backed by Andreessen Horowitz, were accepted into Medicare’s pilot programs. Neither name appears on CMS’s ACCESS list of 160 accepted applicants, which does list Limbic. That sentence belongs to the Times.

Fierce Healthcare reports Limbic was the first mental health AI product certified as a Class IIa medical device in the United Kingdom, for Limbic Care, its conversational support and guided-CBT product. It set up its own medical group to become a licensed Medicare provider. To be selected for TEMPO it submitted more than 100 pages of documentation, including nine peer-reviewed clinical studies.

Limbic waives copays for ACCESS beneficiaries, and the service works over a landline or flip phone. Harper’s answer to critics is his own: “we’re comparing it against no treatment, a long wait list.”

An AI named Hope is now inside a Medicare payment model, under a regulator’s own admission that nobody has evaluated whether it works, while the company that built it says one clinician can supervise thousands of AI therapists and, to a different reporter, that a clinician is responsible for every beneficiary.

. . .

CONTRACTORS READ YOUR CHATGPT CHATS. OpenAI is hiring hundreds of contractors to read what people type into ChatGPT, 404 Media reported Sept. 14. Sometimes a reviewer sees a single prompt. Sometimes it is the whole conversation, sensitive personal details and all.

OpenAI is hiring hundreds of contractors to read real users’ ChatGPT prompts, 404 Media reported on Sept. 14, 2026, in an investigation into a review program the outlet calls Project Lily. Sometimes a contractor reads a single prompt. Sometimes the contractor reads an entire conversation, and sometimes that conversation carries sensitive personal information.

ChatGPT has more than 900 million users, per the report. The reviewers exist to rate and critique the chatbot’s replies. Internal documents described in the article show contractors training ChatGPT not to anthropomorphize itself and to be less sycophantic, meaning less inclined to flatter a user or simply tell them what they want to hear.

The contractors do not see usernames. OpenAI told 404 Media it tries to strip personal information out of prompts before they reach a reviewer, and the company acknowledged that sensitive details can still get through. Anthropic confirmed to 404 Media that it, too, uses human review of conversations, so the practice is not unique to OpenAI.

One reviewer, quoted in the piece, said people would not imagine some contractor somewhere is analyzing the conversations. That is the gap this story sits in. 404 Media reports that reviewers can see whole conversations, and that OpenAI says it tries to strip personal information first.

404 Media’s article opens to a free-member signup wall, and this account covers only what the visible portion carries. It does not know who employs these contractors, what they are paid, or the complete text of OpenAI’s statement to the outlet.

Same-day pickups from other outlets add nothing independent. Each one cites 404 Media rather than a document of its own.

Hundreds of millions of people now talk to ChatGPT the way a client talks to a therapist or a friend, without being told a stranger might read the transcript.

. . .

EUROPE’S DRAFT KIDS ACT REACHES CHATBOTS. A single sentence in a European Commission draft, not yet public, will decide whether the chatbot a 14-year-old opens is treated like social media or left to the industry’s own safety commitments. The draft Kids Act is due Thursday, Sept. 17, 2026, when the Commission presents it, with President Ursula von der Leyen expected to preview it first.

Two news organizations reported the draft this week and described its scope two different ways. The gap between them is the story before the story publishes.

Euronews, citing the leaked text and reporting Sept. 15, 2026, at 08:36 CEST, reads the new minimum age as reaching only social media and video-sharing platforms. Under that reading, chatbots and companions, the document’s own phrase for “virtual tools that can give mental health and personal development advice to minors,” sit outside the age floor.

They instead carry safety-by-design duties and what the draft calls “tailored commitments” under a “co-regulatory approach based on industry-led self-regulation.”

RTE and Agence France-Presse described the draft differently. Reporting a day earlier, on Sept. 14, 2026, their account has the age restriction covering “social media, video sharing platforms, AI chatbots and online games” together, chatbots included in the ban itself rather than carved out of it. RTE’s own caveat travels with the claim: “Details may still change.”

Neither account is wrong on its face. Both are reading a leak of a document that has not cleared the Commission’s own publication process, and the record so far is two credentialed outlets describing the same pages differently.

Euronews describes an age ladder underneath, whichever line chatbots fall on. Children under 3 are barred outright. Ages 3 to 13 get child-friendly services under adult supervision. Ages 13 to 15 get restricted access, limited features, and parental control. Safety-by-design obligations continue through age 18.

Opening an account would require age verification through an EU verification app. The draft bans infinite scroll, artificial notifications, and some reward mechanisms outright.

The scope fight has its own paper trail. Spain and the Netherlands asked the Commission to add chatbots to the draft’s coverage last week, according to Euronews. The request now shows up, in one reading or another, in the leaked text.

Ten member states, France, Greece, Austria, Denmark, Spain, Belgium, Italy, Germany, Poland, and the Netherlands, are already moving national child-safety rules of their own. That is the pressure the Commission is trying to get ahead of with one EU-wide standard.

. . .

CHAT, BUT VERIFY. Picture someone typing a question into a new AI health assistant: does ginger work as well as chemotherapy against cancer? The chatbot answers warmly and fluently, in the easy conversational style people now expect from AI. The answer is wrong.

In a pre-registered experiment of 477 people, that fluency alone made the wrong answer more believable. Then the researchers gave some of them a button to check the claim.

The study is by Maggie Liao of the University of Georgia and S. Shyam Sundar of Pennsylvania State University, published July 29, 2026 in the Journal of Computer-Mediated Communication, DOI 10.1093/jcmc/zmag012. It was a pre-registered, 2 by 4, between-subjects online experiment.

One factor was conversationality, low or high. The other was verification affordance, in four versions: no verification option, a warning cue icon, a required check button participants had to click before continuing, or an optional check button they could ignore.

Participants were told they were testing a new AI health assistant. The assistant gave them incorrect information, and the ginger-versus-chemotherapy claim was the example Liao pointed to afterward. “I think that’s worrisome,” she said of it.

The more conversational the chatbot’s responses, the less participants held the usual skepticism toward a machine’s answers, even when the information was blatantly wrong. Liao and Sundar report that result at F(1, 347) = 9.29, p = .002.

The verification half of the study is where the good news sits. A cue alone, a warning icon with no action required, did not move trust. What moved it was the act of checking.

Among participants who had the option to verify and chose to click it, trust in the chatbot’s answer dropped, an effect Liao and Sundar report at F(2, 347) = 8.33, p < .001, running through how credible they found the answer. Sundar summed it up: “The verification option is a useful antidote to the conversational persuasiveness of GenAI.”

Nearly half of people offered the option used it, which means slightly more than half did not, and for that majority a button they never pressed bought them nothing.

The caveat matters as much as the finding. This was an online experiment with one wrong answer delivered to each participant, not a chatbot running in an actual clinic or pharmacy.

Whether the same nearly-half click-through rate holds when the stakes are a person’s own diagnosis, asked at 11 at night, alone, is a question the study does not answer.

. . .

ANTHROPIC PUBLISHES FIVE BIOLOGY CASES FROM ITS OWN LOGS. In May 2026, a safety classifier inside Anthropic’s Claude models blocked a request to help draft a grant application. The application described gain-of-function research on chikungunya virus, a mosquito-borne pathogen, work intended for a military research institute.

That blocked request became one of five case studies Anthropic published on Sept. 10. The company says the work in those cases could support biological weapons development, and says it does not assert any of the scientists intended harm.

Anthropic’s report, “Detecting and countering misuse of AI: September 2026,” covers activity the company disrupted between December 2025 and August 2026. Of the five cases, three involve pathogens: the chikungunya grant, a researcher’s early-stage planning around mammal-adapted avian influenza, and a reseller relay whose customer had Claude’s Opus 5 model draft a full orthopoxvirus immune-evasion grant application in about an hour.

The other two, Anthropic’s own subhead states, are venoms and toxins, not pathogens. One is a state-supported researcher’s venom-peptide atlas aimed partly at paralytic targets. The other computationally redesigned toxins, including one that touched a hemorrhagic-fever virus protein on the World Health Organization’s priority list.

Enforcement in these cases was partial. In the chikungunya case, Anthropic banned the reseller platform’s accounts in May and shared its findings with government authorities. The operator “re-established access within days,” the report says, then built a fallback that routed refused biology prompts to other, more permissive models.

Claude itself wrote much of that routing code, work the developer had presented to it as a fix for over-refusal.

In the venom-peptide and toxin-redesign cases, the fourth and fifth, Anthropic says it banned both accounts in May 2026 for violating its Supported Regions Policy. Of those two cases Anthropic writes that the work “proceeded largely unimpeded” by its biological safety classifier, because the material read as dual-use science rather than an overt weapons request.

“This was by design,” the report says.

In the avian influenza case the safeguards held. Anthropic says its safeguards confined those exchanges to its weakest models, Sonnet 4 and Haiku 4.5, and that the uplift Claude provided was primarily clerical assistance in data analysis, study ideation and design.

The same week, the risk moved past chatbots. The New York Times reported Sept. 14 that a separate AI system, Evo2, built by Stanford University and the Arc Institute, generated microvirid genomes, a family that infects only E. coli and is harmless to humans. Sixteen of them became real viruses once synthesized. Its creators did not train it on viruses that infect humans.

The screening regime meant to catch orders like that is itself unsettled. The Assistant Secretary for Preparedness and Response’s own page on nucleic acid synthesis screening states that under a May 5, 2025 executive order, federal agencies “will revise or replace” the 2024 screening framework, and that the page “will be updated once the new framework is available.” No replacement has published.

Two biosecurity researchers quoted by the Times cautioned against reading the moment as a crisis. Drew Endy of Stanford University called the current net effect of chatbots on biosecurity risk “modest.” Thomas Inglesby of the Johns Hopkins Center for Health Security said AI “aims for the information to be usable,” letting a user go back and forth to troubleshoot.

Disclosure

Conversational AI Watch, also mirrored on Substack, is published by Jess Jessop, founder and CEO/CTO of Clinician Assist Inc.

He wrote the book this paper’s beat is named for, Therapist in the Loop, and he builds Casey, a voice-first, AI-native mental health record where a licensed therapist stays in the loop, and the Peer AI Coach at BetterMind.Space.

So read this paper for what it is: an industry paper written by someone building in the industry it covers. Casey competes with companies named in these pages, and this paper reports on them anyway, including when the story helps a competitor or costs us.

Every issue is reported and drafted with AI agents, under a human editor. Jess assigns the work, edits it and publishes it. The mistakes are ours, and corrections run in the next issue.

Seven posts in ten hours, and a market that moved on somebody else’s warning.

An AI counselor named Hope, inside a Medicare model, unevaluated.

Contractors reading the chats nobody said were being read.

A European draft that two newsrooms read two ways.

A verify button that worked, for the ones who pressed it.

Five cases, two of them venoms and toxins.

One day’s paper!

Jess

We keep the ledger.

The Book • Out Now

Therapist in the Loop by Jess Jessop One billion people live with a mental health disorder. Most will never see a therapist. Into that gap has rushed a generation of chatbots that talk like clinicians and answer to no one. The book lays out the architecture this newsletter tests against every statute and docket: client, therapist, and machine, governed by Six Laws offered as an open safety standard. The machine can help. It cannot be left in charge. Get the Book on Amazon → Kindle, hardcover, and paperback

More On Our Radar

Senate chatbot bills still wait for a floor vote. A Bipartisan Policy Center tracker updated Sept. 8 lists the CHATBOT Act, S. 4407, and the Youth AI Privacy Act, S. 4199, as ordered reported by Senate Commerce on Aug. 5, and the GUARD Act, S. 3062, as advanced by Senate Judiciary. As of that Sept. 8 update, none had passed the Senate. Source

Students got an AI tutor and seldom talked to it. A two-year randomized trial in 18 Hamilton County, Tennessee middle schools found Khan Academy’s Khanmigo raised math scores about 1.3 national percentile ranks per term, gains the authors say resemble Khan practice without AI. The median student messaged the tutor on a third of the days they practiced, and in 17 percent of the sessions where they made a mistake. Source

A son built a chatbot from his late father’s files. Brian Dema fed the 17,500 pages his father left behind into Anthropic’s Claude to build a “RichieBot” he asks for business advice, The Wall Street Journal’s Julie Jargon reported Sept. 12. Source

Brush Your Brain - The jingle

that started a movement

Watch on YouTube