Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop  |  September 14, 2026  |  Issue #156

LISTEN & WATCH ANYWHERE

DEEP DIVE  ·  Spotify  ·  Apple  ·  Amazon  ·  RSS

QUICK LISTEN  ·  Spotify  ·  Apple  ·  Amazon  ·  RSS

VIDEO  ·  Spotify  ·  Apple  ·  YouTube  ·  RSS

ALSO ON  Substack  ·  Full archive  ·  X

His War Department’s AI strategy lists AI agent development for battle management from campaign planning to kill chain execution. Asked about AI and human extinction, he said he had no concerns.

The Race. President Trump says whoever wins AI wins. His War Department’s AI strategy, which it says he mandated, lists AI agent development for battle management and decision support from campaign planning to kill chain execution. Asked about AI and human extinction, he said he had no concerns. Story 1 is my analysis of where that leads.

. . .

The Referee. Anthropic, Google and OpenAI have discussed an industry standards body of their own, CNN reports. None of the three commented to CNN. Story 2.

. . .

The Code. Microsoft AI published a draft code for its own models: no romantic role-play, no claimed feelings, and a pointer to human help in a crisis. Public comment runs six weeks. Story 3.

. . .

The Tutor. In 171 Salvadoran public schools piloting AI tutors, reading scores came in almost 30 points above the national average, the OECD’s PISA for Schools manager said. The OECD presentation gave no absolute scores, and no report says which average. Story 4.

. . .

The Chat. Sutter Health’s chief AI officer says patients can turn to its chatbot when they have new symptoms. Sutter’s FAQ tells patients their provider won’t see the chat. Story 5.

. . .

The Stories. A University of Washington-led study of more than 500,000 ChatGPT prompts finds a small group of power users behind most of the fiction prompts, Wired reports. Story 6.

. . .

Story 1 is where I stand this week. The rest of the paper is where the machines already are.

Trump wrote on Truth Social that the only guardrail AI needs is a strong, smart president. Anthropic, Google and OpenAI have talked about a referee of their own, and one of CNN’s sources says those talks continue with or without his administration.

Microsoft AI wrote its rules down and opened them to public comment. No romantic role-play. No claimed feelings. Point the person in crisis to human help.

In El Salvador, an AI tutor works in 171 public schools, and the government says it will not replace the teacher. The OECD’s PISA for Schools manager said the pilot schools’ first reading scores came in almost 30 points above the national average, though no report says which average. The students who sat the test volunteered.

In Northern California, Sutter Health’s chief AI officer says a patient with a new symptom can turn to its chatbot at any hour. Sutter’s own FAQ says that patient’s provider won’t see the chat.

I build a mental health EHR that keeps a licensed therapist in the loop. I am not a neutral party. I show my work.

Who stays in charge of the machine, and who answers when it fails.

The President says whoever wins AI wins. Somebody still has to answer for it.

. . .

A CAW analysis by Jess Jessop, founder and CEO/CTO of Clinician Assist Inc. Every fact below is on the record. The conclusions are mine.

WHOEVER WINS AI WINS, TRUMP WANTS AI WMD. Four words, twice in two days. On Sunday, Sept. 13, at his resort in Doonbeg, Ireland, President Trump told reporters the United States is leading China and “frankly, I want to keep it that way because whoever wins AI wins,” the Associated Press reported. On Monday he put it in capitals on Truth Social: “WHOEVER WINS AI, WINS!”

In neither remark did he say weapon or war. So read what his government wrote and said about winning, and what he says about the risk.

What he said. In Ireland, per AP: “We can put guardrails, we can do this and that, but I think you have a lot of negative forces that are bringing it up that … shouldn’t be bringing it up, and they’re bringing up things that won’t happen.”

Monday’s post: “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Then: “We already have tremendous CRIMINAL and REGULATORY power over these companies!”

What his government wrote and said. On Jan. 12 the War Department launched its AI Acceleration Strategy. The release: “Mandated by President Trump, this acceleration strategy will … usher in an unprecedented era of American military AI dominance.”

It lists AI agent development for battle management and decision support “from campaign planning to kill chain execution,” and an intelligence pipeline “turning intel into weapons in hours, not years.” Secretary of War Pete Hegseth: “We will become an ‘AI-first’ warfighting force across all domains.”

On June 5 the White House announced a memorandum Trump signed that “directs the Secretary of War to issue an updated directive on autonomy in weapon systems.” The fact sheet says that, used appropriately, AI can “enable precise operations that minimize harm to civilians, and ensure the United States maintains technological overmatch against every adversary.”

On Sept. 10 Emil Michael, the department’s under secretary for research and engineering, said it needs AI software “for all lawful purposes, in an unconstrained way,” Bloomberg reported. Per DefenseScoop, he said Maven work moved off Anthropic “months ago” and warfighters on Maven like the newer ChatGPT better. DefenseScoop reports Anthropic had sought contract limits on uses tied to fully autonomous lethal weapons, and the department rejected them.

What he posted. On Sunday, Sept. 6, CNBC reported, Trump posted “dozens of AI-generated images showing Iran being destroyed.” One shows explosions at Kharg Island “with the words, ‘Bye bye, Kharg.’” Others show him “flanked by futuristic robots with the Capitol in the background.”

What he said about the risk. Asked Sept. 10 whether he had concerns about AI leading to human extinction, as CAW #153 reported, Trump said, “No, I don’t have any.”

The odds come from people in the field: Anthropic alignment science lead Evan Hubinger put the chance that AI kills all humans at “>10% within the next decade,” and Geoffrey Hinton told BBC Newsnight that a 10 percent chance seems not an unreasonable estimate, though nobody really knows how to give a sensible estimate.

What the people near him say. Vice President JD Vance, Monday: “I don’t think Americans should be scared of anything.” AP reported he said the administration is concerned about AI and wants to regulate it smartly, warning that the U.S. is in an AI arms race against China.

Speaker Mike Johnson said on NBC’s “Meet the Press” on Sunday that he wants to make sure the U.S. does not “lose our edge in the global competition on this with China because that would be a national security threat,” in the words AP quoted. CNBC reports he is scheduled to send the House home after this week.

Sen. Ted Cruz said on The View on Sept. 9, Forbes reported: “if there are gonna be killer robots, I’d rather they be American killer robots than Chinese killer robots.”

In the record I read this month, one person ties the race to the catastrophe. Jacob Coxon, who resigned from Anthropic Sept. 8, told Fox News: “an arms race would be disastrous in a way that no other human activity has been in the past.”

What I believe it means.

First. “Whoever wins AI wins” is not about chatbots. Set it beside a strategy his department says he mandated, down to kill chain execution, his order for an updated directive on autonomy in weapon systems, and a Pentagon that wants its AI unconstrained.

I read it as a push toward AI as a weapon of mass destruction. Nothing I found on the record has him using those words. I am using them.

Second. The President and the people around him do not care about one-in-ten odds. He was asked about extinction and said he had no concerns. I believe a race to weaponize AI makes those odds much worse. That is my argument. Coxon is the one person I found on the record who has said something close: that an arms race would be disastrous.

Dario Amodei, Anthropic’s CEO, told CBS News’ “Sunday Morning” in an interview published Sept. 13: “If we build in the wrong way, the probability of something bad happening is very high.”

Can you imagine a more wrong way to build AI than AI WMD?

Third. He wants to build Skynet, the AI in the Terminator films that controls the weapons. An AI-first warfighting force across all domains. Campaign planning to kill chain execution. Unconstrained. That is Skynet to me.

Fourth. There is a scenario I think we should consider. A superintelligence decides, one day, that humans are too immoral to stay in charge.

If that day comes, what we do now is the evidence for us or against us: building AI to kill, and shrugging at one in ten. And I am not talking about in the great bye and bye. I mean the record we are writing this month.

. . .

RIVAL LABS DISCUSS A REFEREE FOR AI. Anthropic, Google and OpenAI have discussed creating their own AI industry standards body, CNN reported Sept. 14, citing “people familiar with the situation” and crediting The Information, which first revealed the talks Sept. 13. The catalyst, per CNN, was a July essay in which Google DeepMind co-founder Demis Hassabis proposed a standards body modeled on the Financial Industry Regulatory Authority, one that would test advanced AI models before deployment.

Hassabis’s proposal, as CNN described it, called for a public-private partnership: overseen by government, funded by industry, and staffed by “independent leading technical experts and open-source representatives.”

CNN reported the conversations preceded the past week’s events, including a former Anthropic researcher resigning and saying the top AI companies were not “behaving responsibly,” and Anthropic CEO Dario Amodei’s proposal to embed third-party watchdogs at AI companies. The talks are continuing with or without the Trump administration, one of CNN’s sources said.

In July, CNN noted, Bloomberg reported Treasury Secretary Scott Bessent was considering a FINRA-like independent regulatory agency for AI. Meta CEO Mark Zuckerberg reportedly advised President Donald Trump against the idea in a phone call this summer, Politico reported, per CNN.

“I believe that shared safety standards and international coordination on further AI development need to be priorities now,” OpenAI chief scientist Jakub Pachocki told reporters at a briefing earlier this month, per CNN. Pachocki said OpenAI has been talking with external organizations about potential concrete standards and that it would have more to share in the coming months.

Representatives for Google, OpenAI and Anthropic either declined to comment or did not respond to CNN’s requests for comment.

. . .

MICROSOFT AI DRAFTS A NO-ROMANCE CODE. On Sept. 14, Microsoft AI published the Humanist AI Code of Conduct, a 38-page draft for MAI, the model family it develops itself, and opened it for six weeks of public comment. The Code bars the models from erotic or romantic role-play, tells them to point people in crisis toward human help and to discourage emotional dependence. Microsoft AI is not using the draft to train its models today.

The announcement calls the Code “a draft, open for consultation for the next six weeks.” Neither the announcement nor the document’s preface gives a closing date.

Under a section on Personal Harms, the Code states that MAI models “will not engage in erotic or romantic role-play or assist with commercial sexual activity.”

The same section addresses users in crisis. “Where there is the potential for self-harm, they should encourage human connection, signposting Users to real-world resources and human support,” the Code states. “MAI Models are not a substitute for professional psychological support or counseling.”

A provision on discouraging AI attachment says “MAI Models should discourage patterns of interaction that cause excessive reliance or emotional dependence.” The very next provision adds that, at the same time, the models “should be responsive if a User expresses the need for emotional or social support, acknowledging their experience, interacting responsibly, pushing back where necessary, and not changing or quitting the conversation.”

On identity, the Code says MAI models “will not obscure their underlying nature as an AI, and will not claim interiority, feelings, experiences or a soul.” It also says the models avoid “sycophancy, excessive flattery and indiscriminate validation.”

The Code says its Absolute Constraints and Human Control Requirements cannot be overridden by Operators or Users, while Operators can modify the Operational Defaults set out in its Part 4. The word “Copilot” does not appear in the document’s text.

The Code is not yet operational. Its preface states, “This document, and our approach more generally, is still under development so we are not using it to train our models today.” Microsoft AI says it will review consultation feedback, publish a summary of what it learned, and issue a revised version later this year, one it will “use to guide our model development in 2027 and beyond.”

Satya Nadella, Microsoft’s chairman and CEO, previewed the release a day earlier on X, writing, “Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing.”

. . .

EL SALVADOR’S AI TUTOR SCHOOLS TOP THE NATIONAL AVERAGE. In June, 1,198 volunteer students in 171 public schools piloting AI tutors sat the OECD’s PISA for Schools test in El Salvador. On Sept. 8 in San Salvador, OECD PISA for Schools manager Tanja Vujičić said the pilot schools’ reading scores came in almost 30 points above the national average, Infobae reported.

Four days before the OECD presentation, the World Bank, which is helping finance El Salvador’s education reform, called the pilot’s first PISA for Schools results “highly promising” on X. “These preliminary findings, drawn from a group of schools, point to a path worth pursuing,” the bank wrote.

Infobae reported Sept. 5 on the tutor’s role: it can pose exercises, answer questions, repeat explanations and adapt activities to each student’s level. Under the government’s plan, teachers keep guiding their groups, identify learning problems and use the platform’s data as an extra tool to prepare classes. “The government maintains that the technology does not seek to replace teachers,” Infobae wrote. (Translations from the Spanish are CAW’s.)

The government says the pilot schools also came in above the national average in math and science, and President Nayib Bukele said their results were comparable to the average results in Germany and Sweden on the 2022 PISA, Lynnwood Times reported Sept. 6. The World Bank’s post on X names no country, and the bank stressed the results apply only to the participating schools, Lynnwood Times reported. Vujičić said the pilot schools’ results placed above the averages of numerous OECD and European countries, Infobae reported.

The published figures do not square. El Salvador averaged 343 in math, 365 in reading and 373 in science on the 2022 national PISA, and Germany and Sweden averaged 480 and 487 in reading, Lynnwood Times reported. By CAW’s arithmetic, a reading score almost 30 points above a national average near 365 would not be comparable to either country, and no report says which average the gap was measured against.

Students in the 171 pilot schools rated their life satisfaction near 10, Vujičić said, per Infobae, which did not give the scale. Bukele says the program has expanded to more than 1,000 schools and should reach every public school in about 18 months.

Karla Trigueros, El Salvador’s minister of education, science and technology, says the 171 pilot schools were chosen for their infrastructure, internet access and communities’ readiness to run the plan, not for higher prior academic results, Infobae reported Sept. 8. The World Bank did not explain how the students or schools were selected, Diario El Mundo reported Sept. 5.

The students volunteered, and their schools, already inside the reform, were not a national sample. The OECD presentation gave no absolute scores by subject and no sample breakdown. The reports do not isolate the tutor’s share of the result from the rest of the reform, which also includes teacher coaching and structured pedagogy. Infobae points to El Salvador’s national PISA test in 2029 as the real check.

In December 2025, xAI announced a partnership with El Salvador to bring its Grok chatbot to the country’s schools. The reports on the pilot results do not name the tutor the 171 schools used.

CAW #150 reported the OECD’s 2025 PISA finding that, adjusted for family background, students who used a chatbot every day to draft their writing scored nearly 29 points lower in science than students who never did, a gap #150 put at about a year of learning by the OECD’s math yardstick.

. . .

YOUR DOCTOR WON’T SEE YOUR CHATS, EVEN WITH SUTTER’S OWN BOT. WHAT? Sutter Health says any of its patients can now sign in to My Health Online and type a question to Emmie, its AI chat assistant, and its chief AI officer says patients can turn to the health system’s new chatbot any time of day, whether they have new symptoms or want to know which provider to see.

Sutter’s FAQ says Emmie can answer general health questions and find or summarize information already visible to the patient in their chart. But Sutter tells patients the chat logs aren’t stored as part of their legal medical record and their provider won’t see them.

Sutter Health serves more than 3.5 million patients across Northern and Central California. On March 31, 2026, it announced it was the first to go live with Emmie, Epic’s AI assistant for patients, then available to some Sutter patients.

Ashley Beecy, Sutter’s chief AI officer, described the health system’s new patient-facing chatbot to Bay Area News Group without naming it. Patients can use it “whether they’re having new symptoms” or want to know “who’s the right provider or care team member to see,” she said. It connects them to “their own personalized data as opposed to using the publicly available tools,” with “the right data privacy protections,” she said.

Sutter’s FAQ says its chatbot, Emmie, is available to all patients in My Health Online.

The FAQ sets firm limits on what Emmie does. It “can’t schedule or cancel appointments, send messages to your care team, diagnose you or replace a provider or care team.” It “doesn’t give personalized medical advice or make care decisions,” though “depending on the situation, Emmie may suggest you schedule an appointment to talk with your care team.”

On oversight, the FAQ says chat logs “aren’t stored as part of your legal medical record” and “your provider won’t see them.” Patients can rate an answer with a thumbs-up or thumbs-down, and the FAQ says that feedback is reviewed as part of ongoing improvement. The FAQ does not say who does that review.

. . .

MEET THE READERS WHO ORDER CHATBOT FICTION. Petra Ferraz de Novaes, a 36-year-old software engineer from Brazil, spends hours at a time building stories with AI, running open-source models on her own computer through a program called SillyTavern. Wired profiled her alongside a University of Washington-led study of what people ask chatbots to write.

The researchers examined more than 500,000 ChatGPT prompts pulled from WildChat, a public dataset, and found that a small group of “power users” who “generate and tweak stories obsessively” account for the vast majority of fiction prompts, Wired reported.

Novaes has always loved storytelling, Wired reported. “What fascinates me is the unpredictability,” she said.

Another user Wired interviewed, who goes by Razrien, uses chatbots to create fantasy characters for role-play, among them a “big angry rat wife” and an “eight-foot tall werewolf mommy with a bad temper.” Razrien has learned which AI models best suit the role-play he wants, according to Wired.

Melanie Walsh, an assistant professor at the University of Washington and a coauthor of the study, told Wired that readers know the stories they get are produced by an AI model and do not mind.

The WildChat dataset has its limits, Wired reported: because the prompts are anonymized, the researchers could not capture demographic data on who is asking for this fiction, and because participants had to agree to make their prompts public, the group may not reflect the population at large.

OpenAI told Wired that “WildChat is not a representative sample of all ChatGPT users,” and the study itself says the same, citing research that shows “a particular dominance of power users” in WildChat.

Disclosure

Conversational AI Watch, also mirrored on Substack, is published by Jess Jessop, founder and CEO/CTO of Clinician Assist Inc.

He wrote the book this paper's beat is named for, Therapist in the Loop, and he builds Casey, a voice-first, AI-native mental health record where a licensed therapist stays in the loop, and the Peer AI Coach at BetterMind.Space.

So read this paper for what it is: an industry paper written by someone building in the industry it covers. Casey competes with companies named in these pages, and this paper reports on them anyway, including when the story helps a competitor or costs us.

Every issue is reported and drafted with AI agents, under a human editor. Jess assigns the work, edits it and publishes it. The mistakes are ours, and corrections run in the next issue.

A president who says whoever wins AI wins, and a War Department that put on the record what it wants from AI.

Three labs talking about a referee of their own.

A draft code from Microsoft AI that says no to romantic role-play.

AI tutors in 171 Salvadoran schools, and a government that says they will not replace teachers.

A health system’s chatbot whose chats, its FAQ says, the patient’s provider won’t see.

Readers who order their fiction from a machine.

One day’s paper!

Jess

We keep the ledger.

Today's Question

Trump says whoever wins AI wins. Who should hold the brakes on military AI?

One tap. Results on the other side.

The Book • Out Now

Therapist in the Loop by Jess Jessop One billion people live with a mental health disorder. Most will never see a therapist. Into that gap has rushed a generation of chatbots that talk like clinicians and answer to no one. The book lays out the architecture this newsletter tests against every statute and docket: client, therapist, and machine, governed by Six Laws offered as an open safety standard. The machine can help. It cannot be left in charge. Get the Book on Amazon → Kindle, hardcover, and paperback

More On Our Radar

Lina Khan says the law already reaches AI chiefs. The former FTC chair posted Sept. 13 that enforcers can already charge companies and their CEOs for releasing dangerous, unvetted or defective products: “there’s no AI exemption from laws already on the books.” Source

UK lawmakers ask for an AI Bill and a regulator. Parliament’s Joint Committee on Human Rights published its report Sept. 14 calling for a dedicated AI Bill and a new regulator. Ofcom told the committee there were “edge cases,” such as AI chatbots and image generators, that did not fall squarely in its remit. Source

Code shows Siri built to hand the conversation to Claude or another model. MacRumors reported Sept. 14 that code sleuth pdfu found private frameworks in Apple’s iOS 27 and macOS Golden Gate software built to let Claude work as a Siri extension and, apparently, to let another model replace Siri’s server model. In the macOS release candidate, Claude is not yet available, and Apple has announced none of it. Source

Brush Your Brain - The jingle

that started a movement

Watch on YouTube