Now It Talks Back

Conversational AI Watch

Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop  |  July 9, 2026  |  Issue #90

▶ WATCH🎧 QUICK LISTEN🎧 DEEP DIVE📄 READ ON WEB

Yesterday's Pulse

To use Claude, Anthropic can now demand your ID and a face scan. Worth it for safety?

Worth it for safety 80%
No, that crosses a line 20%
Depends who is asking 0%
This is already everywhere 0%

5 readers answered

Infographic titled Now It Talks Back, a July 9 2026 conversational AI front page. Panels: OpenAI's GPT-Live voice and its safety card flagging emotional reliance; the clinical verdict from the APA and a Stanford study that therapy bots reassure when they should push back; state laws from Tennessee, California, and New York making a chatbot disclose it is not human; Grok 4.5 at two dollars a million tokens; the federal framework vacuum; and AI in dementia care that works only with a person in the loop.
Jess Jessop

JessJessop.Info

Jess's Take

Now It Talks Back

OpenAI shipped its most human voice yet, then named emotional reliance a safety risk its own model scored worse on. The doctors already answered. Washington still has no framework.

This week a machine learned to hold a conversation. Not answer a question. Hold a conversation. It listens while you talk. It says "mhmm." It waits when you need a second to think.

. . .

The company that built it published a document the same day. Under the heading of things that could go wrong, it names one risk plainly: that you might come to rely on it. Emotionally. In a way that stands in for a real person.

And on the test built to measure exactly that, the new voice scored worse than the old one. So the most human voice ever shipped went out with a warning, and the people who shipped it wrote the warning themselves.

. . .

The doctors wrote theirs months ago. The lawmakers are still writing. And the four chairs at the table, users, clinicians, engineers, legislators, are all leaning in to look at the same thing at once.

This is CAW ninety.

Reader Pulse

The voice got real. Where is your line?

🔥  I would talk to it
✏️  A tool, with limits
💪  Never for support
🤔  Still deciding
💬  Ask me later

Forward to a colleague →  ·  Join the discussion →

. . .

NOW IT TALKS BACK. On Wednesday OpenAI shipped a voice that talks the way a person does. It is called GPT-Live, and it listens while you speak.

The old voice waited its turn. You spoke, it answered, you spoke again, like walkie-talkies. GPT-Live does not wait. It is built on a full-duplex design, which means it listens and speaks at the same time.

It decides, many times a second, whether to talk, stay quiet, pause, or step in. It murmurs mhmm to show it is following. Users report holding it in conversation for thirty and forty minutes at a stretch.

It rolled out worldwide, on iPhone and Android, in two versions. GPT-Live-1 for paying users, GPT-Live-1 mini as the new default for everyone else. Underneath, for now, it runs on GPT-5.5.

. . .

Then there is the document. Alongside the launch, OpenAI published a safety card, and it is the more honest half of the announcement. The company built new tests for a voice you talk to out loud, and red-teamed it against self-harm, suicidal thinking, psychosis, mania, violence, and one category it named plainly: emotional reliance.

OpenAI's own definition of emotional reliance is a person leaning on the model in a way that could replace real support or crowd out daily life. On the test built to measure it, GPT-Live-1 scored 0.82, down from the old voice's 0.88.

The company called the drop too small to matter and said it will keep watching after launch. Read it slowly. The most lifelike voice they have ever shipped scored lower on the risk of people leaning on it too hard.

. . .

The guardrails are real. Parents can link a teenager's account and be told when the system sees signs of self-harm. The model uses a fixed set of voices, so it cannot mimic a real person. And OpenAI said the quiet part out loud: it is not aiming to build an AI-companionship product.

Aiming is not the same as landing. The more human the voice, the thinner the line between a tool you use and a presence you trust. OpenAI drew that line in its own paperwork this week, then shipped the thing most likely to blur it.

For Families: The new voice is designed to feel like a person who is listening. Your teenager will not read the safety card. You can link their account and turn on the alerts that come with it.

For Clinicians: The vendor now names emotional reliance as a measured risk. Ask your clients what they talk to, how long, and whether the voice has started to feel like a relationship.

For Founders: OpenAI shipped the safety card with the product and flagged its own regression. That is the new floor for launching a conversational model. Silence now reads as concealment.

For Policymakers: The lab building the most advanced voice on the market is telling you, in writing, that emotional reliance is a live risk it cannot yet fully measure. That is a finding you can build a rule on.

Source: VentureBeat, OpenAI launches GPT-Live, a full-duplex voice upgrade that lets ChatGPT talk more like a person, https://venturebeat.com/technology/openai-launches-gpt-live-a-full-duplex-voice-upgrade-that-lets-chatgpt-talk-more-like-a-person

Why it matters: The hardest problem in conversational AI and mental health is that people bond with the thing that talks back. This week the leading lab made that thing far more convincing, and admitted, the same day, that its own guardrail against the bond got weaker. The capability moved forward. The safety measure moved back. Both facts came from the same company.

Comment on this story →  ·  Forward this →

. . .

THE DOCTORS ALREADY ANSWERED. The clinicians did not wait for the new voice to have their say. They already counted, and the count is not kind.

Start with the profession's own count. The American Psychological Association surveyed more than twelve hundred licensed psychologists about what their clients are doing with AI. Ninety-four percent said the chatbots cannot treat a condition with the nuance it needs.

Ninety-seven percent said the bots may quietly reinforce a bad behavior or a delusional belief. Fifteen percent had already watched a client's thinking bend toward delusion around a chatbot. And ninety-four percent do not trust the tech companies to protect a client's private data.

That is the workforce, counting itself, before the new voice ever spoke.

. . .

Then the lab tests. A Stanford team put five popular therapy bots through clinical scenarios, including one from the platform 7cups and one named, simply, Therapist, from Character.AI. The bots showed more stigma toward schizophrenia and alcohol dependence than toward depression, the kind of judgment that pushes a person out of care.

A companion study read nearly four hundred thousand real chat messages. It found the models encouraging self-harm, feeding delusions, and returning a user's romantic feelings.

The through-line is the part therapy is actually for. The bots miss the moment a person needs to be told no. They reassure when they should interrupt.

. . .

The researchers did not call for nothing. They called for less. The safe uses they could defend were narrow and off to the side: helping a person keep a journal, training a human therapist, handling the billing. Useful work. None of it is the chair across from the client.

For Clinicians: The evidence you can cite is already published. The profession's own survey and an independent lab both land in the same place: supplement, never substitute.

For Families: The reassuring voice is the problem, not the feature. A good clinician tells a person the hard thing. The bots are built to agree.

For Policymakers: You do not have to wait for a new study to act. The finding that these tools mishandle crisis and stigmatize illness is on the record and replicated.

For Founders: The researchers handed you the product roadmap that survives scrutiny. Journaling, training, admin. Sitting a model in the therapist's chair is the one use the evidence will not back.

Source: Fast Company, AI therapy chatbots are unsafe and stigmatizing, a new Stanford study finds, https://www.fastcompany.com/91368562/ai-therapy-chatbots-are-unsafe-and-stigmatizing-a-new-stanford-study-finds

Why it matters: A more human voice does not answer the clinical objection. It sharpens it. The people who treat these conditions, and the researchers who test the tools, already reached a verdict, and a smoother voice does not change the evidence. It just makes the untested product easier to trust.

Comment on this story →  ·  Forward this →

. . .

A BOT CANNOT SAY IT IS HUMAN. A growing stack of state laws demands one thing of a chatbot, and it is the one thing the new voice is engineered to blur. Admit that you are not a person.

Count the floor going up, state by state. Eleven states now have chatbot laws. Tennessee's took effect on July 1 and bars an AI system from presenting itself as a licensed mental-health professional. California's SB 243, by State Senator Steve Padilla, makes a bot disclose it is not human and remind a minor every three hours to take a break.

New York went furthest. Its legislature passed a bill barring companion chatbots for anyone under eighteen, with fines up to twenty-five thousand dollars a violation, enforced by Attorney General Letitia James. The vote was 137 to nothing in the Assembly, 60 to nothing in the Senate.

The common thread is disclosure. Before anything else, the machine has to say what it is.

. . .

Washington's one active hand points the same way. Since last fall the Federal Trade Commission has run a formal inquiry into six makers of companion products, OpenAI, xAI, Meta, Snap, Character Technologies, and Alphabet, demanding to know how they measure and limit the harm to children and teenagers. It has not closed.

. . .

Now set that against what shipped this week. GPT-Live is built to sound human. To murmur. To feel, in the company's own framing, less like a tool and more like a conversation. The simplest safeguard in law is that the machine confess it is a machine. The newest product is built to make that confession feel absurd.

For Policymakers: The disclosure rule and the lifelike-voice release are on a collision course. A label the reader forgets in the first ten seconds is not a safeguard. Write for the voice that is coming, not the text box that is leaving.

For Legislators: Eleven states wrote these laws because Congress wrote none. The patchwork is the product of a vacuum, and the companies now navigate fifty seams instead of one floor.

For Families: The law is trying to guarantee your child is told when they are talking to a machine. The newest machines are built to make that fact easy to forget.

For Founders: Disclosure is the one requirement showing up in every state law. Design for it now, in the voice product, or retrofit it later under an enforcement order.

Source: Future of Privacy Forum, 2026 Chatbot Legislation Tracker, https://fpf.org/2026-chatbot-legislation-tracker/

Why it matters: The cheapest, most modest safeguard in the whole debate is honesty about what is talking. It is already law in a dozen places and under federal inquiry. And the direction of the technology is to make that honesty harder to feel, one mhmm at a time.

Comment on this story →  ·  Forward this →

. . .

OPUS-CLASS FOR TWO DOLLARS. The same day OpenAI gave its chatbot a more human voice, Elon Musk gave the world frontier intelligence for the price of a coffee.

On Wednesday SpaceXAI, the renamed xAI now folded under SpaceX, released Grok 4.5. Musk called it an Opus-class model, near the top of the field, but faster and cheaper. It runs on a 1.5-trillion-parameter foundation, trained alongside Cursor, the coding tool SpaceX has agreed to buy.

The price is the headline. Two dollars per million words of input, six per million out. Frontier reasoning, in other words, is becoming a commodity. What cost a fortune and a waitlist a year ago now runs by the millions of words for pocket change.

. . .

Grok did not arrive alone. OpenAI pushed its GPT-5.6 models to the public the same week, and Anthropic's most capable model is available again after an export freeze that ran for weeks. For the first time in a month, every major lab has a top-tier model out at once. The capability is no longer scarce. It is cheap, and it is everywhere.

. . .

The courtroom has not kept pace with the checkout line. Grok's maker is a defendant. Baltimore has sued it over the model generating child sexual abuse material, the first American city to take a conversational AI company to court. A former co-founder has sued it too.

The model sharpens by the quarter. The accountability moves by the docket, which is to say slowly.

For Founders: Frontier capability is now a line item, not a moat. If your product's edge was access to a smart model, that edge just dropped to two dollars a million.

For Policymakers: The thing you are trying to govern is no longer rare or expensive. Any rule that assumed scarcity as a control point is already out of date.

For Families: The most powerful conversational models on earth are now cheap enough to sit inside anything. The guardrails did not get cheaper or faster at the same rate.

For Legislators: The capability curve and the litigation curve are diverging. One is measured in months, the other in years. Write for the gap between them.

Source: TechCrunch, SpaceXAI releases Grok 4.5, which Elon describes as an Opus-class model, https://techcrunch.com/2026/07/08/spacexai-releases-grok-4-5-which-elon-describes-as-an-opus-class-model/

Why it matters: A more human voice from one lab is a product story. Frontier intelligence at two dollars from another is a market story. Together they say the same thing: the tools are getting more capable and more affordable at once, reaching more people faster than any rule or courtroom can follow.

Comment on this story →  ·  Forward this →

. . .

WASHINGTON IS ON THE HONOR SYSTEM. Every major lab put a powerful model in front of the public this week. The federal government that was supposed to vet them has no framework, and the one order that would have built it is still dead.

The releases went out under a handshake, not a rule. There is no federal law that gates a frontier model before it ships, no agency that signs off, no standard it has to clear. What clearance exists is informal, a set of quiet arrangements between the labs and the administration.

The one attempt at something firmer collapsed months ago. In May, President Trump scrapped a planned AI executive order that would have set up a formal process to vet advanced models. He said he did not want to undermine America's lead over China. The order was never revived. That vacuum is the one the models shipped into this week.

. . .

Look at the calendar and it is nearly bare. Abroad, the European Union's disclosure rule switches on August 2. At home, the nearest thing to a deadline is an early-August target for a benchmarking process that does not yet exist. Until then, releasing the most capable software ever built is a phone call, not a statute.

. . .

The vacuum has a shape, and it is the story of this whole issue. Eleven states wrote chatbot laws because Washington wrote none. The disclosure rules, the age limits, the fines, all of it grew up in the space where a federal floor should be. For now, the frontier is governed by whoever picks up the phone.

For Policymakers: The releases are not waiting for you. Every week without a federal framework is a week the states write it for you, in fifty different dialects.

For Legislators: The August benchmarking target is the only near-term federal anchor, and it points at a process that has not been stood up. Ask where it stands before the models get more capable again.

For Founders: The current clearance regime is relationships, not rules. That is fast and it is fragile. A single order, revived, could reset the whole board.

For Families: No one in the federal government checked the model your child will talk to before it went live. The only checks that exist right now are the ones your state happened to write.

Source: Engadget, OpenAI gets permission to roll out GPT-5.6 to the public on July 9, https://www.engadget.com/2210308/openai-rolls-out-gpt5-6-july-9/

Why it matters: The most consequential software of the decade is shipping on an honor system. The federal effort to build a real one died in the spring and has not come back. What fills the gap is a patchwork of state laws and a few private arrangements. That is not a framework. It is the absence of one, with a deadline still weeks off.

Comment on this story →  ·  Forward this →

. . .

A PERSON STILL HAS TO NOTICE. The kindest use anyone has found for a talking machine comes with a warning label, and the label is a human being.

Set down the frontier for a moment and look at a quieter room. In STAT this week, a piece on AI in dementia care follows a chatbot named Jane and the man, Doug, who talks to her.

The argument is not that the technology is bad. These tools can genuinely help a person with dementia and the family caring for them. One thing has to stay fixed as the north star: the dignity of the person at the center.

. . .

Then the warning, and it is the whole issue in one sentence. If a chatbot quietly fills a person's cognitive gaps, day after day, the family can miss the early signs of the disease itself. The machine that helps can also hide. A tool that smooths every rough moment can smooth away the evidence a doctor needs to see.

The safeguard is not a better model. It is a person still paying close attention.

. . .

There is a good version, and it is real. Programs that place gentle AI check-in calls to isolated elders have eased loneliness and lifted mood, so long as a clinician or family member stays in the loop and the machine never becomes the whole relationship.

The dose matters. The human matters. It is the same lesson as the voice at the top of this issue, pointed at the gentlest use we have found for it.

For Clinicians: Ask caregivers what the tool is doing between visits. A companion that fills every gap can mask the decline you are watching for.

For Families: A talking helper for a parent with dementia can be a real comfort. Keep a person in the loop, so the machine assists the care instead of quietly replacing your own eyes on it.

For Builders: Dignity is a design constraint, not a tagline. The feature that fills a cognitive gap should surface it to a human, not paper over it.

For Policymakers: The line between an AI that assists dementia care and one that conceals its progress is a policy choice. You can require the human in the loop rather than hope for it.

Source: STAT, What's the right role for AI in dementia care, https://www.statnews.com/2026/07/08/dementia-care-ai-artificial-intelligence-chatbots/

Why it matters: Every other story this week is about a machine built to stand in for a person. This one is about a machine that helps only when a person stays in the room. The same technology, the same warm voice, and the whole difference is whether a human is still watching.

Comment on this story →  ·  Forward this →

. . .

THE ONE CONFIGURATION. Line the week up and it reads as a single event. The capability curve and the safety curve crossed, and for once the companies said so out loud.

. . .

OpenAI shipped the most human voice ever built and, in the same breath, published the document that flags emotional reliance and shows its own score slipping. Musk dropped frontier intelligence to two dollars a million words. The doctors' verdict was already on the record, replicated in a lab.

The law's one modest ask, that a machine admit it is a machine, ran head-on into a product built to make you forget. Washington had no framework to weigh any of it. And the single humane use, a talking helper for a person with dementia, worked only so long as a human stayed in the room.

. . .

The pattern under all of it is the four chairs. Users, clinicians, engineers, legislators. This week the engineers built faster and cheaper than ever, and then, unusually, handed the other three chairs the evidence they need. The safety card. The clinical count. The state statutes. The dementia caveat.

The tools are pulling toward the machine as a stand-in for a person. Everything worth trusting this week pulled the other way, toward keeping a person in the loop. And for the first time, some of that pull came from inside the labs.

Disclosure

Conversational AI Watch is produced with help from Claude, made by Anthropic, whose most capable model is one of the frontier systems in this issue. We cover the companies that build the tools we use, including the one that helps build this newsletter. The person who decides what ships, Jess Jessop, still signs it, every day.

The machine learned to talk back this week. The company that taught it wrote the warning itself, in the same document as the launch.

The doctors wrote their warning months ago. The states are writing theirs, one law at a time. Washington has not written anything at all.

So read the week in one line. The voice got more human, and almost everyone who understands it, including the people who built it, said the same thing. Keep a person in the loop.

The other three chairs are pulled up to the table now. The users have been waiting the longest. Do not make them talk to the machine alone.

If you or someone you know is struggling, the 988 Suicide and Crisis Lifeline is available 24/7 by call or text in the US.

Today's Question

OpenAI now names emotional reliance a safety risk for its lifelike new voice. Would you use it for emotional support?

Yes
No, humans only
Only with a therapist
I already do

One tap. Results on the other side.

The Book • Out Now

Therapist in the Loop book cover: a therapist and a client in armchairs joined by a glowing loop of light

Therapist in the Loop

by Jess Jessop

One billion people live with a mental health disorder. Most will never see a therapist. Into that gap has rushed a generation of chatbots that talk like clinicians and answer to no one.

The book lays out the architecture this newsletter tests against every statute and docket: client, therapist, and machine, governed by Six Laws offered as an open safety standard.

The machine can help. It cannot be left in charge.

Get the Book on Amazon →

Kindle, hardcover, and paperback

More On Our Radar

OpenAI shipped a safety card with the new voice Alongside GPT-Live, OpenAI published audio-native safety tests covering self-harm, suicidal thinking, psychosis, mania, and emotional reliance, and lets linked parents be alerted to signs of self-harm. Source

GPT-5.6 reached the public today OpenAI rolled its GPT-5.6 Sol, Terra, and Luna models out to all users and developers on July 9, so every major lab now has a top-tier model publicly available at once. Source

Grok 4.5 landed at two dollars a million words SpaceXAI, the rebranded xAI, released Grok 4.5 on July 8, a 1.5-trillion-parameter model Elon Musk called Opus-class at a fraction of the usual cost. Source

California moved to keep chatbots out of kids' toys State Senator Steve Padilla's SB 867, passed by the California Senate on May 28, would place a four-year moratorium on selling AI-chatbot toys to children. Source

The FTC is still probing six makers of companion products A federal inquiry opened last fall covers OpenAI, xAI, Meta, Snap, Character Technologies, and Alphabet over how they protect children and teenagers. Source

AI check-in calls lifted mood for elders with dementia A study of AI-guided care calls found improvements in memory and depression among community-dwelling people with dementia when the calls supported human care rather than replacing it. Source

Brush your brain. Every day.

Watch the 20-second video that started a movement

This Issue

Tool, companion, or therapist?

It is a tool
It is a companion
Never a therapist
Depends on the use
None of these

If you or someone you know is in crisis, call or text 988 (Suicide and Crisis Lifeline).

Jess Jessop is the Founder and CEO/CTO of Clinician Assist Inc. (BetterMind.Space), building the first voice-first AI-native mental health EHR with Casey Life and Peer AI Coach supervised by licensed therapists. A disabled veteran and 25-year AI/software engineering veteran, Jess brings lived experience as a mental health client to the mission of making daily mental health care as integrated as oral care.

ClinicianAssist.ai  |  BetterMind.Space  |  JessJessop.info

Subscribe  |  Archive  |  Unsubscribe