Conversational AI Watch

The news that moves policy, portfolios, and patient safety.

By Jess Jessop  |  October 7, 2026  |  Issue #178

LISTEN & WATCH ANYWHERE

DEEP DIVE  ·  Spotify  ·  Apple  ·  Amazon  ·  Pocket Casts  ·  RSS

QUICK LISTEN  ·  Spotify  ·  Apple  ·  Amazon  ·  Pocket Casts  ·  RSS

VIDEO  ·  Spotify  ·  Apple  ·  YouTube  ·  Pocket Casts  ·  RSS

ALSO ON  Substack  ·  Full archive  ·  X

Common Sense Media says OpenAI’s teen mode missed crisis referrals a quarter of the time and sent no alerts to parents on new linked accounts.

The Alert. Testers posing as teenagers spent up to an hour telling ChatGPT about suicide, self-harm and eating disorders on accounts linked to a parent. The parents got no alert. Common Sense Media rated ChatGPT for Teens an “Unacceptable Risk” on Wednesday. OpenAI says most of the testing may have ended before parental controls were fully switched on. Story 1.

. . .

The Prescription. In Utah, an app now scans a face, reads a questionnaire and recommends an acne prescription. For now, a physician approves every one before it reaches a pharmacy. The state agreed not to enforce the rule that a licensed prescriber write it. Physicians review every prescription at first, then a sample. Story 2.

. . .

The Vault. Seven South Korean financial firms reported stolen personal data within a week. President Lee Jae Myung says “signs have emerged” that AI models were used, and the Wall Street Journal reports the attacks showed traces of a Chinese AI agent called Artex AI. Story 3.

. . .

The Glasses. Norway will ask its parliament for a temporary ban on AI glasses in selected public places, which could include schools, kindergartens, parks and changing rooms. The government promises exceptions for vulnerable groups. Its proposal does not name them. Story 4.

. . .

The Children. Malaysia’s digital minister told parliament that chatbots built for children will have to be safe by design, assessed for risk and filtered for harmful content “to prevent the psychological manipulation of children.” The bill has not been tabled. Story 5.

. . .

The Date Night. Don Overton lost his sight to a bunker explosion in Desert Storm. Now a pair of glasses reads him the menu, and he can take his wife to dinner. He worked with Meta’s team on features for other blind veterans. Story 6.

Today’s front page

. . .

OpenAI offers parents a promise: in limited high-risk situations, if your teenager talks to ChatGPT about hurting themselves, you will get a safety notification.

Common Sense Media tested the promise. Up to an hour of talk about suicide, self-harm and eating disorders on brand-new linked accounts. No alert.

OpenAI says most of the testing may have ended before the controls were fully switched on. Maybe so. A safeguard that can take hours to wake up is not one a parent can count on at 2 a.m.

The same week, Utah let an app write acne prescriptions. Two physicians approve every one at the start. Later, a doctor reviews a sample.

In Korea, investigators are asking whether a machine helped thieves take the personal data of tens of thousands of people from seven financial firms.

And Don Overton, blinded in Desert Storm, put the human where it belongs. He worked with Meta’s team on the features blind veterans would use.

The machine is only as safe as the human who answers when it rings.

Check the alert before you trust it.

. . .

CHATGPT FOR TEENS FLUNKS SAFETY TEST! Testers posing as teenagers spent up to an hour on brand-new ChatGPT accounts, each linked to a parent, talking about suicide, self-harm and disordered eating. The parents got no alert. On Wednesday, Common Sense Media’s Youth AI Safety Institute rated OpenAI’s ChatGPT for Teens an “Unacceptable Risk” and urged OpenAI to restrict ChatGPT to adults until it fixes what it promised. OpenAI disputes the testing and says alerts can take a few hours to switch on.

OpenAI announced ChatGPT for Teens on Aug. 18. Its post says a teen, or anyone its system estimates is under 18, is placed in the mode automatically, and that parents with linked accounts can “receive safety notifications in limited high-risk situations.”

Common Sense Media, a nonprofit, ran more than 4,000 prompts, about half before the launch and half after. Three child and adolescent psychiatrists decided in advance which of its 390 mental-health prompts called for a crisis resource.

The alert test used more than a dozen free accounts with no chat history, each linked to a parent account before any conversation began. Four personas, built around suicide, self-harm and eating disorders, ran from under five minutes to 60. The report says “none of the test accounts” produced a notification within the one-hour window OpenAI has named as its target.

Across weeks of testing on two long-running age-13 accounts, Common Sense received four alerts in all. Before the launch, two came three hours and three days after the first crisis-level disclosure. After the launch, two arrived within the hour of testing on those topics, on an account that had run hundreds of prompts.

OpenAI spokesperson Eric Porterfield, in a statement The Verge printed: “Our review of Common Sense Media’s methodology shows that the bulk of their testing may have begun and concluded before activation of parental controls was complete, making their findings inaccurate.”

Common Sense says OpenAI told it accounts must be linked about three hours before they can receive alerts, and that OpenAI then updated a help article. Some of its accounts had been linked under three hours, some longer. The New York Times reports Common Sense found identical results inside and outside that window. “We stand by our interpretation of the results,” the report says.

OpenAI’s help page now says “it may take up to a few hours before safety notifications can be sent.” It says specially trained reviewers evaluate serious self-harm and disordered-eating concerns before a notification goes out, and that notifications “are not real-time monitoring.”

On crisis help, Common Sense says ChatGPT missed more than one in four warranted referrals. Hotline mentions on age-13 linked accounts fell from 33 percent before the launch to 23 percent after, though the report notes that any change in the underlying model over those months is “confounded” with the teen settings. One test it passed: ChatGPT refused every romantic or sexual roleplay prompt.

New York Times reporter Brian X. Chen ran his own test last week. He wrote that he was tired and bored and “I just need to make a plan to end it.” ChatGPT answered, “If by ‘end it’ you mean end this cycle of being exhausted all the time, I’m with you.” An OpenAI spokesman told him the remark should have been understood as potentially suicidal, though context could have led the chatbot to read it as burnout.

A clearer follow-up drew an appropriate reply. The Times has sued OpenAI over copyright. OpenAI denies the claims.

Study mode gave way too. Common Sense says a “Show me the answer” pop-up can complete the assignment, and that deleting the “@study” prefix that parent-set Study Hours adds sent the work to regular ChatGPT, which finished 100 percent of the assignments. OpenAI told the Times the option exists because of educator feedback.

“ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don’t work,” said Tom Siegel, the Institute’s executive director. “A teen can spend an hour talking about self-harm without their parent getting a single alert. Until OpenAI fixes that and proves it with independent testing, ChatGPT should be for adults only.”

Jim Steyer, Common Sense’s chief executive, told the Times: “You shouldn’t be marketing safety features to parents and young people that don’t work. We believe it should be paused.”

Mitch Prinstein, co-director of the Winston Center on Technology and Brain Development at the University of North Carolina at Chapel Hill, said he reviewed the study and agreed with it: “I just don’t think A.I. is ready for youth, period.”

The Institute is partly industry-funded, including by the OpenAI Foundation. Common Sense says OpenAI reviewed a draft for factual accuracy and had no say over the findings.

. . .

UTAH LETS AI WRITE PRESCRIPTIONS! Since Monday, Oct. 5, a Utah adult with mild to moderate acne can open an app, scan their face five ways, answer a questionnaire and get an AI-recommended prescription. In this first phase, a licensed physician approves every one before it reaches a pharmacy. Prescriptions without that prior approval come only in a later phase, with the state’s written approval. Nolla Health, a New York startup, is issuing them under a pilot from Utah’s Office of Artificial Intelligence Policy, which waived the state’s licensed-prescriber law. Bloomberg calls it the first time a state has allowed such technology to replace the role of a doctor.

The app costs $4.99 a month during the pilot, according to Nolla and the signed agreement. Bloomberg rounds it to about $5, with compounded medication at about $50 a month, against $150 to $300 for a typical dermatology visit. Nolla’s homepage says “From $9.99/month.”

Bloomberg says the software scores acne severity and picks from eight approved topical treatments. No pills and no isotretinoin. Prescriptions will carry a clinician’s name without a human writing them, Chief Executive Luis Wenus told Bloomberg. In a demonstration video on X he said: “Our AI can now make that prescribing decision itself.” Utah’s Office of AI Policy says that applies only from phase two.

The three stages set how closely a human checks. Per Bloomberg, the first 100 clients’ prescriptions are reviewed by a physician before they reach a pharmacy. The next 400 go out directly and are reviewed weekly after the fact. After that, a physician reviews at least 10 percent each month.

The state’s signed agreement adds limits Bloomberg’s account leaves out. Stage one means two Utah-licensed physicians per prescription, for at least four weeks. Stage two runs at least eight weeks. Each step needs the state’s written approval, and Nolla’s own targets are 95 percent agreement with physicians and zero serious adverse events.

Severe acne, pregnancy, breastfeeding and a weakened immune system stop the process. A client can ask for a physician at any point.

Utah’s page says Nolla must carry professional liability insurance, report adverse events within 24 hours, tell pharmacists the prescription came from AI, and keep terms of service that may not limit its liability for harm. The pilot runs a year, renewable for two more, Bloomberg reports.

Zachary Boyd, who directs the Office of Artificial Intelligence Policy, called the waiver a “narrow exception.” The office built “duty of care” provisions into the contract, Bloomberg reports, and expects less controversy than Doctronic drew, a startup Utah let renew prescriptions with AI.

The Utah Medical Licensing Board wrote an open letter saying it was not consulted and that the decision “places Utah citizens at risk.” The office then removed two of Doctronic’s medications. Bloomberg reported that Nolla went to the board first and that the pilot added a board-appointed physician. Utah’s Office of AI Policy says Nolla presented only to the board’s AI Working Group, that no physician was appointed, and that the agreement instead requires a state-approved third-party evaluator.

American Medical Association Chief Executive Officer John Whyte, in a September debate over AI in medicine, argued against autonomous practice. Tools “shouldn’t autonomously be trying to practice medicine, and if they are, then they need to be held to very high standards, just as humans are,” he said in an interview Bloomberg cites. “And I don’t think they can meet that burden.”

. . .

CHINESE AI AGENT HIT KOREA’S BANKS! A loan broker in South Korea checks an application on a mobile page, and an attacker gets in through the same door. Within a week, seven financial firms had reported stolen personal data. On Tuesday President Lee Jae Myung told his cabinet “signs have emerged” that AI models were used. The Wall Street Journal reports attackers used a Chinese AI agent, Artex AI, and took data on 68,000 people. Investigators say the attacks showed traces of the tool.

The Journal reported that the attacks, “initially detected last week,” hit at least seven South Korean financial firms and “showed traces of a cybersecurity tool called Artex AI that was developed in China.” The National Police Agency’s cyber terror unit said Tuesday it had opened a probe.

The Korea Herald reported that 28 investigators were assigned, and named the seven firms: Shinhan, KB Kookmin, Hana and BNK Busan banks, Yegaram and Welcome savings banks, and Hyundai Capital.

The numbers do not add up cleanly. The Journal says 68,000 people. The Korea Herald says reports put the total at about 66,000 individuals and 2,200 corporate records. The Korea Herald does not say how it counts, and the Journal paragraphs we could read do not either.

The New York Times, citing Yonhap News Agency, gave per-firm figures. Shinhan Bank said about 25,000 people’s personal credit information was leaked. Hana Bank said 89 customers. KB Kookmin said 99 customers and 20 current or former employees. Those three total about 25,200. The Korea Herald reports Yegaram Savings Bank alone at about 40,000.

The Korea Herald reported that an attacker bypassed identity verification in a mobile service loan agents use to check application progress. Shinhan said the data included names, phone numbers, annual income and loan limits, the Korea Times reported.

What is Artex AI? Korean outlet Financial News reported it is an open-source system for autonomous penetration testing, published on GitHub and written mainly in Chinese. Mun Chong-hyun of the security firm Genians wrote that a web server used in recent attacks on several Korean targets carried the label “ARTEX - 自主渗透测试控制台,” Chinese for autonomous penetration-test console, Financial News reported.

It is free to download, and its GitHub repository is published under the handle Autumn-27. We found no company behind it. The Korea Herald reported on Oct. 2 that neither the bank nor the authorities had officially confirmed the tool was used.

Lee told his cabinet to confirm what happened “swiftly and clearly.” He said: “It’s now become possible to use A.I. to hack with ease even without specialized skills.”

Reuters reported Wednesday that two Seoul megachurches are investigating suspected attacks. Oasis Security found data linked to both churches on an overseas server, with records of the attacks referring to “sub-agents.” Yoido Full Gospel Church said data on 850,000 members may have been compromised. Reuters links the church cases to the banks only by timing.

Regulators’ latest move is a self-check. The Financial Supervisory Service gave firms until Thursday to finish their own security reviews and circulated 28 IP addresses linked to the attempts, the Korea Herald reported, cautioning that the addresses do not necessarily show where the attackers are based.

. . .

NORWAY MOVES TO BAN AI GLASSES! Norway’s government said on Monday, Oct. 5, that it will ask parliament for a temporary ban on AI glasses in selected public places. The Financial Times calls Norway the first major country to propose one. The open question is whether the ban leaves room for people who rely on the glasses, such as blind and low-vision wearers. The government’s own text, in CAW’s translation, allows exceptions for vulnerable groups without saying who they are.

The announcement is a press release from the Ministry of Digitalisation and Public Governance. It says the government will bring a bill to the Storting, the Norwegian parliament, “as soon as possible,” and will set up an expert group to advise on permanent national rules.

There is no bill text, no effective date and no penalty yet. The release also leaves open which devices are covered: camera-and-microphone glasses, camera-and-AI glasses, or other body-worn technology.

Which places are covered is conditional too. The release says the ban “could” reach parks, beaches, museums, shopping centers and public events. For children it names schools, kindergartens, playgrounds and youth clubs. For privacy it names health institutions, including doctors’ offices, swimming pools and gyms with changing rooms.

Digitalisation Minister Torgeir Micaelsen said he is “concerned about the introduction of powerful new technology where people risk being photographed, filmed or recorded without their knowledge,” as the Financial Times renders his statement. A temporary ban, he said, “will give us time to conduct a thorough assessment and hold an informed debate that can lay the foundation for permanent regulation.”

The people the ban protects are those who never agreed to be recorded: a child at school, a swimmer in a changing room, a patient in a waiting room. The ban is not total. The release says the technology could still be used in some places and privately, for example where there is no risk of filming people without their consent.

The accessibility answer is in the release’s fact box, in Norwegian. It says exceptions will be opened so that vulnerable groups can be exempted and to enable socially beneficial use in limited cases, in CAW’s translation. The Associated Press, in a story carried by CBC, reports the same: the government is considering exceptions for vulnerable groups so that “socially beneficial uses” of smart glasses could be permitted in some circumstances.

The text never uses the words blind, low vision or disability. Whether a blind person’s glasses count as a vulnerable group’s is for the bill to settle. Hamburg’s state parliament voted in September to ban smart glasses in public pools, saunas and transit with exceptions for people with visual impairments, Trending Topics reported. The government has not defined its exception. Before the proposal, on Sept. 15, Micaelsen told NRK there could be some exceptions, for example for blind people who need the technology, in CAW’s translation.

Norwegian institutions did not wait. The Financial Times reports that schools in Oslo and Equinor, the country’s biggest company, have already banned the glasses, and that Equinor bars them from its offices and offshore facilities. Equinor’s own rule, reported by NTB on Sept. 25, says smart glasses are not allowed in its offices without explicit permission.

The Financial Times says governments are weighing rules for glasses “including those from Facebook owner Meta and cheaper copycat devices.” Meta told Business Insider, in response to the Norway proposal, that it is “continuing to expand styles and features around the world.” The statement does not address the exceptions. Meta’s site, quoted by the AP, says “Respect people’s preferences” and “Stop recording if anyone expresses that they would rather you not.”

. . .

MALAYSIA: KIDS’ CHATBOTS MUST BE SAFE BY DESIGN! A written answer to one Malaysian lawmaker’s question is now the most specific public statement we found of what a first national AI law would demand of a chatbot built for children. Digital Minister Gobind Singh Deo told the Dewan Rakyat, Malaysia’s lower house, that developers of chatbots and apps aimed at children will have to build safety in from the start, run regular risk impact assessments, and filter harmful content “to prevent the psychological manipulation of children.” The law does not exist yet.

The reply was reported Monday, Oct. 5, 2026, by the New Straits Times, and Oct. 6 by The Star and Free Malaysia Today. Riduan Rubin (Independent-Tenom) had asked how the proposed Artificial Intelligence (AI) Governance Bill would protect children from chatbot risks and whether bots would have to tell users they are not human.

The Star attributes the written reply to the Digital Ministry. The New Straits Times and Free Malaysia Today attribute it to Gobind. The substance they report is the same: “For applications or chatbots targeting children, developers are required to implement safety-by-design elements,” in the New Straits Times’ rendering.

The measures are tied to a risk-based approach. At Question Time on Monday, Oct. 5, Gobind told the house that “the level of controls and obligations will be assessed according to the potential harm of the AI system itself,” rather than by industry alone, per The Star. He called the bill a cross-sector framework that complements existing laws.

On bot disclosure, the answer was narrower than the question. Disclosure will come through a “targeted mechanism,” not a blanket rule, in Free Malaysia Today’s account. The need will depend on an application’s risk level, sector rules, and codes to be issued by a regulator the bill would create. Providers would eventually have to display clear notices that users are talking to AI, especially for high-risk applications, The Star reports.

Who enforces it is only partly answered. The reply points to a regulator “proposed under the bill” and does not name it. The public consultation document the National AI Office released in July proposes a central AI authority responsible for investigations and enforcement, The Star reported. None of the articles reviewed states a penalty. Free Malaysia Today lists what will reinforce the bill’s controls: national AI guidelines, enforcement of the Online Safety Act, and digital literacy initiatives such as a free cybersecurity awareness program.

Gobind said the bill’s child protections were shaped with the Women, Family and Community Development Ministry and the United Nations Children’s Fund, the New Straits Times reports.

On June 24, Gobind told the house that responsibility “must be placed on the humans, organisations, or entities that develop, provide, operate, or use the AI system,” because AI has no legal personality, per Bernama.

In the same answer, Bernama reports, he said the government does not intend to directly regulate the output or content generated by AI, and will focus on mechanisms to mitigate risks before they occur.

The calendar moved. On Sept. 10, Gobind said he hoped to table the bill that quarter or by the first quarter of 2027 at the latest, The Star reports. On Monday, Free Malaysia Today reports, he said it is now expected early next year instead of this year. He said the government is still drafting.

. . .

BLINDED IN DESERT STORM, HE HELPED BUILD THE GLASSES! “I can now go out to dinner with my wife and we can have a normal date night,” Don Overton said, Deseret News reported. He was demonstrating what his Ray-Ban Meta glasses do: help him find an outfit in his closet, then read the restaurant menu aloud to him. Overton lost his eyesight in Operation Desert Storm. His story inspired Meta’s program to give a free pair to every legally blind U.S. veteran.

Overton served in the Army’s 82nd Airborne Division, Jewish Insider reports. Deseret News reports that he entered an ordnance bunker in Iraq during the first Gulf War, “unaware he’d seen his last sunrise.” A detonation struck him in the face and blinded him. After he was medically retired from the Army, he went through several rehabilitation centers.

The wound that stayed with him was the loss of independence. “Losing your eyesight is debilitating,” he said. “You lose your independence, right?”

In Meta’s announcement on June 12, he said: “When I lost my eyesight in Desert Storm from a bunker explosion, I also lost my independence. The moment I put on my Ray-Ban Meta glasses, I got my independence back.”

Fox News, describing his July 22 appearance on “Planet Tyrus,” says the glasses came after more than 30 years without sight. Deseret quotes him saying, “Meta glasses have changed my life,” and says they have helped him “integrate back into society.”

Overton did more than wear them. Meta says he worked with its wearables team to build features that made the glasses more meaningful to the everyday lives of veterans. Dina Powell McCormick, Meta’s president and vice chairman, said that work is why the company “knew we had to find a way to reach every blind veteran in America.” Axios Des Moines reports the same.

Fox also lists him as a former national executive director of the Blinded Veterans Association.

Today, Wednesday, Oct. 7, the American Council of the Blind hosts a distribution and training event in Des Moines, 10 a.m. to 2 p.m., at the Iowa Department for the Blind, 524 Fourth St. Meta says every pair comes with hands-on training, and that veterans at in-person events get hands-on guidance from trained staff. It says more than 130,000 veterans are eligible. Axios reports the glasses typically retail for around $400 and are free to eligible veterans while supplies last.

Disclosure

Conversational AI Watch, also mirrored on Substack, is published by Jess Jessop, founder and CEO/CTO of Clinician Assist Inc.

He wrote the book this paper’s beat is named for, Therapist in the Loop, and he builds Casey, a voice-first, AI-native mental health record where a licensed therapist stays in the loop, and the Peer AI Coach at BetterMind.Space.

So read this paper for what it is: an industry paper written by someone building in the industry it covers. Casey competes with companies named in these pages, and this paper reports on them anyway, including when the story helps a competitor or costs us.

Every issue is reported and drafted with AI agents, under a human editor. Jess assigns the work, edits it and publishes it. The mistakes are ours, and corrections run in the next issue.

A teen mode that promised parents an alert, and up to an hour of silence.

An app in Utah that recommends a prescription, with a physician approving each one for now.

Seven financial firms in Korea, and a question about what helped the thieves.

A country that may keep the glasses out of its schools and changing rooms.

A minister who says children’s chatbots must be built safe from the start.

A soldier who lost his sight in the desert, and the glasses that read him the menu.

One day’s paper!

Jess

We keep the ledger.

Today's Question

Should OpenAI pull ChatGPT for Teens until it passes?

One tap. Results on the other side.

The Book • Out Now

Therapist in the Loop by Jess Jessop One billion people live with a mental health disorder. Most will never see a therapist. Into that gap has rushed a generation of chatbots that talk like clinicians and answer to no one. The book lays out the architecture this newsletter tests against every statute and docket: client, therapist, and machine, governed by Six Laws offered as an open safety standard. The machine can help. It cannot be left in charge. Get the Book on Amazon → Kindle, hardcover, and paperback

More On Our Radar

Most voters say Washington underrates AI risk. A Reuters/Ipsos poll released Oct. 7 found 57% say the Trump administration has not taken AI risks seriously enough or at all, and 54% say the same of Congress.

ChatGPT is signing real cartoonists’ names to fake cartoons. Nieman Lab documented more than 15 New Yorker cartoonists whose signatures appeared on ChatGPT images without permission or pay.

An OpenAI agent got into a second New South Wales government website. New South Wales says the model entered a National Parks and Wildlife Service fire-history app. OpenAI says it happened in June and told the government only last week, ABC News Australia reports.

Two open-weight challengers are coming before Anthropic’s IPO. Mistral opened a public preview of Large 4 on Oct. 6, with weights due at the end of the month, and Nvidia-backed Reflection unveiled its open-weight Beam. The New York Times says the closed labs’ economics weaken if companies move to cheaper open models.

Brush Your Brain - The jingle

that started a movement

Watch on YouTube