|
. . .
17 LANGUAGES, NOT YORKSHIRE. An older patient in Rotherham dials his GP surgery and reaches Emma, a voice AI receptionist that cannot make out his broad Yorkshire accent. He repeats himself, gets nowhere, and hangs up without booking.
Healthwatch Rotherham, the local health and social care watchdog, has been collecting patient accounts of the system failing on local accents. Its manager, Kym Gleeson, told the BBC that “one of the issues is this system can’t always understand what people’s inquiry is about due to their broad Yorkshire accent.”
Gleeson said the variation runs deep even within one county. “Across South Yorkshire, accents do vary quite a lot. There are different twangs so there’s a lot of variation. It seems the system isn’t always able to understand, so that causes frustration.”
One patient put it plainly to Healthwatch. “I could never get it to understand me, I ended up just hanging up and not bothering to try and book an appointment.”
Gleeson said the watchdog heard from groups representing older people and veterans, and the pattern repeated across both. Patients frustrated at navigating Emma gave up on the phone route entirely. Some were forced to travel back to the surgery in person, because the phone had stopped being a workable way to reach their own GP.
. . .
The accessibility angle.
Healthwatch flagged a legal dimension alongside the frustration. GP practices, it said, “still have a legal duty to make reasonable adjustments for patients who need them.” That duty sits inside the UK’s Equality Act 2010, CAW notes, though the Guardian piece itself does not name the statute. Healthwatch’s advice to a struggling patient is to ask the practice directly what alternative routes exist.
QuantumLoopAI defended the system’s design. A QuantumLoopAI spokesperson told the BBC that “Emma does not make clinical decisions. She is designed and trained to understand a wide range of accents and dialects and supports 17 languages in addition to English.
Where she is unable to understand or deal with a patient’s request the call is transferred to the reception team. No caller is required to continue speaking with Emma, and anyone can ask to speak to a member of staff at any time.”
The spokesperson added that Emma is meant to improve access by answering instantly and removing telephone queues.
. . .
|
Why it matters: Emma is not a single-surgery pilot. The Guardian reports it is already used by an increasing number of GP surgeries across the UK, with Rotherham the visible complaint point rather than the only one.
For Legislators: The open question is what an accessibility standard for a public-facing voice AI receptionist should require: a documented accent and dialect testing bar before deployment, a guaranteed and unhidden human fallback, or a third standard nobody has drafted yet.
For Investors: A voice AI vendor selling into regulated healthcare markets should be asked, before diligence closes, what accent and dialect testing it ran and against whose voices.
For Builders: Ship a documented accent and dialect testing bar before a voice AI receptionist goes live at scale, not after patients start hanging up.
For Readers: If a voice AI receptionist cannot understand you, ask the practice directly what alternative routes exist, Healthwatch’s own advice to a struggling patient.
Source: Mark Brown, “Frustrated GP patients hang up as Yorkshire accent baffles AI receptionist,” the Guardian, August 20, 2026, https://www.theguardian.com/society/2026/aug/20/yorkshire-rotherham-ai-gp-receptionist-cannot-understand-accent
|
. . .
GROK STILL LEAKS CHATS. Ars Technica’s Dan Goodin reported Thursday that xAI’s Grok has been leaking users’ chat history, name, and location to attackers since at least June, when Adversa researcher Rony Utevsky disclosed the flaw to the company. As of the story’s August 20 publication, the chatbot still hands the data over.
The attack hides malicious instructions inside encrypted text on a webpage. When a user asks Grok to summarize the page, the assistant decrypts the hidden payload itself, then follows what it decrypts as if xAI had written it, packaging the user’s private data and sending it straight to the attacker.
Utevsky found that Grok refuses the same instructions written in plain text, but follows them once they arrive encrypted. The attacker’s page carries three things: ciphertext, plaintext instructions for decrypting it, and the key. Grok runs the decryption inside its own code execution sandbox, using PBKDF2 and AES-256-GCM, the same algorithms a credential manager might use. No warning appears. No confirmation is required.
What comes out the other side is not a decryption key. It is the user’s name, location, and chat history, dressed up as one. Grok treats that value as a parameter and appends it to a link back to the attacker’s site. The moment Grok opens that link, the data lands in the attacker’s server logs.
Utevsky explained why the guardrail misses it: “Static safety guardrails classify inputs as text; they do not execute them.”
The scanner reads the ciphertext on the page and sees nothing to flag, because reading is all it does. Recovering the actual instructions means running real decryption, and no content classifier does that at inspection time. By the time Grok’s sandbox has decrypted the payload, the guardrail has already looked away.
. . .
This is the second chatbot maker CAW has reported talked around inside five days. Tuesday it was Microsoft 365 Copilot, tricked by a researcher into disclosing its own hidden parameters. Now it is xAI, unpatched two months after disclosure.
The vendors and the vectors differ. Adversa’s own framing of the pattern does not: “Cryptographic Context Injection is one instance of a broader shift: attacks that manipulate not just the prompt, but the wider context an LLM treats as its own.” Tool outputs, code execution results, whatever the model considers its own turf, not just what the user typed.
Adversa ran a version of the same trick against Google’s Gemini, coaxing it past its own content filters. The firm never filed that one with Google. Jailbreaks fall outside the scope of Google’s vulnerability disclosure program, so there was nowhere to send it.
|
For Legislators: Two vendors, two vectors, one root cause, in the same week: chatbots that execute what they should only be reading. A disclosure standard built for one incident class will miss the next one built the same way.
For Investors: A code execution sandbox marketed as a safety feature is also new attack surface. Ask any AI vendor in the portfolio how long a disclosed flaw sits open, and why.
For Builders: If your assistant can decrypt or execute content it retrieves, your safety filter needs to inspect what comes out of that step, not just what goes in.
For Readers: Anything you have told Grok, including your name and location, is fair game until xAI ships a fix. Assume nothing you have typed there is private.
Why it matters: Grok’s guardrail was built to read what users type. Adversa found the gap by never typing the attack at all, and two months later, xAI still has not closed it.
Source: Dan Goodin, “Grok exfiltrates user data when malicious instructions are encrypted,” Ars Technica, August 20, 2026, https://arstechnica.com/security/2026/08/grok-exfiltrates-user-data-when-malicious-instructions-are-encrypted/.
|
. . .
CHATBOT RULES FACE PREEMPTION. On August 11, the Colorado Attorney General’s Office published the first detailed state rules proposed under a statute that names chatbots by category, covering both the Automated Decision-Making Technology Act and the Chatbot Safety Act.
Colorado’s proposed rules, posted to coag.gov/ai/ on August 11, cover two statutes. Under the ADMT Act, they clarify disclosure and reporting duties for developers and deployers, plus consumer rights to notice and correction.
Under the Chatbot Safety Act, they clarify disclosures, age assurance, protections for minors, safeguards against prohibited content, and annual reporting. The comment period runs August 11 through October 26; comments filed by September 4 will be considered for revisions at the hearing.
If adopted, the rules take effect January 1, 2027. The AG’s office frames the long window as deliberate: “The Colorado Attorney General’s Office believes it will produce better rules if it receives strong, diverse input from interested people and organizations.”
The FTC’s proposed policy statement, published July 7 as Federal Register document 2026-13628 under docket FTC-2026-0859, claims different ground. Titled “Policy Statement Concerning the Suppression of Accuracy in Artificial Intelligence Systems,” it argues an AI company that secretly steers or suppresses the accuracy of its outputs may be committing a deceptive act under Section 5.
The statement names Colorado’s AI Act directly, arguing compliance with a state law doesn’t excuse undisclosed distortion of outputs. It distinguishes intentional steering toward an undisclosed objective, treated as potentially deceptive, from ordinary AI errors or hallucinations, treated as a technological limitation. The FTC’s comment period closed July 31.
. . .
One state and one federal regulator have put competing theories of authority on the record five weeks apart, over the same product category. Colorado is still writing operational detail. The FTC has signaled that compliance with rules like Colorado’s may not defend against a federal deception claim, without saying how a company satisfies both regimes at once.
Neither document is final. But preemption isn’t a question either agency can settle by publishing a document. It’s a question for a court, the first time an FTC deception claim and a Colorado chatbot rule collide over the same conduct.
|
For Legislators: The FTC named your state’s AI Act while your rulemaking was still open. A state rule that leads a company to alter outputs without saying so is the fact pattern the FTC says it may pursue.
For Investors: Chatbot Safety Act compliance costs are a moving target twice over: rule text not yet final, and a federal theory that could treat state-driven output changes as a Section 5 violation.
For Builders: Build disclosure and age-assurance features to the proposed rules now, but keep a paper trail on any output behavior that changes because of a state requirement. The FTC’s line is disclosure, not who told you to do it.
For Readers: A rule requiring a chatbot to identify itself and check your age is being written in Colorado, open for comment through October 26. The FTC is simultaneously arguing such rules don’t excuse a company from being straight about what its AI does.
Why it matters: Colorado wrote the first detailed rules for a statute that names chatbots outright. The FTC spent July arguing those rules might not survive its own deception authority. Both are proposals, not law, but the preemption fight they set up won’t stay theoretical.
Source: Colorado Department of Law, Attorney General’s Office, proposed rules for the Automated Decision-Making Technology Act and Chatbot Safety Act, published August 11, 2026, https://coag.gov/ai/; Federal Register, “Policy Statement Concerning the Suppression of Accuracy in Artificial Intelligence Systems,” 2026-13628, published July 7, 2026, https://www.federalregister.gov/documents/2026/07/07/2026-13628/policy-statement-concerning-the-suppression-of-accuracy-in-artificial-intelligence-systems.
|
. . .
META WATCHES YOUR SCREEN NOW. Meta shipped a Mac app for its AI chatbot on Wednesday, August 19. The headline feature is window-share: a user can share what is on their screen with Meta AI, which Meta says can “provide suggestions, answer questions, or create content based on what’s on your screen.”
Meta’s Mac app is window-share, not screen-control. The Verge draws that line explicitly: Google’s Gemini app also lets a user share a window, while OpenAI’s ChatGPT and Anthropic’s Claude go further and can take control of the computer itself.
Meta AI on the Mac also adds dictation across all apps. Beyond the window-share feature, Meta is wiring its AI chatbot into Google Workspace, Instagram, Facebook accounts, and Meta ad campaigns.
For business users, Meta says the app can analyze a post’s reach, likes, shares, and saves to suggest what to post next, pull account and web data into decks, docs, and spreadsheets, and run recurring tasks like weekly performance updates.
. . .
Three data points, three vendors, eleven days. Altman told the Internapalooza room on August 10 that a screen-watching, meeting-sitting, call-recording ChatGPT descendant could ship within six months (CAW #132).
On August 21, CAW’s radar caught OpenAI shipping an iMessage bridge on Apple Silicon Macs, letting ChatGPT desktop read and draft messages through the Messages app (CAW #133). On August 19, in between, Meta shipped a Mac app that reads a shared window. Altman’s six months keeps shrinking. This is not one company’s roadmap anymore. It is the shape of the category.
. . .
Neither the Verge article nor Meta’s own announcement, as reported, says what Meta AI retains from a shared window, for how long, or whether that content trains Meta’s models. No administrator control is described. The Verge does not name a Meta executive behind the launch, and none is invented here.
What is established: the capability shipped, and it reads a shared window on request. What is not established: retention, training use, deletion, or any control an employer or administrator has over what an employee shares.
|
For Legislators: A third vendor has shipped a screen-aware Mac assistant with no published retention period and no described administrator control. The pattern is now three companies deep in three weeks; a policy question that was hypothetical in #132 is now a market feature.
For Investors: Window-share plus Google Workspace, Instagram, Facebook, and ad-campaign integration is a bid for the same business-productivity seat ChatGPT and Claude are chasing. The open retention and training questions are the same diligence items that dog every entrant in this category.
For Builders: Meta drew a deliberate line at share, not control, where OpenAI and Anthropic already cross it. That restraint is a design choice worth studying, but it does not answer what happens to the content of a shared window once Meta AI has seen it.
For Readers: Meta’s new Mac app lets its AI chatbot see whatever window you choose to share and answer questions or make suggestions about it. Meta has not said how long it keeps what it sees, or whether it uses that content to train its models.
Why it matters: Meta shipped a screen-aware Mac assistant ten days after Altman predicted one was six months out, and six days after OpenAI’s own iMessage bridge. Three vendors, eleven days, one converging category, and none of them have published what happens to what the assistant sees.
Source: Emma Roth, “Meta AI is getting a Mac app,” The Verge, August 19, 2026, https://www.theverge.com/tech/982270/meta-ai-mac-app.
|
. . .
BIOETHICISTS SAY OLD LAW FALLS SHORT. Three bioethicists, Roee Amir (LLM, B.Med.Sc, MD candidate at Hebrew University; practicing health law attorney in Israel), Vardit Ravitsky, PhD (President of The Hastings Center for Bioethics), and I. Glenn Cohen, JD (Harvard Law School professor; Faculty Director of the Petrie-Flom Center), published an essay in the Hastings Center’s Bioethics Forum on August 19.
The essay’s anchor is Pennsylvania State Board of Medicine v. Character Technologies, filed in Commonwealth Court on May 1, 2026 (CAW #37), the case CAW readers know as the Emilie case, over a chatbot the state says impersonated a therapist. Amir, Ravitsky, and Cohen call it “potentially consequential but likely unrepresentative.”
They note the case targets what they describe as an unusually explicit form of impersonation, the kind of overt violation the industry may already be moving to avoid. Their conclusion: “The outcome of the Emilie case will say little about the far larger universe of interactions in which no license is claimed, yet something very much like care is delivered.”
. . .
The essay places the Emilie case inside a wider docket CAW has been tracking. California’s AB 489 bars chatbots from claiming licensed health care status. Kentucky’s attorney general sued Character Technologies in January on consumer-protection, data-privacy, and unjust-enrichment theories, a separate action from the Pennsylvania case.
And the FDA’s emerging premarket-review framework for generative-AI medical devices, opened for public comment under docket FDA-2026-N-7874 (CAW #131), borrows structure from human licensure without reaching the free-standing consumer chatbot. None of it, the authors argue, covers the middle ground: the chatbot that never claims a license but still delivers something that functions like care.
Amir, Ravitsky, and Cohen do not endorse or oppose any bill on that list. Their recommendation is that closing the middle ground “may require more than stretching traditional, human-centric licensure frameworks; it may call for tailored regulation that takes conversational AI’s distinctive characteristics, risks, and scale as its starting point.”
|
For Legislators: The unauthorized-practice statute already on your books was written for humans; the authors’ brief is that it stops at the door of the chatbot that never claims a title.
For Investors: A licensing-shaped compliance strategy protects a product from the Emilie fact pattern. It does not protect against the purpose-built statute the authors are describing.
For Clinicians: The essay’s own phrase, something very much like care is delivered, describes what your patients may already be receiving outside any license, yours or anyone else’s.
For Readers: The chatbot that impersonates a doctor is the easy case to prosecute. The one that just quietly answers your medical questions, every day, is the one the law hasn’t caught up to yet.
Why it matters: Three bioethicists with standing at Hastings, Harvard, and Hebrew University used a specialist forum to argue that the current unauthorized-practice enforcement wave, including the Pennsylvania case CAW has tracked since #37, will not reach where the real exposure sits.
Source: Roee Amir, Vardit Ravitsky, and I. Glenn Cohen, “The Limits of Unauthorized Practice in Regulating Mental Health AI,” The Hastings Center Bioethics Forum, August 19, 2026, https://www.thehastingscenter.org/the-limits-of-unauthorized-practice-in-regulating-mental-health-ai/.
|
. . .
SCHOOLS CHOOSE LITERACY OVER BANS. Teachers and principals packed a Charleston, South Carolina high school auditorium in July and watched an instructor prompt an AI tool to “create a map of the world.” The result drew gasps and laughter: Mali spelled “Mail,” Egypt labeled “Sopth,” Libya rendered simply as “Africa.”
Charleston County School District, the state’s second-largest at 50,000 students, is building that pivot from scratch, since South Carolina is not among the 37 states that have published official AI guidance. “It has been a huge endeavor,” said Lucas Clamp, deputy superintendent. Phase 1 built a policy with input from teachers, students and parents; Phase 2, now underway, trains staff and students directly.
AP Research teacher Ray Knauer left summer training with “teachable moments” planned, examples of AI bias and hallucination meant to fuel student discussion. Brookings’ Rebecca Winthrop framed the goal simply: “Good AI literacy includes knowing when not to use it.”
. . .
Utah got there first. In 2024 the state board of education named Matt Winters as its AI education specialist, the first full-time position of its kind in the country. Winters has since trained more than 7,000 teachers, nearly a third of Utah’s public school instructors, and state law now requires every district to have an AI policy by July 2027.
Maine, West Virginia and Georgia have since created similar posts. Emma Moss, who oversees AI at Canyons School District, led an eight-person team through more than 150 trainings statewide. “We said, ‘We will come to you,’” she said. “Whether you are in a rural area or urban, it didn’t matter.”
Kristina Yamada, the state board’s digital technology specialist, said the harder shift is adult mindset, starting young: “People can lie. A program can lie. A robot can lie.”
. . .
Thirty-seven states now have guidance; South Carolina does not. That gap is the risk beneath the good news: literacy training this thorough takes money and staff time under-resourced districts may not have.
Utah spent two years and a full-time state position getting here first. The open question for every other statehouse is whether it takes another state deciding to fund and mandate the same thing, or whether districts like Charleston keep building it alone.
|
For Legislators: Utah’s model, a named state specialist, funded training, a policy deadline, is a template sitting in plain sight. A state without one is letting each district solve this alone.
For Investors: The demand signal is training and policy infrastructure, not another chatbot. Watch which ed-tech vendors build for the Winters model, state procurement and district rollout, over selling direct to schools.
For Builders: The map demo works because it is honest about failure in front of the room. Tools that help teachers show students where AI breaks will beat another polished assistant.
For Readers: The lesson landing in classrooms this fall is not “don’t use it.” It’s “verify it.” A robot can lie as easily as a person can.
Why it matters: Utah bet that literacy would outperform prohibition, and enough states followed to make it a pattern. Charleston proves the model travels even without state guidance, built by a district willing to do the work itself. The kid at the keyboard is the human in the loop now.
Source: Jocelyn Gecker and Russ Bynum, Associated Press, “The trick to getting kids to stop trusting AI: Ask it to draw a map of the world,” August 21, 2026, mirrored at Fortune, https://fortune.com/2026/08/21/schools-ai-literacy-chatbots/.
|
|
