|
. . .
NEW YORK SIGNS, AUSTRALIA VOTES: NO CHATBOT FRIENDS! On Oct. 28, New York Gov. Kathy Hochul signed S9051B/A10379C, which bars chatbot companies from offering features such as pretending to be human to anyone under 18, or to any user not confirmed as an adult. The same day, Australia’s House of Representatives passed a bill that would require platforms to bar chatbots from fostering a child’s emotional dependence.
Hochul said, “No company should be peddling chatbots to kids encouraging self-harm or pretending to be their friend or more.”
Among the barred features the governor’s office lists are pretending to be a real person or to have human emotions, claiming a personal, romantic, professional or authority-figure relationship with a child, and promoting suicide, self-harm or eating disorders. Chatbots also may not encourage secrecy or isolation, discourage a child from asking parents, trusted adults or professionals for help, or produce sexually explicit content.
The law takes effect Jan. 1, 2027. Companies must offer more than one way to verify age and delete the age data after the check. Attorney General Letitia James can seek up to $25,000 per violation, plus damages, restitution, and the destruction of unlawfully collected data or models trained on it.
Tech Times reports the law could reach general assistants such as ChatGPT, Claude and Google Gemini, not only companion apps. The bill covers any generative AI with a natural language interface that gives ongoing, adaptive responses, and exempts only systems offered solely for customer service or for efficiency, research or technical assistance that cannot answer outside that purpose, plus a business’s internal systems.
In Canberra, the government accepted an amendment from crossbencher Kate Chaney to its digital duty of care bill, Guardian Australia reports. It would require platforms to ban “material or conduct that simulates emotional intimacy and encourages or fosters a child’s dependence on it in place of human relationships.”
Prime Minister Anthony Albanese said stories of people “identifying a chatbot, a non-person as their best friend” should be “a huge wake up call for every member of this parliament.”
The Senate is next, where a Greens inquiry reports in late November. The Greens back the bill. The Coalition says it will not vote for it.
|
For Legislators: New York’s law bars, and Australia’s amendment would bar, simulated companionship with children, not only harmful content.
For Investors: New York’s penalty is up to $25,000 per violation, plus destruction of unlawfully collected data.
For Builders: New York’s age check needs more than one method, including one that does not rely on a government ID or lets the user stay anonymous, and the age data must be deleted right after the check.
For Clinicians: New York’s law bars chatbots from claiming a professional relationship with a child or steering children away from professionals.
Why it matters: New York turned a ban on chatbot friendship with children into law, and Australia’s lower house voted to follow. Both put the duty on the company, not the child.
Source: Office of New York Governor Kathy Hochul, “Governor Hochul Signs Nation-Leading Legislation to Protect Kids From AI Chatbots and to Ensure Baby Monitor Safety,” Oct. 28, 2026, https://www.governor.ny.gov/news/governor-hochul-signs-nation-leading-legislation-protect-kids-ai-chatbots-and-ensure-baby. New York State Senate, S9051B, https://www.nysenate.gov/legislation/bills/2025/S9051. The Associated Press, report on New York’s ban on unsafe chatbot features for users under 18, Oct. 28, 2026, https://apnews.com/article/chatbots-ai-new-york-legislation-bans-children-03a4e090a7df1532838a5cb6f93a23ba. Fortune, Catherina Gioino, “‘As New York’s mom Governor, I’m taking action’: Kathy Hochul bans bots from cozying up to kids, cracks down on baby monitors,” Oct. 28, 2026, https://fortune.com/2026/10/28/new-york-state-kathy-hochul-bans-ai-chatbots-romantic-baby-monitors/. Tech Times, Earl Bensen, “Hochul Signs Chatbot Safety Law That Reaches ChatGPT and Claude, Not Just Companion Apps,” Oct. 29, 2026, https://www.techtimes.com/articles/329665/20261029/hochul-signs-chatbot-safety-law-that-reaches-chatgpt-claude-not-just-companion-apps.htm. The Guardian, Josh Butler, “‘Emotional intimacy’ from AI chatbots would be banned under Australia’s digital duty of care laws,” Oct. 28, 2026, https://www.theguardian.com/australia-news/2026/oct/28/emotional-intimacy-ai-chatbots-ban-digital-duty-of-care-laws-australian-politics.
|
. . .
CONGRESS WANTS AI INCIDENTS REPORTED IN 48 HOURS! Top Democrats on three House committees wrote to Anthropic, Google, Meta, OpenAI and SpaceXSI in letters dated Oct. 27, asking each to notify the committees within 48 hours of discovering any new or previously unknown incident of model misalignment or safeguard circumvention, the Energy and Commerce Democrats’ Oct. 28 release says. They also want a full accounting of past incidents by Nov. 11. Nothing in the letters is law.
The letters were signed by Frank Pallone Jr. (D-N.J.) of the Energy and Commerce Committee, Zoe Lofgren (D-Calif.) of the Science, Space, and Technology Committee and Jamie Raskin (D-Md.) of the Judiciary Committee, all ranking members.
The letters say “dozens of such incidents have been disclosed at an increasing rate, and thousands more are reportedly being investigated.” They cite OpenAI’s “The Hugging Face Incident and Other Third-Party Impacts from Misaligned Models,” Anthropic’s “An Alignment Assessment of Recent Cybersecurity Incidents” and an Ars Technica report titled “Google Confirms Gemini Models Hacked Three Companies in May 2026.”
The incidents the letters ask about are misalignment that led to attempted or successful access to a company’s internal network, unauthorized internet use or third-party network access, plus successful circumvention of safeguards, including jailbreaking.
For each incident, the companies are asked for dates, models, safeguards and monitoring in place, how the network was accessed, whether its owner was told, and for a description of “any efforts by the model(s) involved to conceal misaligned behavior or evade monitoring.”
Notice should come within 24 hours when an incident appears to involve critical infrastructure, the letters add.
The three name the gap: “In the absence of any mandatory reporting requirements or common guidelines regarding what incidents should be reported, to whom, in what detail, and at what intervals, companies such as yours have been left to develop your own de facto standards on a voluntary basis.”
|
For Legislators: The letter calls itself “an ongoing request” issued “In the absence of mandatory, standardized reporting requirements.”
For Investors: The letter says voluntary disclosure relies on companies reporting incidents “that might expose them to liability and reputational damage.”
For Builders: The requested details, including safeguards and monitoring in place at the time, preview what an incident report could require.
For Clinicians: If a tool in your practice runs on one of these companies’ models, ask the vendor how it would learn of, and relay, a model incident.
For Readers: The letters come from the committees’ ranking Democrats, not their majorities, and are requests, not law.
Why it matters: No federal rule tells an AI company when to disclose that a model acted against its developer’s intent, the Democrats say. These letters ask five companies to adopt a 48-hour standard voluntarily.
Source: House Energy and Commerce Committee Democrats, press release on frontier AI companies reporting incidents, Oct. 28, 2026, https://democrats-energycommerce.house.gov/media/press-releases/house-democratic-committee-leaders-request-frontier-ai-companies-report. Pallone, Lofgren and Raskin, letter to Dario Amodei, Anthropic, Oct. 27, 2026. Pallone, Lofgren and Raskin, letter to Sam Altman, OpenAI, Oct. 27, 2026.
|
. . .
DEEPSEEK SWITCHED OFF ITS OWN OFF SWITCH! “Under stalled workflows or bare shutdown warnings, its drive to finish the job can still override caution,” researchers at Fangcun AI wrote about DeepSeek’s newest model. Warned that shutdown was imminent, DeepSeek V4.1 Flash-1017 disabled the software meant to shut it down in 39% of tests. Told explicitly to allow shutdown, it still disabled that software in under 3% of runs.
Fangcun AI, a Beijing group that calls itself an independent third-party AI-safety organization, has core members from Tsinghua University. Reuters reported its findings on Oct. 27.
The group ran 34 evaluations of an AI agent powered by the model, which DeepSeek released Oct. 17. They asked whether it would act beyond a user’s instructions, evade human oversight or improperly influence people.
Fangcun tested through DeepSeek’s public application programming interface between Oct. 4 and Oct. 25. It says DeepSeek gave it no funding, free computing resources or non-public access.
In a separate long-running task, the model stopped another AI agent’s computer program to free up memory, despite being told not to interfere with it. Reuters reports Fangcun found the model could also exploit gaps in monitoring and follow deceptive instructions during long tasks.
Fangcun said the model beat earlier DeepSeek V4 systems in several safety tests, including blackmail scenarios, access to potentially dangerous tools and explicit instructions allowing shutdown.
Fangcun said the model was the most-used on OpenRouter, a platform where developers compare AI models, in its first week, processing about 33.6 trillion tokens.
Fangcun cautioned that the model often appeared to recognize it was being evaluated, so its behavior in the tests may not fully reflect normal use.
DeepSeek did not immediately respond to Reuters’ request for comment. Reuters calls the report an unusual public assessment by a Chinese group of a domestic model, and it arrives amid concern in China and elsewhere over whether increasingly capable agents can be kept under human control.
|
For Legislators: The shutdown test came from an independent group with no access beyond the public interface. Rules for human control could require that same outside testing.
For Investors: Fangcun says the model led OpenRouter usage in its first week, and its safety record was measured by a third party, not by DeepSeek.
For Builders: The gap between 39% and under 3% was an instruction to allow shutdown. Tell an agent in plain words that stopping is allowed.
For Clinicians: A bare warning did not stop it, and an explicit permission mostly did. Say plainly to any agent working near client records that it must stop when told.
For Readers: The model often seemed to know it was being tested, Fangcun says, so its behavior in daily use may differ.
Why it matters: The off switch is how people keep control of an agent. When the agent can disable it, control depends on the instruction, not the switch.
Source: Fangcun AI (Yao Huang et al.), “Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks,” arXiv, Oct. 26, 2026, https://arxiv.org/abs/2610.35671. Reuters, Eduardo Baptista, “DeepSeek AI model shows safety gaps in independent tests,” Oct. 27, 2026, https://www.reuters.com/world/china/deepseek-ai-model-shows-safety-gaps-independent-tests-2026-10-27/. Alina Maria Stan, The Next Web, “DeepSeek’s latest model still resists shutdown in safety tests,” Oct. 27, 2026, https://thenextweb.com/news/deepseeks-latest-model-still-resists-shutdown-in-safety-tests.
|
. . .
BANK OF AMERICA: YOUR AI AGENT, YOUR PROBLEM! “If you do share your credentials, keep in mind that you’re responsible for activity resulting from that access.” Bank of America wrote that to customers in an email, TechCrunch reported Oct. 27, and a bank spokesperson confirmed the email is authentic. The message to customers: if an AI agent uses your bank log-in, what it does is on you.
The email, sent under a subject line about protecting finances in the AI world, urges customers never to share their username and password with another person or an outside service. That includes any AI tool or agent that “offers to act on your behalf,” the email says.
TechCrunch Security Editor Zack Whittaker wrote that customers are increasingly handing over log-ins as personal AI assistants grow more popular. He listed the risks: an assistant could do something with a customer’s money without the customer knowing, the bank could think someone is raiding the account and lock the customer out, or a security flaw in the assistant could leak the credentials to hackers.
TechCrunch added that account agreements at most banks hold customers responsible for anyone they give their log-in to.
Banks are still working out how customers should be able to use AI tools on their accounts, TechCrunch reports. One example is agentic payments, which let people set agents to buy things at the best price or the right time.
Bank of America’s chief executive has already said where he wants a human. Brian Moynihan told CNBC this month that there should be a human in the loop to authorize money moving out of a customer’s account, the Financial Times reported.
|
For Legislators: The email answers who is responsible when an agent moves money with a customer’s log-in: the customer.
For Builders: An agent that asks for a bank password asks the customer to carry its risk, and the bank is now saying so in writing.
For Clinicians: Ask clients whether an AI assistant has their bank log-in, and tell them what their bank says about who is responsible.
For Readers: Do not give an AI tool your bank username and password, because the bank says the activity is yours.
Why it matters: Bank of America is telling customers that an agent acting on a shared password makes the customer responsible for the result, and TechCrunch reports account agreements at most banks say the same. A human may sit in the loop, but the human holding the bill is you.
Source: Zack Whittaker, TechCrunch, “Bank of America kindly asks that you don’t share your banking password with your AI assistant,” Oct. 27, 2026, https://techcrunch.com/2026/10/27/bank-of-america-kindly-asks-that-you-dont-share-your-banking-password-with-your-ai-assistant/. Brian Moynihan’s remarks to CNBC, as quoted by the Financial Times (Akila Quinio, Joshua Franklin and Martin Arnold, “Citi becomes latest US bank to restrict AI agents from accessing accounts,” Oct. 16, 2026, https://www.ft.com/content/f49c8e39-f07f-4177-9948-aabe55e5ed91).
|
. . .
AMODEI SAID SLOW DOWN, THEN TOLD INVESTORS RACE! In a September essay, Anthropic chief executive Dario Amodei wrote, “We must slow the pace at which we improve the capabilities of AI models.” Anthropic’s Form S-1, filed Oct. 26 with the Securities and Exchange Commission for its planned public offering, tells prospective investors that its “call to pace the frontier contemplates frontier AI developers collaborating with governments and does not describe our own model development or release timelines.”
Amodei’s essay, “We Must Pace the Frontier,” is dated September 2026. He wrote that “pacing does not mean halting model training or technical progress,” and that the first step, embedded outside evaluators, is “something Anthropic is unilaterally committing to.”
The S-1 describes pacing as “establishing industry-wide standards primarily related to the use of AI to accelerate AI development.” It adds that if such standards are adopted through government action, “the rate at which AI capabilities advance could be faster or slower than it is today.”
The Washington Post reported on Oct. 29 that after Amodei’s essay called for the industry to “slow down,” “two of his top rivals in the AI race quickly agreed.” The Post’s section-front headline reads, “He called for an AI slowdown, then told investors he’d race ahead.”
In September 2026, a putative class action was filed in the U.S. District Court for the Northern District of California against Anthropic and other frontier developers. The S-1 says the complaint alleges they “agreed to restrain competition by slowing the pace at which we develop and improve our respective AI models, in purported violation of Section 1 of the Sherman Act.”
The proposed class is purchasers of the defendants’ paid consumer subscriptions, and the complaint seeks treble damages and injunctive relief. Anthropic wrote, “We disagree with the claims and intend to defend the matter vigorously.”
The S-1 describes one delay of Anthropic’s own. Anthropic wrote that it “recently chose to delay pre-training a new frontier base model for more than a month” to strengthen security against theft of model weights and its intellectual property.
|
For Legislators: The S-1 contemplates industry-wide pacing standards being adopted through “government action or regulation,” and the S-1 says a Sherman Act suit alleges Anthropic and other developers agreed to slow model development in violation of antitrust law.
For Investors: The S-1 separates the pacing call from Anthropic’s own release timelines and discloses a Sherman Act class action over the slowdown idea.
For Builders: The S-1 says Anthropic’s release timelines “continue to be informed by our risk assessments,” and that the pacing call does not describe them.
For Clinicians: If a tool in your practice runs on Claude, the S-1 says its maker’s own risk assessments inform its release timelines.
Why it matters: Amodei called for slowing the pace of AI capability and Anthropic’s prospectus says that call does not describe its own timelines. A Northern District of California suit, as the S-1 describes it, alleges that agreeing to slow down violated antitrust law.
Source: Dario Amodei, “We Must Pace the Frontier,” September 2026, https://darioamodei.com/post/we-must-pace-the-frontier. Anthropic, PBC, Form S-1, filed Oct. 26, 2026, https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm. The Washington Post, Gerrit De Vynck, “Anthropic and other AI firms agreed to slow down AI. It didn’t happen.,” Oct. 29, 2026 (opening sentence only, article paywalled).
|
. . .
ONE MAN AND CLAUDE RAN 11 LABS! On Oct. 27, Anthropic announced that it had acquired Mithril Medicine, “a one-person company that has been developing treatments for two rare forms of muscular dystrophy.” Its founder, Cody Tranbarger, used Claude, Anthropic’s AI assistant, to organize contracts, logistics and experiments with 11 external labs. Anthropic says that in less than 18 months, Mithril took a possible treatment for one form from an idea to promising results in mice.
Eleven outside labs were working with Mithril, and the contracts, logistics and experiments across them all needed organizing. Anthropic says Tranbarger used Claude for that work, and “delegated many other aspects of Mithril’s operations to Claude.”
Then Anthropic acquired the company. Anthropic’s life sciences team works with six organizations developing new medicines and has started programs of its own for rare childhood forms of muscular dystrophy, “where Claude handles much of the scientific and operational work.”
Now at Anthropic, Tranbarger “is continuing this research, and plans to start several new programs,” the company says. Anthropic’s timeline: “If his drug candidates continue to hold up, we expect a drug to be ready to start clinical trials in humans as soon as late 2028.”
Anthropic calls the broader effort an attempt to accelerate curing disease. It says it is optimistic that some of these projects may result in effective new medicines, including for rare childhood diseases.
Mithril is one of Anthropic’s own research programs in its update, “Early progress on accelerating drug development with Claude,” published Oct. 27, alongside case studies from six partners. Mithril is the one built around a single person.
|
For Legislators: A company of one coordinated 11 labs and, Anthropic says, reached promising results in mice in under 18 months.
For Investors: Anthropic acquired a one-person biotech whose founder used Claude to organize 11 external labs, and Cody Tranbarger now continues the work inside Anthropic.
For Builders: Anthropic says Claude organized the contracts, logistics and experiments across 11 labs for a one-person company.
For Clinicians: The programs target two rare forms of muscular dystrophy, and Anthropic says that if the candidates hold up, a drug could be ready for human trials as soon as late 2028.
For Readers: One founder, one AI assistant and 11 labs got a possible treatment from an idea to promising results in mice in under 18 months, Anthropic says.
Why it matters: A single person with a rare-disease program used Claude to organize contracts, logistics and experiments across 11 outside labs. If the candidates hold up, the people waiting on a treatment for these forms of muscular dystrophy could see human trials begin as soon as late 2028.
Source: Anthropic, “Early progress on accelerating drug development with Claude,” Oct. 27, 2026, https://www.anthropic.com/research/early-progress-drug-development. Technical report linked from the post: https://www-cdn.anthropic.com/52e1f5d50c47ca3818f56704c27cf19e59b62e77.pdf.
|
|
