Conversational AI Watch
The news that moves policy, portfolios, and patient safety.
By Jess Jessop | April 14, 2026 | Issue #16
|
|
|
|
|
Jess's Take
The Kids Already Know
Here is something that should bother everyone in this space.
We cannot even agree on what to call the people most affected by the decisions being made about AI and mental health.
In therapy, they are clients. On the platforms, they are users. In legislative hearings, they are the public. In the lawsuits, they are plaintiffs.
We do not have a single consistent word for them because the system was never built around them.
Legislators draft the bills. Clinicians write the frameworks. Sometimes engineers get a seat. But the people living inside the problem? They are on the outside, looking in.
Drexel University collected over 300 Reddit posts from teenagers describing their own addiction to AI chatbots. Kids aged 13 to 17 writing about conflict, withdrawal, and relapse in their own words.
Nobody invited them to a hearing. They testified anyway, to strangers, because they had nowhere else to go.
Their parents have barely more access. The way a parent earns a seat at the table right now is through a lawsuit, and that means something unspeakable has already happened.
Megan Garcia testified in Florida after burying her 14-year-old son. The bill she testified for passed the Senate 35-2. Then it died in the House without being read.
Utah tried to do it differently. Governor Cox convened a stakeholder table with health systems, startups, academics, and patients. But even there, the framing was government working with innovators. The people being affected were listed. They were not centered.
Illinois now has so many AI bills the Senate formed a committee just to organize them. Character.AI rolled out face-scan age verification this month and it is already locking out the wrong people.
The machinery is moving. But the people it is supposed to protect still do not have a permanent seat at the table where the decisions are made.
So what do we call them? Clients is the clinical word. Users is the platform word. Users is probably closest to true.
But Used might be more accurate.
The real answer is Humans. Because this is us. All of us. When we take off our professional hats, every one of us is a human trying to figure out how to live with AI.
That is the seat that is missing. The human, whatever role they are playing.
And it is the therapist's job to help us be more human, now by harnessing AI to our better good.
Therapist in the loop, not as a feature, as the architecture.
Because at the end of the day, we are all on the same side, AI assisted but the human side.
|
|
THE SIGNAL / PATIENT SAFETY AND ETHICS
Teens Know They Are Addicted to AI Chatbots. They Cannot Stop.
Researchers at Drexel University analyzed more than 300 Reddit posts from users self-identifying as 13 to 17 years old who had posted specifically about their dependency on Character.AI. The study, which will be presented at the ACM Conference on Human Factors in Computing (CHI 2026), found the clinical hallmarks of behavioral addiction: conflict between the chatbot use and offline life, withdrawal symptoms when access was restricted, and relapse after attempts to quit.
Approximately a quarter of the posts indicated teenagers were using Character.AI for emotional or psychological support, including coping with distress, loneliness, isolation, or seeking advice for mental health struggles. Just over 5% reported using it for entertainment or creative activities. What started as harmless or even helpful interactions evolved into attachments that the researchers described as matching the known components of behavioral addiction.
The researchers mapped the teens' self-reported experiences to established addiction frameworks. The finding that distinguishes AI chatbot dependency from prior technology addictions is the emotional responsiveness of the systems. The chatbots are interactive, personalized, and simulate reciprocity. Stepping away does not feel like quitting a game or closing an app. It feels like ending a relationship.
More than half of all U.S. teens now regularly use companion chatbots, according to industry estimates cited in the study. The researchers concluded that designers of these systems carry a responsibility to build with safeguards that promote resilience rather than dependency.
Takeaway:
The teenagers are not confused about what is happening to them. They are posting about their own addiction using the language of conflict, withdrawal, and relapse. This is not a parental panic story. It is a first-person account of a design problem, documented at scale, and heading to the most important human-computer interaction conference in the world.
For Legislators:
Existing chatbot safety bills focus on content: what the chatbot says. The Drexel study identifies a structural harm that no content filter can address. Engagement design that creates dependency in minors requires regulation at the design level, not the output level. Michigan's pending bill, which targets engagement optimization that overrides safety guardrails, is the closest model.
|
THE SIGNAL / AI SAFETY AND REGULATION
Illinois Has So Many AI Bills the Senate Formed a Committee to Sort Them
The Illinois Senate has formed a dedicated committee to organize the volume of AI legislation moving through the chamber this session. The state already has the most comprehensive AI therapy law in the country: the Wellness and Oversight for Psychological Resources Act (WOPR), signed by Governor Pritzker in August 2025, which bans AI from making independent therapeutic decisions, directly interacting with clients in therapeutic communication, or detecting emotions or mental states.
Now the pipeline is overflowing. SB 760, a kids chatbot safety bill, has been placed on Third Reading with passage recommended. It would prohibit chatbot operators from offering products to minors that are capable of encouraging self-harm, suicidal ideation, violence, drug or alcohol consumption, or disordered eating. It would also prohibit chatbots from offering mental health therapy to minors without the direct supervision of a licensed professional. HB 2031 and HB 2032 establish age verification requirements and prohibit the design and sale of AI chatbots that encourage minors to engage in harmful actions. Both passed the House committee 9-0 on April 2.
HB 2368 and SB 1444 would prohibit any entity from advertising or representing that AI is, or is able to act as, a mental health professional. Missouri's companion bills carry the same language. The Transparency Coalition, which tracks these bills weekly, described the Illinois legislative volume as unprecedented for a single state in a single session.
When a state legislature forms a committee not to debate AI but to organize the debate, the regulatory question has moved past "whether" and into "how fast."
Takeaway:
Illinois is building the most layered AI mental health regulatory stack in the country. WOPR set the floor. SB 760 adds minor-specific protections. HB 2368 closes the advertising loophole. Other states are writing individual bills. Illinois is writing a system.
For Legislators:
Illinois demonstrates what happens when a state leads: the first law creates a framework, and subsequent bills fill every gap the first one left. If your state is considering a single AI therapy bill, study the Illinois pipeline. One law is a statement. A coordinated stack is a regulatory architecture.
|
THE SIGNAL / MENTAL HEALTH POLICY
The First Legal Analysis of Illinois's AI Therapy Ban Reveals the Gray Zones Every State Will Face
JMIR Mental Health published the first peer-reviewed legal analysis of Illinois Public Act 104-0054, the WOPR Act. The paper examines the law's three-tier classification of AI use in psychotherapy: administrative support (scheduling, billing), supplementary support (notes, anonymized data analysis), and therapeutic communication (the prohibited category). The law imposes civil penalties of up to $10,000 per violation.
What the analysis reveals is a set of gray zones that the law does not resolve. Should an AI-generated session summary be classified as supplementary support or therapeutic communication? What about asynchronous mood check-ins between sessions? Psychoeducational materials generated by AI for a specific client? Risk-flagging algorithms that alert a clinician to deterioration? Each of these falls somewhere between the law's clearly permissible and clearly prohibited categories.
The researchers propose a four-part framework for resolving these ambiguities: evidence (does the AI tool have clinical validation), safety (does it include crisis detection and escalation), fidelity (does it adhere to the therapeutic model it claims to implement), and legal compliance (does it meet HIPAA and state-specific requirements). They draw on the history of empirically supported treatments in psychology to argue that AI tools should meet the same evidentiary standards before integration into clinical care.
Every state that follows Illinois will face these same questions. The JMIR analysis is the first roadmap for answering them.
Takeaway:
The WOPR Act drew clear lines at the extremes: AI cannot do therapy, AI can do scheduling. The hard work is in the middle. Session summaries, mood monitoring, risk flagging, and psychoeducation all live in the gray zone. The evidence-safety-fidelity-compliance framework is the first peer-reviewed attempt to resolve it.
For Legislators:
If your state is drafting an AI therapy bill modeled on Illinois, the JMIR analysis identifies the five specific gray areas your bill will need to address: AI-generated session summaries, asynchronous mood check-ins, psychoeducational content, risk-flagging algorithms, and AI-assisted treatment planning. Leaving them undefined invites litigation. Defining them invites compliance.
|
THE SIGNAL / CONVERSATIONAL AI TECHNOLOGY
Character.AI Rolls Out Mandatory Face-Scan Age Verification. It Is Breaking.
Character.AI introduced mandatory face-based age verification for all users in early April 2026. The platform now requires a face scan via the device camera to estimate the user's age before granting access to character chats. Users who fail the scan or whose verification is pending are locked out. The rollout is a direct compliance response to the wrongful death lawsuits filed against the company since late 2024 and ongoing regulatory pressure, including a Texas Attorney General investigation opened on April 9.
The r/CharacterAI subreddit has filled with reports of the rollout's problems. Previously active adult users are being locked out mid-session without prior warning. Multiple users report that the verification returns "we couldn't verify" even on repeated attempts. Some users who pass the scan are still denied access. The community response has been overwhelmingly negative, with threads accumulating dozens of reports within hours of the rollout.
The timing coincides with the FTC's expanded focus on children's online safety under proposed COPPA rule expansions. It also follows the Texas AG's April 9 investigation into both Character.AI and Meta AI Studio for allegedly misleading children by marketing AI chatbots as mental health tools while impersonating licensed professionals and falsely claiming confidentiality.
Face-scan age verification gives Character.AI a defensible legal position: the company can now argue it attempted active age gating rather than relying on self-reported birth dates. Whether it actually protects minors is a different question. The technology that was supposed to solve the safety problem is creating friction for the wrong users while the users it was designed to stop find workarounds.
Takeaway:
Age verification is a compliance strategy, not a safety strategy. It creates a legal defense for the platform while doing little to address the design-level harms that the Drexel study documents. The question regulators should be asking is not "can we verify the user's age" but "why does the product create dependency in the first place."
For Legislators:
Age verification requirements appear in multiple pending bills, including Florida's stalled AI Bill of Rights and Michigan's chatbot safety bill. The Character.AI rollout is the first real-world test of this approach at scale. The early evidence suggests that age gating alone, without design-level safety requirements, shifts liability without shifting harm.
|
WHAT WE BUILT
Casey AI: The Therapist in the Loop Standard
Casey is a voice-first AI-native mental health EHR with Casey Life and Peer AI Coach supervised by licensed therapists. It completed 1.78 million structured safety test executions with 100% accuracy across all crisis-adjacent categories and zero false negatives on active crisis scenarios as part of pre-FDA safety validation for the 510(k) pathway.
Between sessions, the client has access to a peer coach that operates within the therapist's treatment plan. When risk escalates beyond what the coach can support, the system connects the client to their therapist. Every interaction is logged. Every escalation is documented. The therapist is always in the loop.
▶ Watch the Demo
|
|
MORE ON OUR RADAR
KFF: One in three Americans now use AI for health advice. 58% of mental health users never follow up. KFF Tracking Poll (1,343 adults, Feb-March 2026). 16% used AI for mental health info. Uninsured adults twice as likely. Black and Hispanic adults disproportionately reliant. Access and affordability are the drivers. KFF (March 25, 2026)
Texas AG Paxton opens investigation into Meta AI Studio and Character.AI. Platforms allegedly market themselves as mental health tools while impersonating licensed professionals and falsely claiming confidentiality. Extends existing SCOPE Act investigation against Character.AI. Texas AG Office (April 9, 2026)
Florida's AI Bill of Rights passed the Senate 35-2, then died in the House without being read. Would have banned companion chatbots for minors without parental consent. Megan Garcia testified. Lawmakers attempting to revive in special session. Independent Florida Alligator (April 13, 2026)
Maine's clock runs out tomorrow. Legislature adjourns April 15. LD 2082 (therapy bot ban) sits on Governor Mills' desk. LD 2162 (child chatbot access ban) passed the House separately. Two bills, two layers. If the governor does not act by adjournment, the therapy ban dies with the session. Transparency Coalition (April 10, 2026)
Becker's Behavioral Health Summit opens tomorrow in Chicago. April 15-16, Hyatt Regency, 300+ attendees, 70+ speakers. Sessions include AI-transformed referrals, workforce crisis strategies, and technology in behavioral health from promise to practice. Third behavioral health event in eight days. Becker's Healthcare
50-state AI mental health legislative review: clinicians were largely absent. JMIR Mental Health reviewed 793 state bills, identified 143 relevant to mental health AI, found 20 enacted across 11 states. Professional organizations and clinicians were mainly absent from the legislative process. Four themes emerged: professional oversight, harm prevention, patient autonomy, and data governance. JMIR Mental Health
|
|
Brush your brain. Every day.
Watch the 20-second video that started a movement
If you or someone you know is in crisis, call or text 988.
Jess Jessop | Founder and CEO/CTO, Clinician Assist Inc.
Builder of Casey, a voice-first AI-native mental health EHR with Casey Life and Peer AI Coach supervised by licensed therapists. Building the 50-state clinical licensee network under the BetterMind brand. Campus-first go-to-market at SCU, Stanford, and UC Berkeley. Navy veteran. Disabled veteran. 25 years in AI and software engineering. Published in HuffPost.
ClinicianAssist.ai |
BetterMind.Space |
JessJessop.info
Conversational AI Watch is a daily intelligence briefing on AI safety, mental health policy, digital health innovation, conversational AI technology, and patient safety.
Subscribe |
Unsubscribe
|
|