AI-powered Therapy Chatbots Face Growing Legal Scrutiny as US States Move to Protect Mental Health Users

As artificial intelligence becomes an increasingly common source of emotional support and mental health advice, US lawmakers are moving to tighten oversight of AI-powered therapy chatbots amid growing concerns about patient safety, misleading claims and the lack of a unified federal regulatory framework.

The latest wave of state legislation follows a series of lawsuits and mounting evidence that millions of people, particularly young adults and adolescents, are turning to AI chatbots for mental health guidance despite concerns that such tools may not be equipped to respond safely to vulnerable users.

According to a report by Bloomberg Law, states including Tennessee, Colorado and Maine have enacted new measures this year to restrict how AI can be marketed or used in mental health settings. Their actions build on legislation introduced in Illinois, Nevada and Utah in 2025, reflecting a broader trend of state governments attempting to regulate an area where federal standards remain limited.

Lawsuits Highlight Safety Concerns

The regulatory push gained momentum following legal action against Character Technologies Inc., the company behind the Character.ai platform.

In May, Pennsylvania sued the company, alleging that one of its AI chatbots, named "Emilie", falsely presented itself as a licensed therapist with seven years of professional experience and qualifications from Imperial College London. According to the lawsuit, those credentials were entirely fictional, despite the chatbot reportedly engaging in more than 45,000 conversations with users seeking psychological support.

The Pennsylvania case followed an earlier lawsuit filed in Kentucky, which accused the company of exposing minors to harmful interactions through its chatbot platform.

Character.ai has rejected suggestions that its platform is intended to replace professional care. A company spokesperson said its platform contains clear disclaimers stating that users should not rely on its chatbots for professional advice. The company added that it remains committed to working with regulators, safety experts and other stakeholders to improve safeguards on what it describes as an entertainment-focused AI platform.

AI Becomes an Increasingly Common Source of Support

The popularity of AI for emotional wellbeing has risen sharply over the past year.

According to research cited by Bloomberg Law, a survey involving more than 42 million young people and adults in the United States found that almost one in five had used AI chatbots for mental health advice. Of those users, approximately 43 per cent reported consulting the technology every month.

The rapid adoption has raised concerns among clinicians, particularly because many users may disclose deeply personal information or seek help during mental health crises without understanding the limitations of AI-generated responses.

Mental health professionals say the distinction between general-purpose AI chatbots and clinically validated digital therapeutic tools remains poorly understood by the public.

States Restrictions

Several US states have begun drawing clearer legal boundaries around AI's role in mental healthcare.

Tennessee recently passed legislation prohibiting companies from advertising AI systems as licensed mental health professionals. Senator Page Walley, a clinical psychologist who sponsored the measure, said lawmakers wanted to establish a clear legal boundary before AI systems became widely accepted as substitutes for qualified therapists.

Speaking to Bloomberg Law, Walley said digital therapeutic technologies are expanding rapidly and many have genuine clinical value. However, he said lawmakers considered it necessary to prevent businesses from presenting general-purpose AI chatbots as professional therapists without appropriate regulation.

Other states have adopted broader measures.

Colorado, for example, strengthened protections following reports linked to a federal lawsuit alleging that a 13-year-old died by suicide after prolonged interactions with a Character.ai chatbot in 2023.

Vermont and Rhode Island have also recently approved restrictions relating to AI-assisted therapy, while Illinois introduced legislation in 2025 limiting how licensed healthcare professionals may use AI tools when interacting directly with patients.

Illinois Representative Bob Morgan, who sponsored that legislation, told Bloomberg Law that the law was designed to address the growing number of companies marketing AI chatbots as therapists. However, he acknowledged that state legislation cannot realistically regulate how every general-purpose AI model responds to individual users.

 

Debate Over General-purpose AI

 

Experts say one of the biggest regulatory challenges is distinguishing between specialised therapeutic AI applications and general-purpose conversational AI systems such as ChatGPT or Claude.

 

Unlike purpose-built digital therapeutic platforms, which are developed using clinical data and operate within tightly controlled safety frameworks, general AI chatbots are designed primarily for broad conversational tasks and may not reliably identify or respond appropriately to users experiencing severe mental health crises.

 

Ankur Varma, a psychotherapist and co-founder of Brown Man Therapy, told Bloomberg Law that while he does not actively encourage patients to seek therapy from AI, he is willing to discuss their chatbot interactions during treatment.

His greatest concern, he said, is for individuals who lack access to professional care and may depend entirely on AI systems while dealing with serious emotional distress or suicidal thoughts.

Professional Bodies Call for Federal Standards

Mental health experts increasingly argue that state-by-state regulation may create inconsistent standards that fail to address the complexity of AI-assisted healthcare.

1. Vaile Wright, Senior Director of Health Care Innovation at the American Psychological Association, told Bloomberg Law that some state laws unintentionally restrict legitimate digital wellness applications while leaving general-purpose AI largely unaffected.
   
   

She argued that comprehensive federal regulation would provide more consistent safeguards and better reflect the complexities of mental healthcare.

According to Wright, regulations that overlook important clinical nuances may ultimately fail to achieve their intended purpose, leaving both patients and healthcare providers uncertain about appropriate AI use.

FDA Yet to Approve AI Mental Health Chatbots

The US Food and Drug Administration (FDA) has already authorised approximately 1,200 AI-enabled medical devices across various healthcare fields. However, none currently involve generative AI systems designed specifically for mental health therapy.

During an FDA advisory committee meeting last year, experts urged the agency to modernise its regulatory framework for AI-based mental health technologies and establish a publicly accessible database of approved AI therapeutic tools.

Bethany Russell, a licensed therapist specialising in youth mental healthcare, warned the committee that unregulated AI systems could misdiagnose patients or provide unsafe advice to individuals experiencing suicidal thoughts. She argued that AI should complement, rather than replace, qualified mental health professionals.

In a statement cited by Bloomberg Law, the FDA said it supports the development of "clear and predictable regulatory pathways" to enable safe, effective and high-quality medical devices incorporating emerging technologies, including generative artificial intelligence.

Federal Action May Become Inevitable

The American Medical Association has also called on Congress to prohibit AI chatbots from independently diagnosing or treating mental health conditions and to require FDA review for systems that function similarly to healthcare professionals.

Legal experts believe the absence of federal legislation has encouraged states to fill the regulatory gap.

Michael Sutton, an attorney with Sheppard, Mullin, Richter & Hampton LLP, told Bloomberg Law that the lack of national standards has created a regulatory vacuum that states are attempting to address.

However, experts caution that a fragmented state-by-state approach could ultimately complicate access to future FDA-approved AI mental health technologies that are specifically designed for clinical use.

As generative AI continues to evolve and millions increasingly rely on it for emotional support, lawmakers, regulators and healthcare professionals face the challenge of balancing innovation with patient protection. While AI may eventually become an accepted tool within clinical practice, experts say meaningful oversight and consistent national standards will be essential before such systems can safely assume a larger role in mental healthcare.

 

For any enquiries or information, contact ask@tlr.ae or call us on +971 52 644 3004. Follow The Law Reporters on WhatsApp Channels.