California pitches bill to block use of artificial intelligence chatbots in mental health treatment

Senate Bill 903 would require a licensed professional to validate AI-generated advice

California has proposed a bill to prevent artificial intelligence chatbots from being used in mental health therapy, reported The Associated Press.

The legislation will ban chatbot companions and AI tools from making therapy-related decisions without undergoing a licensed professional’s assessment. It would also block companies from marketing chatbots as therapists.

Health providers would be mandated to inform patients if AI tools will be used to record therapy sessions or to triage mental healthcare; patients must provide consent for these tools to be used. The bill has been met with support from professional associations representing psychologists, therapists, and counselors.

San Diego-based senator Steve Padilla introduced the bill, which is currently sitting before the assembly’s fiscal committee, on January 21. Padilla said the law was a necessary safeguard for consumers and for licensed professionals alike, as it would prevent the latter from being replaced by AI.

The National Union of Healthcare Workers has backed the legislation due to its alignment with the union’s challenge against using AI and digital tools in mental healthcare without regulation. The union had recently filed a complaint against Kaiser Permanente with state regulators because the organization supposedly used an automated algorithm in mental health triaging.

“The difference between a licensed clinician and an automated response is not technical. It can be life altering,” said Le Ondra Clark Harvey, the California Behavioral Health Association’s chief executive officer, in a statement published by AP News.

Harvey explained that in lieu of appropriate protections, chatbots could disclose erroneous information and mishandle crises. The law is being introduced following a wave of lawsuits against AI chatbots that reportedly exacerbated users’ mental health problems, leading to suicide in some cases.

However, TechNet, a representative of tech-sector companies, countered that the bill could keep AI from being applied to the benefit of healthcare.

“At a time when every county in California faces a shortage of behavioral healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster,” said Robert Boykin, California executive director at TechNet, in an email statement, a snippet of which was published by AP News.

A Harvard Business Review study revealed that people often used AI chatbots for therapy and companionship because they are typically free and always available, unlike standard mental healthcare. However, the American Psychological Association has cautioned that AI was unequipped to properly handle issues like suicidal ideation.

Illinois was the first US state to ban AI use in therapy services last year. Colorado, Maine, Rhode Island, Tennessee, and Vermont have implemented similar laws, according to Becker's Behavioral Health.