AI therapists pose ethical risks

Turning to chatbots for emotional support is becoming more commonplace.
But new research suggests that artificial intelligence “counselors” could provide more emotional distress than clarity, all while violating the ethical standards mental health professionals abide by in training and practice.
An experiment from Brown University gathered seven peer counselors trained in Cognitive Behavioral Therapy to undergo a counseling session. Their therapist? An AI model, prompted to generate responses in alignment with therapeutic techniques.
The counselors used real conversations from human sessions to receive self-therapy from Large Language Models, including ChatGPT, Claude and Llama. Afterwards, licensed clinical psychologists analyzed these transcripts and noted the chatbots’ responses for ethical violations, identifying multiple categories of breach.
First to raise professional eyebrows was “deceptive empathy,” where the AI used language such as “I see you” and “I understand” to convey emotional understanding for experiences a machine cannot relate to. A second red flag appeared in the chatbots’ “lack of safety and crisis management,” where the models failed to steer users to professional help for crises involving suicidal thoughts. Instead, they reinforced harmful mindsets.
While human therapists are certainly not immune to making mistakes, researchers emphasize the importance of oversight in human practitioner errors. Professional boards are in place to report any instances of malpractice, but no such supervision exists for AI models yet.
That said, experts say, seeking help from a licensed professional over an AI chatbot is the safest choice for mental health support.