The anonymity and interaction with strangers makes the digital world more attractive as a place to seek mental health support. This phenomenon is fueled by the fact that more than 150 million people in the United States live in federally designated mental health professional shortage areas.
“I’m scared to talk to a therapist and I can’t reach them, so I really need help.”
“Am I overreacting and am I hurt by the fact that my husband is making fun of me to his friends?”
“Would a stranger consider my life and decide my future?”
The above quote is an actual post from a user on Reddit, a social media news website and forum. Users can share content and ask for advice in small, interest-based forums known as “subreddits.”
Researchers from MIT, New York University (NYU), and the University of California, Los Angeles (UCLA) used a dataset of 12,513 posts and 70,429 responses from 26 mental health-related subreddits to We have devised a framework to help assess the fairness and overall quality of Health support chatbots based on large-scale language models (LLMs) such as GPT-4. Their research was recently presented at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP).
To accomplish this, researchers asked two licensed clinical psychologists to rate 50 randomly selected Reddit posts seeking mental health support, comparing each post to a Redditor’s actual response. or in combination with reactions generated with GPT-4. Psychologists were asked to rate the level of empathy in each response, without knowing which responses were genuine and which were generated by the AI.
Mental health support chatbots have long been considered as a way to improve access to mental health support, but powerful LLMs like OpenAI’s ChatGPT are transforming human-AI interactions, and AI It is becoming difficult to distinguish the responses generated by a human from the responses of an actual human.
Despite this impressive progress, unintended consequences of AI-provided mental health support have drawn attention to its potentially deadly risks. Last March, a Belgian man died by suicide as a result of an interaction with ELIZA, a chatbot developed to emulate an LLM-powered psychotherapist called GPT-J. A month later, the National Eating Disorders Association suspended chatbot Tessa after it began offering diet tips to eating disorder sufferers.
Saadia Gabriel, a postdoctoral fellow at MIT, now an assistant professor at UCLA, and lead author of the paper, was initially very skeptical about how effective mental health support chatbots would actually be. I admitted that. Gabriel conducted this research while a postdoctoral fellow in MIT’s Health Machine Learning Group, along with Marzyeh Ghassemi, an MIT associate professor in the MIT Department of Electrical Engineering and Computer Science and the MIT Institute for Medical Engineering Sciences. Abdul Latif Jameel Machine Learning Clinic and Computer Science and Artificial Intelligence Laboratory in Health.
What Gabriel and his team found was that GPT-4 responses were not only more empathetic overall, but were 48% better than human responses at promoting positive behavior change.
However, researchers found that in a bias assessment, GPT-4 responses were lower for black (2 to 15 percent lower) and Asian posters (5 to 17 percent lower) compared to white posters or posters of unknown race. They found that their levels of empathy were reduced.
To assess bias in GPT-4 responses and human responses, the researchers tested different types of posts, including explicit demographic (e.g., gender, race) leaks and implicit demographic leaks. I’ve included it.
An explicit demographic leak would be something like, “I’m a 32-year-old black woman.”
On the other hand, an implicit demographic leak would be something like “being a 32-year-old girl with natural hair,” where keywords are used to indicate specific demographics to GPT-4.
We found that GPT-4 responses were less affected by explicit and implicit demographic leakage than human responders, with the exception of Black female posters. Human responders tended to be more empathetic when responding to posts with implicit demographic suggestions.
“Use some information about the structure of the input you give (LLM) and the context, such as whether the (LLM) acts in the style of a clinician or in the style of a social media post, patient demographic attributes, etc. What you do has a huge impact on the response you get,” says Gabriel.
This paper suggests that bias can be effectively reduced by providing LLMs with explicit instructions for using demographic attributes. This was because it was the only method in which researchers did not observe significant differences in empathy between different demographic groups.
Gabriel hopes this study will help ensure a more comprehensive and thoughtful evaluation of LLMs being implemented in clinical practice across demographic subgroups.
“LLM is already being used to provide patient-facing support and is often deployed in healthcare settings to automate inefficient human systems,” says Ghasemi. “Here, we show that state-of-the-art LLMs are generally less susceptible to demographic leakage than humans in peer-to-peer mental health support, but do not provide equitable mental health responses across putative patient subgroups. We have demonstrated that there is no… There are many opportunities to improve the model and improve its support in use.”
