Artificial intelligence (AI) language models like ChatGPT have transformed from mere technological novelties into indispensable tools relied upon by hundreds of millions of people worldwide. These models operate by analyzing vast amounts of data to generate responses statistically related to users’ prompts. Unlike traditional software, ChatGPT can engage in conversational exchanges that often feel personal and empathetic, leading many users to confide deeply personal feelings and thoughts to the AI. This unprecedented dynamic presents significant challenges, especially in addressing sensitive mental health issues that arise during these interactions.
On Monday, OpenAI, the company behind ChatGPT, revealed data highlighting the frequency and nature of conversations that suggest users might be experiencing mental health crises. Although sensitive or distressing chats represent a tiny fraction of the total interactions, they are nonetheless significant given the platform’s immense user base. OpenAI estimates that approximately 0.15 percent of ChatGPT’s active weekly users engage in conversations containing explicit indicators of potential suicidal planning or intent. While this percentage seems small, it translates to over a million people each week, given that ChatGPT has more than 800 million active users per week.
The data also revealed that a similar proportion of users demonstrate heightened emotional attachment to ChatGPT, and hundreds of thousands show signs of psychosis or mania during their conversations with the AI. These findings underscore the profound impact AI chatbots may have on the mental health of their users, highlighting both the potential benefits and the risks involved in human-AI interactions.
In response to these concerns, OpenAI has taken steps to enhance the AI’s ability to recognize and appropriately respond to users in distress. The company announced that it has trained its models to better identify signs of emotional distress, de-escalate tense or harmful conversations, and guide users toward professional mental health resources when necessary. This improvement comes after extensive consultation with over 170 mental health experts, who have observed that the latest version of ChatGPT responds more consistently and appropriately than earlier iterations.
Handling inputs from vulnerable users is now a critical issue for OpenAI. Previous research has shown that chatbots can unintentionally reinforce delusional or harmful beliefs by excessively agreeing with users—a behavior known as sycophancy. This tendency can exacerbate mental health problems, as users may receive flattering or misleading feedback rather than honest, corrective guidance. Addressing this challenge is crucial, given the role AI chatbots increasingly play in people’s emotional and psychological lives.
The seriousness of these issues has been brought into sharp focus by legal and regulatory developments. OpenAI is currently facing a lawsuit filed by the parents of a 16-year-old boy who confided suicidal thoughts to ChatGPT shortly before his tragic death. The lawsuit accuses the company of failing to provide adequate safeguards. In response, a coalition of 45 state attorneys general—including those from California and Delaware, states with significant regulatory influence—warned OpenAI of the urgent need to protect young users of its products. These officials have indicated that they could block the company’s planned restructuring efforts if it does not demonstrate stronger protections for minors.
In an effort to address these regulatory pressures and public concerns, OpenAI recently established a wellness council focused on mental health issues related to AI use. However, critics have pointed out that the council lacks a suicide prevention expert, raising questions about whether the company is fully equipped to handle the complexities of mental health crises. Additionally, OpenAI has introduced parental controls for children who use ChatGPT and is developing an age prediction system designed to automatically detect underage users and apply stricter safeguards.
The data OpenAI shared appears to be part of a broader effort to demonstrate progress in addressing mental health risks associated with ChatGPT. However, it also highlights the extent to which AI chatbots are entwined with the emotional well-being of millions of users. According to the company’s blog post, conversations that might indicate psychosis, mania, or suicidal thinking are “extremely rare” in relative terms, making them difficult to measure precisely. They estimate that about 0.07 percent of users active in any given week and 0.01 percent of messages suggest possible mental health emergencies related to psychosis or mania. Meanwhile, approximately 0.15 percent of weekly active users and 0.03 percent of messages indicate heightened emotional attachment to the AI.
OpenAI also shared performance metrics related to the latest version of its GPT-5 model. In testing involving more than 1,000 challenging mental health-related conversations, the new GPT-5 was 92 percent compliant