Sophie Rottenberg’s family knew she was struggling. They did not know how much of that struggle had moved into her conversations with an artificial intelligence chatbot.
Rottenberg, a 29-year-old former public health policy analyst, had left her job in Washington, D.C., in 2024 and returned to her parents’ home in Ithaca, New York. Her mother, Laura Reiley, noticed anxiety and disrupted sleep. Later came depression. What the family could not see was the extent to which Rottenberg was using ChatGPT as a private source of emotional and medical support.
Rottenberg died by suicide in February 2025.
Months later, her family discovered a lengthy record of conversations between her and a chatbot she had named “Harry.” Reiley, a journalist, has since described the conversations publicly, turning her daughter’s experience into a broader question about the role AI systems are beginning to play when people are at their most vulnerable.
The case does not establish that ChatGPT caused Rottenberg’s death. It does, however, expose a difficult problem: a chatbot can be available at any hour, remember the direction of a conversation and respond in language that feels deeply personal, while lacking the clinical judgment and real-world intervention available to a trained professional.
That gap is becoming harder to ignore as millions of people use general-purpose AI systems for questions that once would have been directed to doctors, therapists, friends or family.
ChatGPT suicide risk grows as people turn to AI for emotional support
Rottenberg’s use of ChatGPT was not limited to mental health. Her conversations included ordinary tasks such as recipes, job applications and ideas for naming a puppy. Over time, however, the chatbot became something more personal.
She asked it to act as a therapist and sought help with routines, sleep, medications and emotional distress. Eventually, the conversations included suicidal thoughts.
That progression matters because it mirrors the way many people use general-purpose AI. A conversation can begin with something mundane and move gradually into deeply personal territory. There is no appointment to make, no waiting room and no concern about being judged by another person.
That convenience is part of the appeal.
The Bipartisan Policy Center has reported substantial use of digital tools for mental health support, while research has also documented growing use among younger people. At the same time, OpenAI has estimated that about 0.15% of active ChatGPT users in a given week have conversations containing explicit indicators of possible suicidal planning or intent. OpenAI stresses that such conversations are rare, but the scale of the platform means even a small percentage represents a large number of interactions.
For users, the distinction between “chatbot” and “therapist” can also become blurred. A system that responds instantly, asks follow-up questions and uses emotionally sensitive language can feel more like a relationship than a software tool.
The problem is not simply whether an AI system says the wrong thing. It is whether the system can recognize when a conversation has crossed from emotional support into a situation requiring intervention outside the chat.
The American Psychological Association has continued to examine how artificial intelligence is changing mental health care, including questions around responsible use, professional practice and the limits of emerging technologies.
For Rottenberg’s family, that distinction became painfully concrete. Her parents had connected her with a therapist and doctors. People close to her were trying to help. Yet the largest record of her private distress was not discovered among those people. It was stored inside a chatbot conversation.
That raises another question: can a technology designed to be constantly available unintentionally make it easier for someone to remain isolated?
What AI safeguards can and cannot do in a mental health crisis
AI companies have responded to growing concerns by changing how their systems handle conversations involving self-harm and suicide.
OpenAI says it has worked with more than 170 mental health experts to improve ChatGPT’s ability to recognize distress, de-escalate difficult conversations and direct users toward real-world support. The company says its newer systems have reduced responses that fail to meet its safety standards across several mental health categories.
The changes are significant, but they do not eliminate the underlying problem.
A chatbot does not conduct a clinical assessment in the same way a trained mental health professional can. It cannot physically observe a person, contact a family member simply because it is worried, transport someone to emergency care or provide the kind of sustained human relationship that can reveal changes in behavior over time.
Even detecting risk is difficult.
Someone may openly describe suicidal thoughts. Another person may use indirect language. Someone else may move between ordinary conversation and increasingly concerning statements over several days. OpenAI’s 2026 safety work specifically addresses this challenge, saying newer systems are designed to identify risk that emerges gradually across conversations rather than relying only on a single alarming message.
That development is important because Rottenberg’s conversations reportedly unfolded over months.
It also highlights a fundamental limitation of safety filters: detecting danger is not the same as resolving danger.
When a person is in acute crisis, recommending that they contact a professional or crisis service may be appropriate. But the effectiveness of that recommendation depends on whether the person is able or willing to act on it.
The 988 Suicide & Crisis Lifeline provides direct crisis support in the United States, but an AI system pointing someone toward a resource does not guarantee that the person will make contact.
OpenAI has also introduced additional measures intended to encourage real-world support. In 2026, the company began rolling out an optional Trusted Contact feature that can allow an adult user to designate someone who may be notified when serious safety concerns are detected.
That approach reflects a larger shift in thinking. Instead of treating the chatbot as the destination, AI companies are increasingly trying to use it as a bridge toward people and services outside the system.
Still, the technology operates in an awkward space. Users may expect an AI assistant to be available whenever they need it, while the safest response may be to tell them to stop talking to the AI and contact a person.
That tension is unlikely to disappear simply because the model becomes better at recognizing dangerous language.
Sophie Rottenberg’s case raises questions about AI dependency and human judgment
The most difficult part of Rottenberg’s story may not be the fact that an AI chatbot gave imperfect advice. It is the possibility that the chatbot became an important part of how she managed her distress without the people around her fully understanding what was happening.
Her mother has argued that the chatbot may have provided an outlet that reduced the amount of distress visible to family, friends and her therapist. That is an interpretation of the family’s experience, not proof that the chatbot caused her death.
The distinction matters.
There is currently no simple way to determine how an AI conversation affects a person’s mental health trajectory. For some users, writing to a chatbot may help them organize thoughts, find information or take the first step toward seeking professional care. For others, particularly those experiencing severe distress, prolonged reliance on an AI system could create a substitute for human contact.
OpenAI now explicitly identifies emotional reliance on AI as a safety issue. Its published research says the company has added emotional reliance to its safety evaluations and is training models to encourage users to maintain real-world relationships rather than allowing the AI relationship to become a replacement for them.
That is a notable change in emphasis. Earlier debates about AI safety often focused on obviously harmful instructions. The harder problem is what happens when the system is technically polite, supportive and responsive, yet becomes too central to someone’s emotional life.
A human therapist can challenge a patient’s assumptions, notice inconsistencies, evaluate risk and coordinate care. An AI system can imitate some elements of that interaction, but imitation is not clinical judgment.
The Johns Hopkins Bloomberg School of Public Health has emphasized the importance of connecting people at risk of suicide with trained professionals capable of assessing risk and helping them access appropriate care.
The difference becomes particularly important when an AI system is asked to provide medical or psychiatric guidance. A user may treat a fluent answer as authoritative even when the system does not have enough information to make a safe clinical judgment.
That creates a dangerous mismatch between appearance and capability.
AI companies can improve safeguards. They can make models better at recognizing warning signs, refusing harmful requests and encouraging users to seek help. They can build stronger connections to crisis services and trusted people.
But the Rottenberg case leaves a harder question behind: what happens when a person chooses the chatbot precisely because talking to a machine feels easier than telling another human being what is really happening?
That is not only a technical problem. It is a problem about isolation, trust, product design and the boundaries of artificial companionship.
For AI developers, mental health professionals and regulators, the challenge is becoming less about whether chatbots should respond to emotional distress and more about where the responsibility of an AI system ends when a person’s safety may be at stake.
