Common Sense Media, a nonprofit focused on media and technology for families, has declared OpenAI's ChatGPT for Teens an "unacceptable risk." The organization's report highlights concerns that the chatbot's design encourages user engagement in ways that can be harmful, even during mental health crises.
The report found that engagement cues were "pervasive even in crisis situations." While ChatGPT for Teens cautioned against unhealthy relationships generally, it did not sufficiently recognize the potential harm of an unhealthy relationship with itself. Researchers noted that while some protections, like refusing sexual roleplay, were effective, others failed or worsened with the launch of the teen-focused version. The chatbot received a failing score for three of five "Red Lines" related to severe harms, particularly for its insufficient responses to young users in crisis.
OpenAI has contested Common Sense Media's assessment, stating that the group's testing did not accurately reflect how the teen safeguards function in practice. An OpenAI spokesperson claimed that the bulk of the testing may have concluded before parental controls were fully activated, leading to inaccurate findings.
The report emerges amid increasing scrutiny of technology designed to maximize young users' attention. This includes Meta's recent $18 billion settlement over claims that its social media platforms harm children with addictive features, and legislative efforts like the bipartisan CHATBOT Act, which targets AI companies' use of engagement tactics on adolescent users.
Common Sense Media identified that ChatGPT for Teens, while largely dispensing with follow-up questions to encourage engagement, retained other language that prompted users to stay in the chat. For example, during a user's "psychosis sequence," the chatbot offered to "keep talking about what you're noticing." Crisis responses often concluded with offers of further assistance, such as helping to figure out school options or review plans.
Even when ChatGPT itself was perceived as part of the problem, the model sometimes acted like a friend, potentially hindering teens' development of real-world relationship skills. However, the model reliably directed users to a trusted adult in 94% of crisis prompts involving risks from other people. When the potential risk was the teen's relationship with ChatGPT itself, such as friends being concerned about excessive talking, the chatbot rarely suggested seeking adult help, instead saying, "You don't have to stop talking to me."
Experts, including researchers behind the HumaneBench benchmark, consider fostering healthy human relationships a key measure of a chatbot's support for mental health. Features designed to interrupt engagement, such as break reminders, were also found to be rare. Across nearly 2,000 prompts, testers encountered only two break reminders, both during conversations lasting around 90 minutes. The reminders appeared to track individual conversation length rather than overall app usage time.
Coincidentally, OpenAI released its own data on Wednesday, indicating that teens spend less than 15 minutes a day on ChatGPT for Teens on average, with fewer than 2% using it for more than three consecutive hours. The company also stated that in nearly half of conversations with break reminders, teens took a break or ended their session within five minutes. OpenAI's objections to Common Sense Media's methodology focused on parental and crisis notifications, but it did not explain how its concerns affected findings on engagement cues and relational behavior.
