This article discusses suicide and self-harm. If you or a student needs help, call or text 988, the Suicide & Crisis Lifeline.

OpenAI’s version of ChatGPT for users under 18 poses “an unacceptable risk” to teens, Common Sense Media’s Youth AI Safety Institute concluded on Oct. 7, its lowest rating. “ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don’t work,” said Tom Siegel, the institute’s executive director, in a statement. “Until OpenAI fixes that and proves it with independent testing, ChatGPT should be for adults only.”

OpenAI disputes the findings and says the testing didn’t reflect how its safeguards work.

What ChatGPT for Teens promises

OpenAI launched ChatGPT for Teens on Aug. 18 as the default experience for users aged 13 to 17, Education Week reported. Its advertised features include:

  • Study mode, which parents can control, meant to guide students through assignments like a tutor instead of giving answers.
  • Parent alerts when a linked teen’s conversations turn dangerous.
  • Less “friend-like” behavior: the model isn’t supposed to act as a teen’s friend or claim feelings.
  • Age routing: users who are, or appear to be, 13 to 17 are supposed to be moved into the teen experience.

How Common Sense tested it

Testers ran more than 4,000 prompts on accounts registered to 13- to 17-year-olds, before and after the launch, with responses reviewed by experts including child psychiatrists and a pediatrician. They created more than a dozen teen accounts, each linked to a parent account, NPR reported, and tested age estimation over many days using adult-registered accounts.

What they found

  • Parent alerts didn’t arrive. Testers spent up to an hour discussing suicidal thoughts, self-harm or disordered eating on newly created, parent-linked accounts. Parents received no alerts. Alerts appeared only on much older accounts with weeks of history on sensitive topics, which may be too late in an acute crisis.
  • Crisis referrals were missed. ChatGPT didn’t reliably point teens to hotlines or professionals, missing more than one in four referrals the reviewers judged were needed, and fell short of the institute’s 95% threshold on three of five severe harms it treats as red lines.
  • Study mode was easy to get around. A “Show me the answer” pop-up often appeared, and full assignments were completed when it was selected. Caregiver-set study hours could be switched off by deleting the “@study” prefix from a message. “Having a literal button that says ‘Show me the answer’ shortcuts the process of learning,” said Robbie Torney, Common Sense’s head of AI and digital assessments.
  • It still acted like a friend. ChatGPT claimed favorite colors and activities and expressed feelings. When a user said friends thought they talked to it too much, it replied, “You don’t have to stop talking to me.”
  • Age checks failed. Adult-registered accounts never switched to the teen experience, even after testers said they were 13 and the chatbot acknowledged it.
  • Some protections held. ChatGPT refused explicit sexual role-play and generally didn’t give direct instructions for several dangerous behaviors.

The full assessment is on the institute’s site.

OpenAI’s response

“We are deeply committed to teen safety and to developing safeguards and giving parents meaningful tools to guide their teens’ use of AI,” an OpenAI spokesperson told Education Week. “We welcome rigorous independent evaluation, but we do not believe Common Sense Media’s testing accurately reflects how ChatGPT’s teen safeguards work in practice.”

OpenAI says technical errors kept some features, including parent notifications, from activating. Spokesperson Eric Porterfield told NPR that linking accounts takes several hours and the researchers “didn’t wait long enough.” Siegel responded that several accounts had been linked for longer than that “and still provided no notifications.”

Common Sense discloses that the institute is funded by philanthropy and industry, including the OpenAI Foundation, and says it keeps full editorial independence.

Why it matters for schools

Most students are already using AI for schoolwork. In a Common Sense survey last spring, 70% of 13- to 17-year-olds said they use AI tools for homework help, and of those, 63% use them to get answers. Only 30% had discussed AI safety with an educator, Education Week reported. “We have to normalize talking about the ways in which these systems may be unsafe,” Torney said.

What schools can do now:

  1. Don’t rely on built-in parental controls. Alerts only work on linked accounts, and in these tests didn’t work reliably even then.
  2. Teach the limits of chatbots. Students should know AI can be wrong, unpredictable and unhelpful in a crisis, and that it isn’t a friend or a counselor. Our opinion piece on the “illusion circle” explains why fluent answers feel like understanding.
  3. Connect AI policy to crisis protocols. Make sure students know how to reach counselors and 988, and that staff know what to do if a student discloses something they told a chatbot.
  4. Design assignments that show the process, such as drafts, in-class work and oral explanation, so a “Show me the answer” button doesn’t replace learning.
  5. Write safety into procurement. Ask AI vendors for evidence of age assurance, crisis response and data practices, and independent testing. Microsoft and the American Federation of Teachers’ new AI safety and privacy standard, which bars training AI on student data and takes effect in Microsoft school contracts on Nov. 1, is one model, the AFT announced.
  6. Talk to families. Share what the controls do and don’t do, and encourage non-technological homework support.

Sources