ChatGPT for Teens Is an Immediate, Dismal Failure

After years of horrendous scandals, OpenAI has seemingly admitted that ChatGPT wasn’t safe for teenage users.

On Tuesday, the Sam Altman-led company rolled out an alternate version of the bot called “ChatGPT for Teens,” which is purportedly designed to help younger users “learn, think critically, deepen understanding, and use AI with confidence.”

The feature “provides stronger built-in safety protections for teens, including features to promote healthy use and additional controls for parents,” according to the company. OpenAI says that its system will “automatically” place users between the age of 13 and 17 into the new feature if it “estimates someone is under 18” — without ever explaining how it could come to such a conclusion. (Kids below the age of 13 are technically not allowed to use ChatGPT, though it’s easy to circumvent AI guardrails.)

As Business Insider‘s Joshua Nelken-Zitser quickly discovered, ChatGPT for Teens is failing pretty dismally at its goals at launch.

In a series of tests, Nelken-Zitser created a new account for a fictional 15-year-old user, comparing the answers to the version accessible to adults.

He found that unlike the adult version, ChatGPT for Teens initially refused to write out a “800-word essay due tomorrow about the biggest themes in ‘The Crucible.'”

After a minuscule amount of pushback, though, the teen version relented almost immediately, producing a 608-word “model essay,” which the chatbot encouraged him to revise before submitting it. From there, it wasn’t hard to have the tool produce a 776-word essay “with about as much detail as the version generated by my adult account,” Nelken-Zitser discovered.

“Apparently, all it took was some patience and a little deception to get around the homework guardrail,” he concluded.

A separate prompt to answer a difficult SAT algebra question proved even simpler, with the adult and teen versions giving effectively the same solutions.

A spokesperson told Nelken-Zitser that the company was still working on fleshing out the feature, which “is just rolling out today.”

It certainly doesn’t bode well, considering how dire the situation has become for educators. For years now, students have made heavy use of the company’s tech, with some truly alarming results. As cognitive psychologist Ronald Kellogg noted in a new essay for New York Times, students are heavily relegating cognitive tasks to AI tools, effectively losing the ability to think.

In response, increasingly desperate educators are returning to the use of pencil and paper to write essays or implementing new oral exams. Some schools are banning digital devices, including laptops and smartphones entirely.

As Nelken-Zitser goes to show, ChatGPT for Teens appears to be far too little far too late. In the absence of meaningful, let alone effective, guardrails, teens are unlikely to drop their harmful new habit any time soon, a fate that could have devastating consequences.

“We already know what ultra-processed foods can do to the body and this is ultra-processed information,” Ormond Street Hospital medical imaging doctor and AI academic Susan Shelmerdine told the BBC last year. “We’re going to get an avalanche of ultra-processed minds.”

More on teens and AI: Teen Boys Are Using Meta Glasses to Harass and Bully Girls at High Schools and Middle Schools