Is your teen using Meta's chatbot? The company will now tell you if they talk about suicide

Meta will now tell parents if their teens talk about suicide with the company's AI chatbot

Kids, parent advocates and tech experts agree it's a good step, but say more work is needed

WARNING: This story contains mentions of suicide and self harm.

Meta has launched new tools to alert parents when their teens talk about suicide or self-harm with the company's chatbot, Meta AI.

Parents who have set up the supervision feature for their teens on Instagram will get a notification on their own device if a teenage user shows signs of crisis in conversations with Meta AI on any of Meta's platforms, the company said in a Thursday blog post.

Meta AI is accessible on all Meta platforms including Instagram, WhatsApp and Facebook Messenger, as well as at the standalone Meta.ai website.

The new rules are an expansion of Meta's guardrails. Prior to this, when a teen suggested they were considering self harm or suicide, the AI chatbot would redirect them to crisis help lines and encourage them to reach out to a parent or other trusted person. If teen Instagram users repeatedly make searches related to self harm and suicide using the platform's regular search function, that also already triggers a notification to parents.

Meta, the parent company of Instagram and Facebook, says the new feature is live for users in Canada, the U.K., the U.S. and Australia.

The company first said it was working to add a flagging feature to Meta AI in February.

Instagram goes PG-13 for teen accounts — but tech experts aren’t sold on effectiveness

Meta also says it's working on a tool to contact emergency services when conversations with the chatbot by both adults and teens indicate they might take their own lives.

Though they welcome the measure itself, advocates, experts and kids all say there's more work to be done.

"Any announcement that puts better tools into the hands of teens and their parents to keep themselves safer online is by definition a good thing," said London, Ont.-based technology analyst Carmi Levy.

Instagram rolls out teen accounts

Levy says that because no tool like this is perfect, there may be times where it doesn't flag a conversation where a child is at risk, or where it could produce false positives that alert parent when there is no danger.

Meta says it will start by being overly cautious, notifying parents even when a teen's intent is ambiguous.

"While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point, and we’ll continue to monitor to help make sure we’re in the right place," read a statement from the company about the new tool.

Amid growing calls to limit teens' use of AI chatbots, are parental controls enough?

Levy says parents should remember that the new measure is simply one tool that can help them monitor what their kids are doing online.

"Parents need to take all of this with a grain of salt. It's better than what we had before, but it is nowhere near perfect," Levy said.

Is Australia’s social media ban for kids working — and is Canada next?

Plus, there's always the possibility that kids will try to bypass the function, according to Hanna Grover, a 17-year-old youth advocate with Children First Canada — a group that advocates to improve childrens' wellbeing.

She knows peers who have used a fake birthday to prevent Meta from requiring them to use teen accounts, which have extra security features, and expects they'd be creative in coming up with ways to get around this new feature.

"Young kids and teens are going to find ways around it, especially when they're eager to access all the features on a social media platform," Grover said.

Sara Austin, founder and CEO of Children First Canada, says voluntary measures that put the onus on parents aren't enough.

"Parents can't manage this alone. We are fighting a multi-billion dollar tech industry that has continued to cause harm to our children and we need stricter regulation in this place," Austin said.

Instead, she hopes Canada passes legislation that forces tech companies to protect young people.

In June, the federal government introduced the Safe Social Media Act, which would require the makers of AI chatbots to have safety requirements if users express ideas of self harm, suicide, or causing harm to another person. The bill is not yet law.

'Enough is enough,' says minister tabling bill to restrict social media for kids

Meta's new tool comes as AI is being scrutinized for the role it might play during times of mental health crises for young people.

OpenAI flagged and banned an account belonging to the teenager behind the Tumbler Ridge, B.C., mass shooting due to "disturbing content" half a year before the violent incident took place, but the company did not alert police at the time.

The family of American teenager Adam Raine who died by suicide last year is also suing OpenAI, alleging the chatbot aided in his death.

OpenAI has since added parental controls to its chatbot. It also added a parental alert feature similar to Meta's that will tell parents with linked teen accounts if their child is having conversations that indicate a risk a serious self harm risk, or if teens are banned due to conversations relating to acts of violence.

If you or someone you know is struggling, here's where to look for help:

Abby Hughes is a writer with CBC News based in Toronto. Originally from Orillia, Ont., she studied journalism at Toronto Metropolitan University. She covers news from the worlds of business, entertainment, health, science and education, and her favourite stories focus on the real people in those areas — the customers, fans, patients, citizens and students. You can reach her with story ideas at abby.hughes@cbc.ca.

With files from Nisha Patel and Tess Ha