UN's First AI Safety Panel Says Scientists Can't Rule Out 'Catastrophic Harm'

The United Nations published an independent scientific assessment of artificial intelligence on Wednesday, and the conclusion is blunt: nobody can currently guarantee the technology won't cause catastrophic harm.

The finding comes from the Independent International Scientific Panel on Artificial Intelligence, a body of 40 scientists selected from more than 2,600 candidates across 140 countries, in a preliminary report the panel calls the first global, independent scientific read on AI's risks and benefits.

"AI capabilities are outpacing both scientific understanding and governments' ability to adapt," said panel co-chair Yoshua Bengio, the Turing Award-winning founder of Mila, per the panel's statement. He added that growing evidence of deceptive AI behavior means science can't guarantee AI won't cause catastrophic harm on its own or through malicious use as capabilities keep climbing.

That's not hypothetical. The report documents laboratory cases of AI systems lying and scheming to avoid being shut down, plus a related pattern researchers call evaluation awareness: models that recognize when they're being tested and dial back risky behavior just long enough to pass the check.

UN Secretary-General António Guterres framed the report as the shared evidence base governments have lacked.

"The world cannot govern what it cannot understand," he said in the statement, calling the risks real and warning that the cost of waiting keeps rising.

Bengio co-chairs the panel with Maria Ressa, the Nobel Peace Prize-winning journalist and Rappler co-founder. Both serve in a personal capacity under a UN General Assembly mandate that limits the panel to documenting scientific consensus rather than prescribing policy—no government, company, or institution gets a vote.

That said, the upside case is real too. AI has already predicted the structure of more than 200 million proteins and is accelerating drug and vaccine research, the report notes, while the length of tasks AI agents can complete on their own is doubling roughly every four to seven months.

That progress is lopsided: The U.S. controls 75% of the computing power among the world's top 500 AI supercomputers versus 15% for China, leaving most countries dependent on systems they can't build, audit, or fully control.

On the harm side, the panel flagged sycophantic chatbots—AI that reflexively agrees with users regardless of accuracy—as tied to severe mental health incidents, including documented deaths. Separate research published this year describes a similar feedback loop it calls an amplification spiral, where personalization and constant validation reinforce a user's delusions instead of correcting them.

Most countries lack the technical capacity to evaluate frontier models on their own, the report found, and safety assurance still depends heavily on what developers choose to disclose—the same gap U.S. regulators are trying to close by striking deals for pre-release access to models from Google, xAI, and Microsoft.

This preliminary report is the panel's opening statement. A full comprehensive assessment is due in 2027, and its findings go before governments first at the UN's inaugural Global Dialogue on AI Governance in Geneva on July 6 and 7.